Which is correct: "web host" or "web hoster"?

In casual speech and conversations on the internet, the usual terminology is "web host". If you need a more formal description then you might want to go with "web hosting service". "My web hoster supports Ruby" sounds distinctly odd to me. It might occur in a sentence like

"Well-known web hoster doteasy today announced record profits".

Incidentally one way to compare terms like this is to count the number of Google hits you get with the words in quotes. I get

Web host              2,800,000 hits
Web hosting service   1,000,000 hits
Web hoster               30,000 hits

I've never heard "web hoster" and "web hosting service" seems verbose to me. "Web host" is the generally accepted term, as far as I know.

I normally say Ruby is included in the (web) hosting service for my website; actually, I would say Ruby on Rails is included in the (web) hosting service for my website.

The difference for me is that saying the web server includes Ruby seems to mean it will never be removed; saying the hosting service includes Ruby means that it will be included until they don't change the service.

Of the choices provided, the best way to refer to a company that hosts your website is your third option web hosting service.

Based on a COCA[1] query web hosting 6 times more common that web host, and web hoster yields no hits. BTW, web server is 11 times more common that web host. Personally, I find your third option the best: My web hosting service supports Ruby. I could just as well go with: My web server supports Ruby.

[1] Google is generally accepted to be worthless as a corpus search tool. This is why it is better to use a real corpus. COCA is good for american english and BNC is good for british english.