Firstly, read the small print. Many 'cheap' hosting packages are short on features and flexibility, adding banner advertising to your site or locking you into a particular configuration for years.
Secondly, consider what infrastructure can be provided by a company that receives so little revenue from each of its customers. Lower prices mean lower investment, perhaps even insecurity in the business itself. What assurances do you have about the level of service, or the health of the company?
Now apply the same reasoning to customer service. The fee you pay somehow has to cover the people you deal with, as well as the service itself. Are you likely to get phone and email support, proactive troubleshooting and friendly advice whenever you need it? What is actually guaranteed?
If you still feel that rock-bottom hosting is right for your business, there are plenty of economy packages out there. Of course you could choose a professional service that's carefully designed around your actual business needs, not squeezed into an eye-catching price point. It's your decision.