Combine CSS and JS files and if possible minify them. Try doing this via pagespeed. If that is not possible, use W3 Total Cache for this. Be very careful with this step as it can break your themes and plugins.
Optimize image sizes with lossless image compression. You can do this either using pagespeed or a WordPress plugin like smush.it.
If HHVM doesn’t work with your plugins and themes, then Nginx with PHP-FPM is best.
When it comes to PHP, always use latest PHP version, even though WordPress supports outdated PHP versions (which exposes WordPress sites to security risks)
Add mysql indexes if possible to speed up queries. Many plugin developers don’t take efforts to add proper mysql indexes for MySQL tables there plugin creates.
Never use shared hosting. If your site is small, you won’t be reading this article. As you are reading this, it means your site is big and need to tweaked. Most of stuff here won’t work on shared hosting.
WordPress specific hostings e.g. WP-Engine makes few things easier but they won’t give you full control.
We recommend VPS with SSD hosting. DigitalOcean and Linode are our favorites!
Notes
Of course, after all of above, your site may still be slow. Chances are high that you missed something from above list.
Unfortunately it’s not possible to solve performance problem for any site over our community support forum or skype/email exchanges. rtCamp can only help if it gets first-hand access to the server! You may contact us if for professional support in that case.