The single best advice I can give for scaling Rails, and it is widely applicable to other frameworks: never block an HTTP request/response cycle on I/O. In Rails, for example, if you have an external API call which routinely takes 3 seconds to complete, then that call costs about three hundred megabyte-seconds of RAM time to service, because the 100 MB or so used by your mongrel is effectively worthless while you wait.<p>When you run into resource constraints in Rails, odds are you are running out of RAM first. It's a memory munching beast in most common deployment scenarios.<p>It is much more efficient to offload those blocking requests to a worker process and poll for results, since a single mongrel can blow through <i>lots</i> of poll requests <i>and</i> useful work in five seconds. Bonus points: your web app will feel snappier, because progressive rendering tricks users into thinking 5 seconds is not actually 5 seconds.
Here's few other ways to optimize your rails application (with the description taken from the doc) :<p>- <a href="https://github.com/flyerhzm/bullet/" rel="nofollow">https://github.com/flyerhzm/bullet/</a> : The Bullet plugin/gem is designed to help you increase your application’s performance by reducing the number of queries it makes. It will watch your queries while you develop your application and notify you when you should add eager loading (N+1 queries), when you’re using eager loading that isn’t necessary and when you should use counter cache.<p>- <a href="https://github.com/sdsykes/slim_scrooge/" rel="nofollow">https://github.com/sdsykes/slim_scrooge/</a> : SlimScrooge implements inline query optimisation, automatically restricting the columns fetched based on what was used during previous passes through the same part of your code.<p>- <a href="http://guides.rubyonrails.org/caching_with_rails.html" rel="nofollow">http://guides.rubyonrails.org/caching_with_rails.html</a> : Caching With Rails from the awesome rails guides<p>- <a href="http://guides.rubyonrails.org/performance_testing.html" rel="nofollow">http://guides.rubyonrails.org/performance_testing.html</a> : Performance Testing Rails Applications from the rails guides
Not to be obnoxious but I wasn't wasn't very impressed by this post ... use explain plans to add indexes to your tables? use passenger instead of mongrel instances? don't use rand() because it does a table scan? these are things a developer should know and be doing already.<p>There are also mistakes in there like saying you cannot add indexes to your tables using rails (can do this using migrations).
We did similar things on our server (though with much much less traffic but all dynamic content + API).<p>We moved from passenger to nginx+unicorn and the performance increase was awesome + it uses less ram (which again means more ram for more app servers!). Our box was a old quad core with 4gb ram so it got a bit tight with everything on one machine. The DB blocking problem could easily be solved with a cheap VPS as a slave DB, then you can do backups off that. :-) Depending on your kind of website, caching with varnish is great, we also made great use of memcached for fragments and other little things. And since our main cpu consuming background task just did one thing in the end it was rewritten in C which also helped a lot. Offloading all files to Amazon S3 is also a very good way of reducing load if you have a lot of them.
I had an article published about MySQL tuning a while ago:<p><a href="http://blog.scoutapp.com/articles/2011/02/10/understanding-disk-i-o-when-should-you-be-worried" rel="nofollow">http://blog.scoutapp.com/articles/2011/02/10/understanding-d...</a><p>Always tune for your workload. Be sure to index your damn tables. And yes, measure.
Even if you aren't "on a budget," it's always a good idea to spend some time benchmarking and tuning your setup. You never know if you'll receive a spike of traffic or what the limits of your setup are. Most of the time, between the OS, Web Server, Application Stack, and Database, there are a few knobs that can be adjusted with minimal effort that will yield big gains in headroom. The author's suggestion of Munin is a great first step is a nice way to keep from getting caught with your pants down, and can usually be installed and configured in a few minutes.<p>This is sort of the systems side of always making sure HTTP compression is working, minifying your JavaScript, crushing your PNGs, and keeping an eye on the aggregate size of web pages. Even if it loads fast for YOU, someone might be on a slow DSL line (or, gasp, DIALUP) or a mobile connection.
As an aside, beware using idiomatic ruby while writing ActiveRecord queries.<p>Model.all.inject { sum or average code } will destroy you once you get into production. Working at an otherwise excellent Rails shop I had to launch a mini-jihad to drive this stuff out.<p>(NB my code is wrong because I haven't written Ruby for a while).
I've faffed around with optimising LAMP for years in order to get Wordpress to perk up. Aside from WP-supercache, the biggest difference I ever made was moving MySQL onto a second server. It <i>halved</i> page generation times and the whole site is much, much snappier.