TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Bring down a poorly deployed WordPress (and how to stop it from happening)

26 pointsby jaladaover 14 years ago

6 comments

bradleylandover 14 years ago
Our application runs an Apache MPM/Passenger/MySQL stack. We conduct real-time purchasing events with anywhere from 5-25 sellers bidding in an event that may have 200+ line items spread out over several lots. Using the "minimum viable product" philosophy, the bidder interface relies on AJAX polling at a 3 second interval to get updated information (yes, we have plans to move to something different [Socket.io, actually]). This means that our app gets hammered during an event. All of this runs within 1 GB of RAM on our VPS, and that includes 6 Passenger instances and MaxClients set to 150 (GASP!). Here's a sample of our passenger-memory-stats output during a large-ish event:<p><pre><code> --------- Apache processes ---------- PID PPID VMSize Private Name ------------------------------------- 480 21168 17.6 MB 0.5 MB /usr/sbin/apache2 -k start 1216 21168 17.6 MB 0.6 MB /usr/sbin/apache2 -k start 5252 21168 17.6 MB 0.6 MB /usr/sbin/apache2 -k start 5340 21168 17.6 MB 0.6 MB /usr/sbin/apache2 -k start 5992 21168 17.6 MB 0.5 MB /usr/sbin/apache2 -k start [REMOVED TO SAVE SPACE] 17163 21168 17.6 MB 0.5 MB /usr/sbin/apache2 -k start 21168 1 17.3 MB 0.4 MB /usr/sbin/apache2 -k start ### Processes: 89 ### Total private dirty RSS: 42.10 MB -------- Nginx processes -------- ### Processes: 0 ### Total private dirty RSS: 0.00 MB ---- Passenger processes ----- PID VMSize Private Name ------------------------------ 5997 55.2 MB 29.6 MB Rack: /var/www/redacted/current 6006 60.8 MB 34.5 MB Rack: /var/www/redacted/current 14262 63.4 MB 50.1 MB Rack: /var/www/redacted/current 20205 63.1 MB 49.7 MB Rack: /var/www/redacted/current 25250 5.1 MB 0.2 MB PassengerWatchdog 25253 14.1 MB 0.6 MB PassengerHelperAgent 25255 10.6 MB 4.8 MB Passenger spawn server 25259 9.0 MB 0.5 MB PassengerLoggingAgent 25412 63.2 MB 49.9 MB Rack: /var/www/redacted/current 32492 51.8 MB 39.6 MB Rack: /var/www/redacted/current ### Processes: 10 ### Total private dirty RSS: 259.50 MB </code></pre> I don't know what the author's experience has been, but I've seen the recommendation to trim Apache back to 10-15 MaxClients all over the place, and when we took that advice, the results were catastrophic to app performance. We almost fell apart during an event because of slow clients tying up Apache processes. We did plenty of benchmarking prior to going live, but getting a picture of real world performance is a lot harder than running `siege -c 250 <a href="http://hostname`" rel="nofollow">http://hostname`</a> against your server. If you're going to trim Apache back that far, I'd recommend turning off KeepAlive altogether. You want that Apache process free immediately. If you are forced to run a config this tight, it's likely that you can afford the CPU and interrupt overhead of setting up a new TCP/IP session, but you can't afford the extra processes lying around idle, sucking up memory and MaxClient slots. In other words you're heavily memory bound.<p>So let me be clear about this: Trimming MaxClients to 10-15 is <i>horrible</i> advice if you don't understand what your actual memory usage (maybe you can run more) is and the impact of slow clients. I trimmed our Apache modules back to a bare minimum, resulting in an average Apache process size of 0.45 MB. Yes, less than 512 KB per Apache process. We run 'MaxClients' 150 with zero fear of running out of memory. If you're running a PHP based site, your Apache process size is going to be larger, but you had better know what that is before you start tweaking your MaxClients config.<p>I should probably write a blog post about tweaking Apache settings, because I've learned a lot of "hard knocks" lessons, not the least of which is "understand your web server's memory usage", but here are some quick tips in the mean time:<p>* Unload modules you don't need! Saving a couple megabytes of memory across 100 Apache processes means saving 200 MB of memory.<p>* To understand how much memory Apache processes are using, DO NOT rely on `top`. Have a look at the way `passenger-memory-stats` calculates <i>real</i> memory usage [1].<p>* Understand the output of `free -m`, especially the "-/+ buffers/cache" line.<p>* Do not rely on benchmarking tools like siege to tell you the whole story. Slow clients are difficult to account for.<p>* If you have to run a limited number of MaxClients, or you run up against your MaxClients limit frequently, try turning off KeepAlive [2].<p>If you follow those tips, you'll see a dramatic improvement in the accuracy of your Apache config for your environment, because that's what it comes down to. Apache is a fantastic web server, but it has to be configured to fit your environment. I don't have any dislike for nginx, but a lot of the nginx recommendations I see are born out of the inability to properly configure their existing web server. If you don't already know/understand Apache, I wouldn't hesitate to make the jump to nginx. It is also a fantastic web server. Hopefully this has helped someone.<p>1 - <a href="https://github.com/FooBarWidget/passenger/blob/master/bin/passenger-memory-stats" rel="nofollow">https://github.com/FooBarWidget/passenger/blob/master/bin/pa...</a><p>2 - <a href="http://serverfault.com/questions/86550/apache-keep-alive-or-not-keep-alive" rel="nofollow">http://serverfault.com/questions/86550/apache-keep-alive-or-...</a>
cagenutover 14 years ago
How to set MaxClients:<p>1 - restart apache<p>2 - run a handful of typical requests through it a few times (or just leave it live for a minute)<p>3 - look for the largest few workers by the RSS column in "ps"<p>4 - average that, divide it into the amount of memory you consider it acceptable for apache to use (64mb per worker, 512mb available for apache, therefore 8)<p>voila, thats your maxclients. If its greater than 256, congrats, you have more ram than you need[1], find something else to do with it.<p>the real trick here is when you realize "oh thats the exact same math I have to do with fastcgi worker processes in nginx" and realize its all the same.<p>[1] this statement will become untrue somewhere around 2013 when 128-core servers become cheap and standard
fat_tonyover 14 years ago
There is an even easier way. Use nginx.
评论 #2183001 未加载
bryanhover 14 years ago
Why no mention of standard caching and turning off KeepAlive during an onslaught?
ICanDoMoreover 14 years ago
What about Varnish?
评论 #2183257 未加载
sabatover 14 years ago
Rather than fuss over Apache settings, use some decent caching strategies and you won't need to worry about this at all. And BTW, nginx is not always faster than Apache, and not always the best choice for any app.
评论 #2183569 未加载
评论 #2183277 未加载
评论 #2183431 未加载