These are great tips for load testing any web service. Don't forget that you can also saturate memory, disk or network too, so watch those graphs too. I've seen load tests unable to saturate CPU because another resource was limited.<p>Also don't forget that you are load testing all the dependencies of your service. Database, caching tier, external services, etc. Make sure other teams are aware!<p>Also nothing beats real world traffic. Users' connections will stay open longer than a synthetic tool may hold them open due to bandwidth, they make very random, sporadic requests too. Your service will behave very differently under large amounts of real world traffic vs synthetic.<p>Other options if you are running multiple web servers is to shift traffic around to increase traffic to 1 host and see where it fails. That is usually a very reliable signal for peak load.<p>And don't forget to do this on a schedule as your codebase (and your dependencies codebases) changes!
If you're looking to run programmable load test scenarios as described in this blog post, consider checking out <a href="https://k6.io/" rel="nofollow">https://k6.io/</a>, which I've found to be extremely easy to use and powerful.
> My initial requirement was to send requests with unique parameters. To the best of my knowledge, no tool could do this.<p>wrk does this with lua. <a href="https://github.com/wg/wrk/blob/master/src/wrk.lua">https://github.com/wg/wrk/blob/master/src/wrk.lua</a><p>Also even things like the venerable jmeter supported pulling parameters from a csv file.
If you'd like to write your load tests in Ruby (plain ruby or browser based), and use your own libraries (internal libraries, specs, ets) browserup does that:<p>Ruby with browser:<p><a href="https://browserup.com/docs/en/load/ruby-load-test.html" rel="nofollow">https://browserup.com/docs/en/load/ruby-load-test.html</a><p>Command-line installation:<p><a href="https://www.npmjs.com/package/browserup" rel="nofollow">https://www.npmjs.com/package/browserup</a>
Having your test database mirror production as closely as possible is also an important habit, however I’m biased since that’s part of the offering that I’m building.
> If the application can’t saturate the CPU, there’s a fundamental problem. It’s a shame because it makes adding more servers less efficient. Money is being wasted on hosting costs, and this should be a priority to address.<p>Talking about efficiency being a priority, but using RoR. I guess that is one way of saturating the CPU.
fwiw, I know it's the right thing to run tests from a different computer. But it's more annoying. And I hereby tell you that 90% of the time it probably doesn't matter.<p>Definitely times it isn't true. But if you're not doing a load test bc it's a pita, do it locally. Most of the time I've wanted to do this, all the action is inside the app. Just be careful to acknowledge that there could be limitations / surprises.