This can be flipped into a simple heuristic known as the "square root staffing rule" (sometimes "... law"). The name comes from call centres.<p>Basically, the number of servers you need to serve X amount of demand with Y probability of queuing is not linear with X. It is proportional to the square root of X.<p>The intuition is that a call centre agent is either talking to a customer, or they are not. If they are not talking to a customer, then they can immediately serve a customer. The odds that a given agent is talking to a customer is on a probability distribution, usually assumed to be the normal distribution.<p>In a normal distribution, larger samples lead to more and more appearances of outliers in the sample. More agents means more "outliers", in this case, more idle agents. And that curve is not <i>linear</i>, it follows the shape of the normal distribution.<p>Where it sucks for call centres is that demand shows seasonality and shift planning has relatively inflexible lead times. But in a software scenario we can typically acquire additional capacity quickly, so precision in forecasts is less of a problem.<p>A decent explanation: <a href="https://www.networkpages.nl/the-golden-rule-of-staffing-in-contact-centers/" rel="nofollow">https://www.networkpages.nl/the-golden-rule-of-staffing-in-c...</a><p>A short scenario: <a href="https://www.xaprb.com/blog/square-root-staffing-law/" rel="nofollow">https://www.xaprb.com/blog/square-root-staffing-law/</a><p>A simple calculator linked from the short scenario: <a href="https://www.desmos.com/calculator/8lazp6txab" rel="nofollow">https://www.desmos.com/calculator/8lazp6txab</a>
If we were to extend the first graph a bit more to the right, the linear improvement would quickly trend downwards into negative latencies, a sure sign that it's not the right answer. But if linear is impossible, then super-linear is just as impossible. The line the author describes as such is clearly asymptotic.<p>If we were discussing overall latency, and not just queue delay, then an asymptotic decrease towards zero could correspond to a super-linear increase in throughput capacity. But that's not what's happening here, either: because average latency is bounded below at one second, total throughput can never exceed <i>c</i>. There is no super-linearity to be found here.