Computer Science includes research into networking. TCP incast is a way of describing network traffic statistical properties, particularly in data centers [1]<p>A typical CS student's view of the network is that "everything is so random that it averages out." The exact opposite is actually the case most often, to the chagrin of your local network admin.<p>Network traffic tends to act like everyone knew you were going to youtube right now, and everybody jumped on the network all at once. The statistical term is "self-similarity" and the Hurst Parameter (H) measures how badly a network's traffic is _not_ averaging out, but bursting as if everyone knew you were going to youtube.<p>It might help to mention another place we see self-similarity: fractals.<p>This article just breaks down the situation in a cluster. Again, self-similar traffic patterns mean everyone tries to talk at once, and their TCP stacks all back off randomly, so the total bandwidth of the network is poor.<p>Unfortunately, the blog post says, "What’s the remedy? We don’t have a good remedy for this yet."<p>Sure we do. Please google some of the relevant terms for great articles on network traffic analysis and optimization. For example "self similar network traffic" and "hurst parameter." Even the CMU site linked from the blog post has a great writeup under the section "SOLUTIONS" :) [2]<p>Additionally, as a comment on the blog points out, larger buffers on routers can be a really _bad_ thing! Buffer bloat tends to hide core issues with larger latencies, but not solve them.<p>[1] <a href="http://en.wikipedia.org/wiki/Long-tail_traffic" rel="nofollow">http://en.wikipedia.org/wiki/Long-tail_traffic</a><p>[2] <a href="http://www.pdl.cmu.edu/Incast/" rel="nofollow">http://www.pdl.cmu.edu/Incast/</a>
> ... head-of-line blocking ... What’s the remedy?<p>I don't know anything about Erlang or the software they're dealing with, but I'm surprised that "program your software to avoid unnecessary head-of-line blocking" didn't make the list.