TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

How a little bit of TCP knowledge is essential

299 pointsby dar8919over 9 years ago

12 comments

Animatsover 9 years ago
That still irks me. The real problem is not tinygram prevention. It&#x27;s ACK delays, and that stupid fixed timer. They both went into TCP around the same time, but independently. I did tinygram prevention (the Nagle algorithm) and Berkeley did delayed ACKs, both in the early 1980s. The combination of the two is awful. Unfortunately by the time I found about delayed ACKs, I had changed jobs, was out of networking, and doing a product for Autodesk on non-networked PCs.<p>Delayed ACKs are a win only in certain circumstances - mostly character echo for Telnet. (When Berkeley installed delayed ACKs, they were doing a lot of Telnet from terminal concentrators in student terminal rooms to host VAX machines doing the work. For that particular situation, it made sense.) The delayed ACK timer is scaled to expected human response time. A delayed ACK is a bet that the other end will reply to what you just sent almost immediately. Except for some RPC protocols, this is unlikely. So the ACK delay mechanism loses the bet, over and over, delaying the ACK, waiting for a packet on which the ACK can be piggybacked, not getting it, and then sending the ACK, delayed. There&#x27;s nothing in TCP to automatically turn this off. However, Linux (and I think Windows) now have a TCP_QUICKACK socket option. Turn that on unless you have a very unusual application.<p>Turning on TCP_NODELAY has similar effects, but can make throughput worse for small writes. If you write a loop which sends just a few bytes (worst case, one byte) to a socket with &quot;write()&quot;, and the Nagle algorithm is disabled with TCP_NODELAY, each write becomes one IP packet. This increases traffic by a factor of 40, with IP and TCP headers for each payload. Tinygram prevention won&#x27;t let you send a second packet if you have one in flight, unless you have enough data to fill the maximum sized packet. It accumulates bytes for one round trip time, then sends everything in the queue. That&#x27;s almost always what you want. If you have TCP_NODELAY set, you need to be much more aware of buffering and flushing issues.<p>None of this matters for bulk one-way transfers, which is most HTTP today. (I&#x27;ve never looked at the impact of this on the SSL handshake, where it might matter.)<p>Short version: set TCP_QUICKACK. If you find a case where that makes things worse, let me know.<p>John Nagle
评论 #10608751 未加载
评论 #10609792 未加载
评论 #10608392 未加载
评论 #10608988 未加载
barrkelover 9 years ago
This is a general problem of leaky abstractions. If you&#x27;re a top-down thinker, you&#x27;re going to have a bad time some day and have a hard time figuring it out.<p>OTOH bottom up thinkers take much longer to become productive in an environment with novel abstractions.<p>Swings and roundabouts. Top down is probably better in a startup context - it&#x27;s more conducive to broad and shallow generalists. Bottom up is great when you have a breakdown of abstraction through the stack, or when you need a new solution that&#x27;s never been done quite the same way before.
评论 #10608521 未加载
评论 #10608480 未加载
jfbover 9 years ago
I really enjoy reading Julia&#x27;s blog. Not only does she have a real, infectious enthusiasm for learning; not only is the blog well written; but I also often learn a lot. Kudos.
评论 #10608365 未加载
p00bover 9 years ago
John Rauser of pinterest gave a wonderful talk about TCP and the lower bound of Internet latency recently that has a lot in common with what&#x27;s discussed in the article here. Worth a watch I think if you enjoyed the blog post.<p><a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=C8orjQLacTo" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=C8orjQLacTo</a>
PeterWhittakerover 9 years ago
Summary: If you know learn a little, you realize that each packet might be separately acknowledged before the next one is sent. In particular, note this quote: <i>Net::HTTP doesn’t set TCP_NODELAY on the TCP socket it opens, so it waits for acknowledgement of the first packet before sending the second.</i><p>By setting TCP_NODELAY, they removed a series of 40ms delays, vastly improving performance of their web app.
colandermanover 9 years ago
You don&#x27;t need to entirely disable Nagle; just flash TCP_NODELAY on then off immediately after sending a packet for which you will block for a reply. This way you still get the benefit Nagle brings of coalescing small writes, without the downside.<p>(Alternatively, turn Nagle off entirely and buffer writes manually or using MSG_MORE or TCP_CORK.)
dantiberianover 9 years ago
I came across this this week working on the RethinkDB driver for Clojure (<a href="https:&#x2F;&#x2F;github.com&#x2F;apa512&#x2F;clj-rethinkdb&#x2F;pull&#x2F;114" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;apa512&#x2F;clj-rethinkdb&#x2F;pull&#x2F;114</a>). As soon as I saw &quot;40ms&quot; in this story I thought &quot;Nagles Algorithm&quot;.<p>One thing I haven&#x27;t understood fully is that this only seems to be a problem on Linux, Mac OS X didn&#x27;t exhibit this behaviour.
评论 #10609282 未加载
bborehamover 9 years ago
Why wouldn&#x27;t an http client library turn off Nagle&#x27;s algorithm by default?
nedumaover 9 years ago
Can wireshark&#x2F;riverbed (application perf tests) profiling help to solve these kind of problems?
评论 #10608573 未加载
评论 #10608330 未加载
rjurneyover 9 years ago
In highschool I carried TCP Illustrated around with me like a bible. I cherished that book. Knowledge of networks would eventually be incredibly useful throughout my career.
mwfjover 9 years ago
This can be generalised. It is also one of my favorite ways of doing developer interviews. Do they have a working&#x2F;in-depth knowledge of what keeps the inter webs running? So many people have never ventured out of their main competence bubble, and that bubble can be quite small (but focused, I suppose).<p>For all I know, they believe everything is kept together with the help of magic. I guess I don&#x27;t trust people who don&#x27;t have a natural urge to understand at least the most basic things of our foundations.
评论 #10608185 未加载
Ono-Sendaiover 9 years ago
This is my proposed solution to this kind of problem: Sockets should have a flushHint() API call: <a href="http:&#x2F;&#x2F;www.forwardscattering.org&#x2F;post&#x2F;3" rel="nofollow">http:&#x2F;&#x2F;www.forwardscattering.org&#x2F;post&#x2F;3</a>
评论 #10608928 未加载
评论 #10609139 未加载