QUIC costs something like 2x to 4x as much CPU time to serve large files or streams per byte as compared to TCP. This is because the anti-middlebox protections <i>also</i> mean that modern network hardware and software offloads that greatly reduce CPU time cannot work with QUIC. When combined with the fact that QUIC is userspace, that's just deadly for performance. I'm talking about TSO, LRO (aka GRO), kTLS, and kTLS + hw encryption.<p>Let's compare a 100MB file served via TCP to the same file served via QUIC.<p><pre><code> TCP:
- web server sends 2MB at a time, 50x times, via async sendfile (50 syscalls & kqueue notifications)
- kernel reads data from disk, and encrypts. The data is read once and written once by KTLS in the kernel.
- TCP sends data to the NIC in large-sh chunks 1.5k to 64k at a time, lets say an average of 16k. So the network stack runs 6250 times to transmit.
- The client acks every other frame, so that's 33333 acks. Let's say they are collapsed 2:1 by LRO, so the TCP stack runs 16,666 times to process acks
QUIC:
- web server mmaps or read()'s the file and encrypts it in userspace and sends it 1500b at a time (1 extra memory copy & 66,666 system calls)
- UDP stack runs 66,666 to send data
- UDP stack runs 33,333 number of times to receive QUIC acks (no idea what the aggregation is, lets say 2:1)
- kernel wakes up web server to process QUIC acks 33,333 times.
</code></pre>
So for QUIC we have:<p><pre><code> - 4x as many network stack traversals due to the lack of TSO/LRO.
- 1000x as many system calls, due to doing all the packet handing in userspace
- at least one more data copy (kernel -> user) due to data handling in userspace.
</code></pre>
Some of these can be solved, by either moving QUIC into the kernel, or by using a DPDK-like userspace networking solution. However, the lack of TSO/LRO even by itself is a killer for performance.<p>Disclaimer: I work on CDN performance. We've served 90Gb/s with a 12-core Xeon-D. To serve the same amount of traffic with QUIC, you'd probably need multiple Xeon Gold CPUS. I guess that Google can afford this.
As far as I’ve been able to determine, QUIC suffers from the same SNI data leak that existing TLS versions with TCP has. I understand that ESNI is being (or is already?) included in the TLS 1.3 spec, but it’s obviously optional at this point.<p>Anyways, since QUIC is being touted everywhere as being very secure:<p>> [QUIC] protects both the data and the transport protocol itself<p>It seems like missing ESNI as a required feature is a bit of a glaring omission. Does anyone have a better understanding? To me, it seems like a great opportunity to make ESNI required for HTTP/3. Much like how browsers made TLS required for HTTP/2. I would love further insights if anyone has any.
This is a timely post, since IETF 104 is happening this week in Prague[1]. The QUIC working group will be meeting on Tuesday and Wednesday to make progress on standardization[2].<p>[1] <a href="https://datatracker.ietf.org/meeting/104/agenda.html" rel="nofollow">https://datatracker.ietf.org/meeting/104/agenda.html</a><p>[2] <a href="https://datatracker.ietf.org/doc/draft-ietf-quic-transport/" rel="nofollow">https://datatracker.ietf.org/doc/draft-ietf-quic-transport/</a>
QUIC will also usher in a new era of volumetric DDoS attacks. No longer can content providers use upstream ACLs to block udp garbage and fragments. The only option will be to use Fastly, AWS, or Cloudflare to ride out attacks.<p>QUIC is the tool to bring about the next phase of Internet centralization by the mega players.
> These interposing network elements, called middleboxes, often unwittingly disallow changes to TCP headers and behavior, even if the server and the client are both willing.<p>There is nothing worse than finding out that someone not even at the company anymore decided years before to deploy some crap like this. Drives me absolutely crazy to impose stuff like this where silos in companies means transitioning involves on the order of 4-5 different "components" need to change.
"<i>TCP Fast Open is a stellar example of one such modification to TCP: eight years after it was first proposed, it is still not widely deployed, largely due to middleboxes.</i>"<p>Anyone remember TTCP?
Hopefully vendors like Forcepoint are trying to keep up. The first rollout of QUIC worked terribly in a lot of corporate environments because these MITM content filtering solutions didn't pay attention.
I'm currently working with HTTP/2 (more specifically HAS with HTTP/2 Server Push) and it's just a huge pain to find a high-level library that can help with this. I fear that it'll take even longer for HTTP/3 to be adopted or HTTP/2 might just be skipped altogether. Why are there so many server-side implementations available for a variety of languages though many still lack some features or a client-side implementation altogether?
/offtopic<p>Good folks at fastly, I hope you're reading... I've been waiting very patiently for part 3 of this series for a good part of 2yrs now: <a href="https://www.fastly.com/blog/building-and-scaling-fastly-network-part-2-balancing-requests" rel="nofollow">https://www.fastly.com/blog/building-and-scaling-fastly-netw...</a>
It seems that QUIC is a new transfer protocol created to replace TCP. QUIC uses UDP and LTS/3 and solves the head of line problem addressed in HTTP2. Furthermore, sending the data encrypted allows QUIC to begin transfering data earlier. An experiment by google shows that in connections with high latency or loss QUIC gives a 15% reduction in highest latency.