Improving NGINX Performance with Kernel TLS and SSL_sendfile

We&#x27;ve been running kTLS + SSL sendfile on FreeBSD at Netflix for the last 6 or 7 years. (We had local patches to nginx, before nginx did them &quot;right&quot;, and 2 versions of kTLS before the 2nd version was upstreamed to FreeBSD). The savings in terms of CPU use and memory BW are pretty substantial. Especially when you use a NIC which can do in-line kTLS offload, then things basically go back to pre-TLS costs because the buffers are not touched at all by the CPU.<p>BTW, FreeBSD 14 supports cha-cha poly. But is far more CPU intensive than GCM, so I&#x27;d advise against using it.

Just a note to anyone wanting to use kTLS: make sure to benchmark it first, like in the article. Depending on the CPU architecture, it might even be slower than plain userspace TLS.<p>Also, while the tx side has seen lots of investment (from CDN companies&#x2F;owners), the receive side usually comes later. For instance, it&#x27;s not supported for TLS 1.3 in openssl (although there&#x27;s an open PR).

How long before we push everything into the kernel?

&gt; Alpine Linux 3.11–3.14 – Kernel is built with the CONFIG_TLS=n option, which disables building kTLS as a module or as part of the kernel.<p>I wonder if this is still the case with 3.15?<p>Edit:<p>I figured I could check for myself. I don&#x27;t know for sure what the default kernel package is, but there apparently is a linux-lts package. After installing this package, it leaves a config-lts file in &#x2F;boot which, when grepped, returns:<p># CONFIG_TLS is not set<p>The more I learn about Alpine (and musl), the more I don&#x27;t want to use them. It appears as if I have an inherent performance penalty serving https web sites with nginx when I do it from Alpine.

(2021)

This would make Nchan even faster, neat.