Caching secrets of the HTTP elders, part 1

This makes me feel old.<p>IM amused by some of the JS community acting like server side rendering and hydration is akin to discovering fire when they just brought back progressive enhancement from circa 2009.<p>Next week we&#x27;re going to have a lesson on semaphores.<p>On a more serious note, there are a lot of places where we could take some lessons from the past to heart. (read stop reinventing the wheel). Permissions systems spring to mind... Something unix&#x2F;ldap like would fit a lot of use cases and be much more clear that some of the awful things I have seen. Good database design needs to make a come back (and people need to stop being afraid of sql). Being able to let go of data, why do some people have so much trouble purging old logs.<p>I could go on but... dam you kids get off my lawn!

I am surprised this doesn&#x27;t list two major resources for learning how caching works:<p>- <a href="https:&#x2F;&#x2F;www.mnot.net&#x2F;cache_docs&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.mnot.net&#x2F;cache_docs&#x2F;</a> for a long time this was the best online resource<p>- <a href="https:&#x2F;&#x2F;developer.mozilla.org&#x2F;en-US&#x2F;docs&#x2F;Web&#x2F;HTTP&#x2F;Caching" rel="nofollow">https:&#x2F;&#x2F;developer.mozilla.org&#x2F;en-US&#x2F;docs&#x2F;Web&#x2F;HTTP&#x2F;Caching</a> is extremely detailed, and based on the previous link too from what I can tell

ETags are also a built-in way to avoid conflicts when multiple clients are mutating the same data. Instead of each client sending<p><pre><code> PUT &#x2F;doc </code></pre> And blindly overwriting what’s there, they send<p><pre><code> PUT &#x2F;doc If-Match: &lt;etag they fetched&gt; </code></pre> If the current server-side ETag had a different value, it can return a 409. Then the client can re-fetch the document, re-apply the changes, and re-PUT it in a loop until it succeeds.<p>You wouldn’t do that for huge, frequently changing docs like a collaborative spreadsheet or such. It’s perfect for small documents where you’d expect success most of the time but failure is frequent enough that you want some sort of smart error handling.

ETags are brilliant at reducing bandwidth and work especially well if the origin webserver is close to the client. Unfortunately, there isn&#x27;t a lot it can do about round trip latency. Even if the payload resides on a proxy cache close to the client, that proxy cannot instantly answer 304 Not changed because it needs to revalidate <i>its</i> cache with its upstream as well (using If-None-Match).<p>So, serving a (relatively small) cvsbase Bay Area to Australia will still be slow unless you&#x27;re willing to accept stale data (Ie. Cache-Control: max-age &#x2F; Expires headers).

The varnish caching lifecycle [1] has some great additional features. Coallate and hold multiple requests, refresh the cache while immediately serving cached content to the requester, serve stale items when the backend is down.<p>[1] <a href="https:&#x2F;&#x2F;docs.varnish-software.com&#x2F;tutorials&#x2F;object-lifetime&#x2F;" rel="nofollow">https:&#x2F;&#x2F;docs.varnish-software.com&#x2F;tutorials&#x2F;object-lifetime&#x2F;</a>

For a file-based HTTP server&#x27;s weak ETags I&#x27;ve used a concatenation of st_dev, st_ino, and inode generation number. For strong ETags I&#x27;ve used a SHA-512 hash.<p>In combination with If-Match: and If-None-Match: this is very powerful.

I really wish HTTP had a no-build solution for caching static assets that didn&#x27;t require either a request or a stale asset period. For example, ability to declare that static assets under a path won&#x27;t change unless a build version at some other path changes.

csvbase looks cool.<p>Apache mod-cache is pretty good on correctedness, not so much in speed. I once did layering with Varnish on top of mod-cache on top of real backend. It was enough. It could even handle moderate traffic WebDAV (that was my main use case).<p>If Windows hadn&#x27;t dropped native support for WebDAV, I would recommend you to take a look at it. If I&#x27;m not mistaken, macos still supports it out of the box, so does GNOME through gvfs.

Is `no-cache` necessary if using `must-revalidate`?

http1.1 was a mistake mostly driven by audience analytics (in other words, advertising)<p>no complex cache (i.e. aggressive cache everything) with a user who can discern how to operate a simple refresh button, was the best solution.<p>cache today is a joke. you cannot press back after going offline anywhere.