This is not a CDN: Content Delivery Network. The value is in the networking bit. Storage all around the world for both resiliency, bandwidth cost, scalability, and low latency.<p>Having 1 server with some static file storage is called a web server.
Varnish is one of those tools that has a very specific purpose (a highly configurable reverse caching proxy that is crazily fast). Most of the time I don't need it - but those places I have had to use it, it's made the difference between working services and failing services.<p>One example of where it made the difference was where we had two commercial systems, let's call them System A and System B. System A was acting as front end for System B, but System A was making so many API calls to System B it was grinding it to a halt. System B's responses would only change when System A made a call to a few specific APIs - so we put Varnish between System A and System B caching the common API responses. We also set it up so that when a request was made to the handful of APIs that would change the other API's for an account, we'd invalidate all the cache entries for that one specific account. Once System A was talking to the Varnish cache the performance of both Systems drastically improved.
Some comments:<p>- You don't really need to repeat built-in VCLs in default.vcl. In the article, you can omit `vcl_hit`, `vcl_miss`, `vcl_purge`, `vcl_synth`, `vcl_hash`, etc. If you want to modify the behavior of built-in VCL, e.g. adding extra logs in vcl_purge, then just have `std.log` line and don't `return` (it will fall through to the built-in VCL). You can read more about built-in VCL on Varnish Developer Portal[1] and Varnish Cache documentation[2].<p>- Related to the above built-in VCL comment: `vcl_recv` current lacks all the guards provided by Varnish default VCL, so it's recommended to skip the `return (hash)` line at the end, so the built-in VCL can handle invalid requests and skip caching if Cookie or Authorization header is present. You may also want to use vmod_cookie[3] to keep only cookies you care about.<p>- Since Varnish is sitting behind another reverse proxy, it makes more sense to enable PROXY protocol, so client IPs are passed to Varnish as part of Proxy Protocol rather than X-Forwarded-For (so `client.ip`, etc. works). This means using `-a /var/run/varnish.sock,user=nginx,group=varnish,mode=660,PROXY`, and configuring `proxy_protocol on;` in Nginx.<p>[1]: <a href="https://www.varnish-software.com/developers/tutorials/varnish-builtin-vcl/" rel="nofollow">https://www.varnish-software.com/developers/tutorials/varnis...</a><p>[2]: <a href="https://varnish-cache.org/docs/7.4/users-guide/vcl-built-in-code.html" rel="nofollow">https://varnish-cache.org/docs/7.4/users-guide/vcl-built-in-...</a><p>[3]: <a href="https://varnish-cache.org/docs/trunk/reference/vmod_cookie.html" rel="nofollow">https://varnish-cache.org/docs/trunk/reference/vmod_cookie.h...</a>
I’ve heard good things about varnish and believe I used it for a few things back in the day. Squid was also good when I used it in the kid 2000s (not sure where it’s today) and I think I heard that Akamai was originally just Squid on NetBSD or something like that!! Can anyone confirm or deny?
The first article in the series offers a better explanation of what and why:<p><a href="https://it-notes.dragas.net/2024/08/26/building-a-self-hosted-cdn-for-bsd-cafe-media/" rel="nofollow">https://it-notes.dragas.net/2024/08/26/building-a-self-hoste...</a>
This article is part of a series, and the goal is to create content caching nodes on hosts scattered around the world. When a user connects, the DNS will return the closest active host to them. On a larger scale, it's not much different from what commercial CDNs do.
Always nice to see a project choosing NetBSD! It's pretty easy to manage with Ansible too, so we sometimes rotate it in on "this could be any *NIX" projects and services.
Useless<p>Varnish is not better in any shape or form than nginx for static content.
Varnish has one single usecase, php-sites. - For everything else it will just add a layer of complexity that give no gains. And since varnish is essentially built on apache there is some issues with how it handles connections above about 50k/sec - where it gets complicated to configure, something that nginx does not have.