I find this to be a bit of an oversimplification. It's certainly not the case that there's only one type of storage, at least from a hardware perspective: there will always be storage that is faster than other storage, and managing that will always be required to optimize performance. I agree with the author that you should program against the simplest abstraction you can, but that's not always clear. Just because your environment is one in which you <i>don't have to</i> doesn't mean you <i>can't</i> or <i>shouldn't</i> in the right circumstances.<p>I'd argue that it's actually <i>more</i> complex now. I programmed in the early 1980s, and did have to worry about where my data physically resided inside the machine, but I never had to worry about whether it was in a different room on a different machine, or sitting on a hard drive in some Amazon Data Center a continent away.<p>As others here note, the idea of using the same abstraction for all storage is at least as old as MMUs and Multics (mid 1960s). What <i>is</i> different is that programmers in the 60s and (early) 70s were usually coding pretty close to the bare metal. Nowadays, Moore's law has allowed us sufficient power to permit programming on top of a big pile of abstractions that try very hard to hide the actual hardware from us.<p>That's a luxury afforded by the sheer power of what we're working with, but the people writing those abstraction layers still have to pay attention to the layer beneath them, and if you go down far enough, you'll find some code that needs to pay attention to what class of memory something is stored in and how to optimally move it around. It's just that work was probably done for you by someone else who wrote your operating system.<p>Just because your programming language doesn't require you to use pointers doesn't meant that indirection isn't being used. You just don't have to deal with it (until it rears up and reminds you it's still there). Joel Spolsky's Law of Leaky abstractions (<a href="https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-abstractions/" rel="nofollow">https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-a...</a>) is relevant here.
The post should be marked as [2006]. A lot has changed since then. For example it doesn't talk about NUMA but it should, if it were written today.<p>I would love to hear some opinions from the authors of more contemporary web servers like one of H2O or Envoy. (The latter may have slightly different use cases though.)
As a historical note, I'll point out that 1975 computers weren't as primitive as the article implies. Virtual memory was introduced commercially in 1961 with the Burroughs B5000. Cache memory was introduced on the IBM System/360 Model 85 in 1969.
Just from the title, I would have expected the opposite approach: not treating RAM as a disk cache, but using RAM only and ignoring the disk. One of the biggest differences between now and then is that we've gone from RAM deficit to RAM surplus. (The VMs that contain so much of today's software can be seen as a way of cutting too-big physical RAM back down to slices that match problems.)<p>This means that batch orientation makes sense for a much smaller slice of problems. Compilers, for example, come from a world of scarce RAM. When my dad started coding, one iteration of change-the-code-and-watch-it-run took days. Now that loop can be seconds: if I pause typing, my IDE starts running my unit tests automatically. It seems to me that things like compilers and runtimes should by default keep everything hot, incrementally updating as fast as possible.
This post elicited a response from antirez back in the day. Then he and "the varnish guy" /g had a meeting of geek minds. You'll learn quite a bit from both.<p><a href="http://oldblog.antirez.com/post/what-is-wrong-with-2006-programming.html" rel="nofollow">http://oldblog.antirez.com/post/what-is-wrong-with-2006-prog...</a>
This article is a bit strange and conflates disk storage with swap and RAM. Modern systems don't necessarily even have swap (looking at you, k8s), so the whole article falls on its face.<p>> And then there were the secondary store, paper tape, magnetic tape, disk drives the size of houses, then the size of washing machines and these days so small that girls get disappointed if think they got hold of something else than the MP3 player you had in your pocket.<p>This is also written terribly.
This is probably the thing that annoys me the most about golang: It abstracts everything as if you were on a 1975 computer, so there's allocations, allocations everywhere! And trying to keep them under control requires intimate knowledge about go's implementation details. Failure to do so dooms your program to run 3-4x slower since allocations are horribly slow to begin with, and now your data is spread all over the place. And even then, there are still some things (like []byte <-> string) that you simply can't get around (unless you want to delve into unsafe land and risk your code blowing up once the runtime implementation changes).
Varnish seemed really popular a decade ago, but I wonder how it fits the modern web and who's using it today and for what purpose.<p>The lack of built-in HTTPS seems killer to me. On the client-facing side it needs a separate daemon for TLS termination and on the upstream side there's no TLS support at all unless you're using the paid version.
Arenas are great for latency sensitive programming. But some of the advice in this piece is a bit limited in that it assumes the presence of swap (often disabled on server hardware) and programs whose important "hot" datasets are likely to fit in memory (i.e. the memory use is nice and predictable--an enviable but not always achievable property).<p>When working with bursty memory demands and code/data you don't always fully control (think a big thick runtime with its own allocation decisions, some of them decidedly "1975") on a system whose swapping behavior you can't predict, hand managing on-disk versus in-memory data residency in userland (via manual memory mapping or something like sqlite/lmdb to take care of it for you) becomes necessary.
TLDR; Varnish is very proud they use memory-mapped files for cache, instead of managing RAM and disk cache separately.<p>That said, don't overestimate the OS ability to understand your usage of RAM. The entity with highest insight on that usage is the application.<p>Maybe there's some fortunate overlap in the case of a caching service (Varnish) using a caching service (OS disk cache / virtual memory). Varnish itself has little clue how their entries will be used beyond what the OS sees.<p>But in more complex applications that's decidedly not the case. Photoshop for example to this day uses a "scratch disk" separate from virtual memory. In fact you better hope you never use swap file with Photoshop, because it becomes unusable.
Since the moderators seem to have collapsed the subthread about it (but not flagged it, so I can't vouch for it), I'd just like to say that the random puerile joke in this article detracts from its technical merit.