Behind AWS S3's Scale

The article states:<p><pre><code> 1956: a 3.75MB drive cost $9k 2024: 26TB drives exist, where 1TB costs $15 </code></pre> I think that radically understates the cost of storage in 1956, when people used mercury delay lines, drum drives, core memory, williams tubes, etc. 1956 was a long time ago and stuff done back then was physically huge, microscopic &quot;modern&quot; units and enormously expensive. Thank goodness photolithography and being able to scale semiconductor transistors...<p>It apparently cost $3200 per month to lease one of them [1] so actually a storage payment model akin to S3...<p>[1] <a href="https:&#x2F;&#x2F;www.dataclinic.co.uk&#x2F;history-snapshot-1956-the-worlds-first-moving-head-hard-disk-drive&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.dataclinic.co.uk&#x2F;history-snapshot-1956-the-world...</a>

The thing that people don&#x27;t realize about AWS is that the hard part, and the thing they do really well, are authorization and billing.<p>Every call is authenticated. Changes to authorization ripple across AWS in realtime. If you revoke a priv, things stop working immediately. That&#x27;s incredibly hard to do, especially when you&#x27;re authenticating billions of requests a second.<p>For billing and telemetry, everything&#x27;s is logged. There are companies that are built on the idea of logging, and at AWS it&#x27;s just something they do - without slowing anything down.<p>AWS just might be one of the most complicated things humanity has ever built, which is a weird thought.

There&#x27;s very little &quot;behind&quot; here, just a bunch of speculation on public articles, many of which are recent and not about how S3 scaled.

&gt; When you aggregate on a large enough scale, a single workload cannot influence the aggregate peak.<p>This is also the “power” behind snowflake and bigquery.

Wow.<p>Also happy that highscalability is still up and about.

One of the crazy parts about S3 which is not touched on in this post is how it&#x27;s becoming a file transfer protocol in its own right. Every cloud vendor now has an S3 compatible interface, but when you look deeper into the actual http contract behind the S3 spec, I don&#x27;t understand how one can shit on FTP and webdav as a protocol and S3 not receive worse treatment. I don&#x27;t want to be reminiscent of the Dropbox FTP deal on HN, but I hope one day people will steer toward open protocols and stop shitting on open ones for reasons that quite frankly 99% of the people couldn&#x27;t give much of a shit about.