Amazon Aurora Limitless Database

Sharing here a wonderful MIT lecture on the architecture of Aurora I recently watched <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=jJSh54J1s5o" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=jJSh54J1s5o</a>

I can&#x27;t wait until people find its limits!

This sounds very exciting, but I would have greatly appreciated some actual technical details. From this statement<p>&gt; Sharded tables – These tables are distributed across multiple shards. Data is split among the shards based on the values of designated columns in the table, called shard keys.<p>It sounds like this is very much managed CitusDB on top of Aurora, but without any details about the implementation its impossible to know if its just repackaged Citus, or some new novel technology built specifically for Aurora.

Things I want in a serverless database:<p>1. Scales to zero, no cost when not using<p>2. Allows for SQL over API like aurora v1. This point is important, it allows for faster access from other serverless technologies, i.e. lambda

This is some seriously impressive engineering. Spanner is the only DB that can achieve this.

&gt; Join the preview of Amazon Aurora Limitless Database<p>would be the non-editorialized submission, and would have advised potential readers that one cannot currently play with it without a ton of hoopjumpery<p>one will also want to watch out for this, buried 7 paragraphs in:<p>&gt; The preview runs in a new Aurora PostgreSQL cluster with version 15 in the AWS US East (Ohio), US East (N. Virginia), US West (Oregon), Asia Pacific (Tokyo), and Europe (Ireland) Regions.

Ah yes, Reinvent season, when all of the AWS announcements make the front page of HN but most won&#x27;t be generally available for 6-12 months.

These architectures are super interesting to me.<p>We are using the ~equivalent SQL Server architecture in Azure right now (Hyperscale tier) and really enjoy the experience. Not having to worry about specific machines and being able to treat it the same way as an express instance (from an app perspective) are extremely compelling selling points.<p>The only downside I saw in the Microsoft offering was the fact that there was a throttle for transaction log writes (100 megabytes&#x2F;s) which is required to maintain resilience across the cluster. You don&#x27;t want your replicas to wind up 3 weeks behind, etc. My hope is this limit will only increase with time, but even so it is quite workable (for us) today.<p>I may spin up an instance of Aurora (when available) to take a look at how it behaves around our worst-case serialized txn paths. Our use of SQL Server is very &quot;light&quot; in terms of its weird claws&#x2F;hooks, so we could pretty easily shift to another RDBMS vendor in the future, assuming we can sell our clients on the 3rd party mix.

I used aurora for a while and liked it quite a bit (especially compared to Alloydb which was absolutely horrible).

That is quite the ambitious name, obviously it can&#x27;t really be &quot;limitless&quot; even if that limit is just whatever spare resources AWS has to run your database.<p>But they must be confident that that whatever the real limit is, is just not realistic for any use case.

AWS team needs to automate migrations from RDS to Aurora

This is a great blog post, I feel that this level of database knowledge doesn&#x27;t get enough coverage.<p>Maybe it&#x27;s because most projects will never need it, but when you eventually do it feels like you&#x27;re just left to yourself

What type of consistency guarantees are involved?<p>Are we giving away something to get multiple writers to same table? Or does the routing to the correct shard somehow solve for that?

It mentions it can scale write transactions to millions per second, but I see no mention of how it can scale read capacity?

&quot;With limitless databases come limitless costs&quot; (c)

It&#x27;s web scale!

Is this available for Aurora Serverless V2 as well?

limitless is hyperbole

&gt; a new capability supporting automated horizontal scaling to process millions of write transactions per second and manage petabytes of data in a single Aurora database.<p>Limitless horizontal scaling is really cool and all, but does anyone not running a Fortune-500 Tech company actually need this kind of firepower?