A Decade of Dynamo

DynamoDB is amazing for the right applications if you very carefully understand its limitations.<p>Last year I built out a recommendation engine for my company; it worked well, but we wanted to make it real-time (a user would get recommendations from actions they made seconds ago, instead of hours or days ago). I planned a 4-6 week project to implement this and put it into production. Long story short: I learned about DynamoDB and built it out in a day of dev time (start to finish, including learning the ins and outs of DynamoDB). The whole project was in stable production within a week. There has been zero down time, the app has seamlessly scaled up ~10x with consistently low latency, and it all costs virtually (relatively) nothing.

I was at AWS from 2008 to 2016. Werner Vogels, Amazon&#x27;s CTO (yep, not just AWS&#x27;, but Amazon&#x27;s, as he had to point out numerous times) has been one of the most talented, humble and generous senior exec I&#x27;ve ever met in my life.<p>Lots of good memories of time spent with him, and one of the sad aspects for me of leaving Amazon.<p>His blog writings are really interesting. If you haven&#x27;t already, I suggest you search the archives, there are several hidden gems there.

I&#x27;m more interested in solutions like Spanner and Cockroach. Different tradeoffs for different applications, but they seem to be the most general purpose of the highly scalable databases. DynamoDB is cool and I&#x27;ve tried to adopt it for things, but it&#x27;s surprisingly hard to imagine an application where the model isn&#x27;t somewhat limiting. The capacity provisioning is also quite painful, which doesn&#x27;t help matters any.

I&#x27;d be interested to hear how others are handling read&#x2F;write capacity configuration for dynamo. It seems like it would be very easy to hit the account limit of 10,000 units once you are querying any significant amount of data. I&#x27;ve also run into issues with auto scaling where you have to endure up to 15 minutes of downtime before the scaling kicks in [0]. Even on a table with ~2000 items I&#x27;ve found it becomes quite slow and costly to fetch data. Also the 25 item limit on batch writes makes it pretty frustrating to edit&#x2F;delete lots of data.<p>- [0] <a href="https:&#x2F;&#x2F;hackernoon.com&#x2F;the-problems-with-dynamodb-auto-scaling-and-how-it-might-be-improved-a92029c8c10b" rel="nofollow">https:&#x2F;&#x2F;hackernoon.com&#x2F;the-problems-with-dynamodb-auto-scali...</a>

When they mention companies using DynamoDB, at least one of those actually uses their own implementation of Dynamo that they wrote to work around cost and performance limitations.<p>The main problems faced are not the ability to scale or reach performance benchmarks or keep data safe. They are operational, and primarily problems of infrastructure complexity and management. Oh, and having developers architect and manage the operations of a really freaking huge service is a bad idea. (No offense intended - those developers don&#x27;t want to be woken up in the middle of the night either)

Direct link to the Dynamo Whitepaper PDF: <a href="http:&#x2F;&#x2F;www.allthingsdistributed.com&#x2F;files&#x2F;amazon-dynamo-sosp2007.pdf" rel="nofollow">http:&#x2F;&#x2F;www.allthingsdistributed.com&#x2F;files&#x2F;amazon-dynamo-sosp...</a>

I don&#x27;t know why amazon is so taken with Dynamodb. I find it to be incredibly unintuitive and lacking real world application, requiring applications to perform gymnastics to work with it.

Now that autoscaling is available for Dynamo DB, my main complaint with DynamoDB is the lack of out of the box backup solution that works at scale.<p>Production DB without backups is unthinkable. It just takes one human mistake to erase tons of data. Consistent and regular backups are must have for any production system.

&gt; The Dynamo paper was well-received and served as a catalyst to create the category of distributed database technologies commonly known today as &quot;NoSQL.&quot;<p>No, sorry, it was Memcached and Bigtable paper that popularized &quot;NoSQL&quot; term. Although there were many NoSQL databases tracing way back to 60s [1], those were the ones that &quot;served as catalyst&quot; for the term &quot;NoSQL&quot;.<p>[1] <a href="http:&#x2F;&#x2F;blog.knuthaugen.no&#x2F;2010&#x2F;03&#x2F;a-brief-history-of-nosql.html" rel="nofollow">http:&#x2F;&#x2F;blog.knuthaugen.no&#x2F;2010&#x2F;03&#x2F;a-brief-history-of-nosql.h...</a>

Congrats to aws on the impact DynamoDb has had on the ecosystem &amp; industry. The article does make it seems like DynamoDb was the first to publish a unique noSQL architecture. Is this true?

I would be interesting to read some comparisons between DynamoDb, CosmosDb and maybe Spanner.

Need a library sdk for a Dynamo Sync feature to allow easy development of offline mobile apps. Similar function to Cognito Sync. Also hope that AWS will release a serverless SQL db. And cheaper price.

Oh <i>that</i> Dynamo. Not this Dynamo: <a href="http:&#x2F;&#x2F;dynamobim.org&#x2F;" rel="nofollow">http:&#x2F;&#x2F;dynamobim.org&#x2F;</a>

What is the machine in the photo?

amazing product

The thing doesn&#x27;t even support a useable cross region replication. On top of that the whole read&#x2F;write capacity is a joke (a painful one at that).<p>Other than a dirty js config or a prototype store this db is useless.