Amazon DynamoDB – a Fast and Scalable NoSQL Database Service from AWS

A feature request (I see that Werner is reading this message thread): for development it woud be very nice to have the Java AWS SDK have a local emulation mode for SimpleDB and DynamoDB. This would allow development while flying or otherwise off the Internet. Similar functionality as AppEngine's local dev mode.

This looks like a great service - there are some really interesting ideas here. The provisioned throughput is a great idea. It's interesting to see it runs on SSDs as well, I wonder if its storage is log based like LevelDB. The composite hashed indexes are really interesting as well - I guess they partition the index across nodes, and each node maintains a range of the sorted index. It'll be interesting to see how usable that is in practise.<p>I read with interest Steve Yegge's mistakenly-leaked email about the oppressive conditions for engineers at Amazon. It's hard to reconcile with the sort of innovation they consistently show.

I simply do not see how competing cloud vendors can keep up with this. Most of them are still struggling to provide anything beyond a simple API to start/stop machines.

This still seems a bit expensive to me for an application that would require thousands of writes per second? ie. 5k writes per second is ~$120/day. Using this for performance based analytics for example would seem out of the realm of reason for the moment.

For about $15/month for minimal "reserved capacity" charges and $1/gigabyte per month replicated disk space this service looks like it will cure a lot of deployment headaches for a reasonable cost.<p>Interesting that when I signed up for the service that they verified my identity with a cellphone call, like Google sometimes does.

"Amazon DynamoDB stores data on Solid State Drives (SSDs)" This is big.

I would love the option to add N index (at cost).<p>My guess is they will add options for additional indexes in the future...everyone needs start somewhere. Even at Amazon's size and scale.

While putting my data completely in the hands of another company makes me nervous, I have used s3 since its release and have had no problems. Amazons really seems to "get" developer needs. I do wish they would ease down a smidge on thier pricing in general, but otherwise I really feel about them the way I feel about google at this point.

<a href="https://img.skitch.com/20120118-etwh2gwu52isnn47fw2jpcir5p.png" rel="nofollow">https://img.skitch.com/20120118-etwh2gwu52isnn47fw2jpcir5p.p...</a><p>You could buy a lot of Riak or Cassie for that.

Here's a brief overview comparison of DynamoDB vs. BigTable:<p><a href="http://vschart.com/compare/dynamo-db/vs/bigtable" rel="nofollow">http://vschart.com/compare/dynamo-db/vs/bigtable</a><p>The site is also able to compare many other different databases.

I work for a popular startup that has been privately testing dynamo for the last few months.<p>It's a fantastic product and even while in private beta has been stable and well supported.

1$/GB/month on SSD and replicated. So basically, 0.25$/rawGB/month if they replicate 4 times.<p>They are making money on the read/write and are selling the capacity at current cost. Which — knowing the tendency for AWS to decrease the prices very slowly combined with the huge decrease of the prices of the SSD drives in the past months/years — is not a bad strategy to convince us to switch.

I posted a comparison to Cassandra here: <a href="http://news.ycombinator.com/item?id=3480480" rel="nofollow">http://news.ycombinator.com/item?id=3480480</a>

There are some bugs in signing up for the service. In console, if I go to dynamoDB tab I am asked to Sign Up first. I click the sign up button and I am told "You already have access to Amazon DynamoDB". Repeat.

Question about Composite Hash Keys that someone might have the answer to (or be able to relate to other known implementations):<p>The composite key has two attributes, a “hash attribute” and a “range attribute.” You can do a range query within records that have the same hash attribute.<p>It would obviously be untenable if they spread records with the same hash attribute across many servers. You'd have a scatter-gather issue. Range queries would need to query all servers and pop the min from each until it's done, and that significantly taxes network IO.<p>This implies that they try to keep the number of servers that host records for the same hash attribute to a minimum. Consequently, if you store too many documents with the same hash attribute, wouldn't you overburden that subset of servers, and thus see performance degradation?<p>Azure has similar functionality for their table service, requiring a partition key, and they explicitly say that there is a throughput limit for records with the same partition key. I haven't seen similar language from Amazon.<p>Whether you scatter-gather or try to cluster values to a small set of servers, you'll eventually degrade in performance. Does anyone have insight into Amazon's implementation?

Awesome. Anybody want to sublease a 3 yr RDS reserved instance? :P

What's the concern for lock-in here - how difficult with DynamoDB be to migrate away from? Amazon seems to be increasingly catching the App Engine Syndrome, though to be fair, it's been mostly network traffic-style functionality in the past.

From the limits page(1) I see you can only have 256 tables per account.<p>(1) <a href="http://docs.amazonwebservices.com/amazondynamodb/latest/developerguide/Limits.html" rel="nofollow">http://docs.amazonwebservices.com/amazondynamodb/latest/deve...</a>

I read through a number of the docs and can't quite find the answer to this question, hopefully someone here can help me out quick.<p>I already have a bunch of (large-ish, deeply nested) JSON objects defined for my application. I don't really want to go about redefining these since they work great between my various node processes and the front end. I am saving them in a nosql database already, I am curious about switching (to save on devops costs). I only request based on 1 Hash Key (int) and 1 Range Key (int) for all my current get operations.<p>Looking through the docs/examples I see a lot of this type of thing:<p><pre><code> {"TableName":"comp5", "Item": {"time":{"N":"300"}, "feeling":{"S":"not surprised"}, "user":{"S":"Riley"} }, "Expected": {"feeling":{"Value":{"S":"surprised"},"Exists":true}} "ReturnValues":"ALL_OLD" } </code></pre> The JSON item has a kinda-of 'type syntax' on it. I really don't want to redefine my deep objects, but would be willing to redefine the Hash key and Range key, while leaving the rest of the nested types alone.<p>Ok, my question: Do my JSON objects need to conform to this 'type syntax' JSON notation in the examples? Or can I save just any JSON object into this database and only annotate the Hash Key and Range Key using this special notation?

Pricing: <i>$0.01 per hour for every 10 units of Write Capacity and $0.01 per hour for every 50 units of Read Capacity.</i>

I think this is huge. My first wish, before secondary indexes, is that they don't round up to the nearest 1KB for pricing.

With regard to reliability, how safe would a service like this be to store your data? They mention "back up to s3" but this sounds more like archiving. I'm wondering about backups in the case of a problem, or is the data so spread across all of their physical locations that data loss impossible?

Fast, Consistent, Secure &#38; Replicated.<p>Game changed.

I'm too scared of the lock in. Afaik this isn't like EC3 where you can move a VM off the EC3 stack. The data is stored in a proprietary format so once your in, you're in.<p>Game changer? I need to see some evidence first on how well it performs and integrates.

Very interesting that there's no mention of the CAP theorem here, despite Amazon's heavy reliance on it in the past when marketing their non-relational stuff.

I hope this means an SSD option for RDS is coming soon.

I wrote an Erlang API for DDB while it was still private. I'm hoping my client will let me opensource it in the near future.

Why is there a 1MB reply limit on BatchGets and obviously no limit on Gets? Did anybody find limits on a single item?

Any word on when boto will support this?

Is there a Ruby SDK?

Trying to read thru all the hype, this looks like Riak hosted on machines with SSDs, with less features, and a nice billing system in front of it.<p>Of course for people who want a hosted solution, the fact that it is hosted, is what gives it a lot of value. There haven't been a lot of hosted NoSQL databases, at least on this scale and availability, out there.<p>But technologically, what's new here? Is there anything here that is really innovative? (not a sarcastic question)