All valid points <i>if</i> compared against a single-instance RDBMS. Except that modern RDBMS used in any large scale web service will also be guilty of these points. When you shard/cluster your data across various master/slave DB instances you essentially lose the ability to store your data in a non-atomic normal form, just like SimpleDB. In both a sharded RDBMS and SimpleDB your data is denormalized.<p>With a DB setup like this (and all major web sites are) you cannot feasibly join, group by (or do associated aggregations), or order by anymore anyway. Constraints are not really a good idea either. It would be nice if SQL was still useful in a situation like this but it just is not.<p>The days of hugely massive single machine databases is coming to and end if not already dead. Even if you are reporting on massive amounts of data and need to do aggregations across large datasets, then OLAP is increasingly becoming the way to go (again, with fact tables and cube partitions sharded across multiple instances to increase query performance).
Finally, someone took time and said it. Moreover, I'd recommend for someone who didn't have exposure to "big iron databases" to look up performance benchmarks for Oracle, Microsoft and DB2, they're very popular not only among these companies but also among big-iron manufacturers like Sun, IBM and HP. It's crazy what they can do if you use them right, good luck building a business that will need such transactional capabilities of your DB.
if you think in SQL every project seems to require the "advantages" of an RDBMS. But the critics are forgetting that:<p>a) when you use sharding your RDBMS looses most of the advantages you seem to like so much<p>b) SimpleDB and friends not only distribute your data but they allow you to parallelize your queries. think map/reduce<p>c) as a programmer I would much rather solve summing and grouping in my code then learn the subtleties of properly configuring and managing a particular RDBMS (and re-learn everything when I switch vendors)<p>I'll agree that SimpleDB is not appropriate for everything, but most of the perceived problems are really just a lack of creativity.
If you needed to, you could solve most of the problems with SimpleDB by writing a server that wraps it and provides consistency checks, sorting and such. I was considering writing that for a little while.<p>Then they announced that they were going to support mountable drives on EC2 images, and it stopped being necessary. When they release that, you'll be able to load up a more straightforward MySQL or PostgreSQL setup, and use traditional clustering mechanisms.<p>Honestly, I find SimpleDB interesting, but I have no idea what I could use it for without writing quite a bit of middleware to wrap it.
Good article. Like anything, there are pros and cons to each approach. I tested SimpleDB pretty thoroughly for the next version of Gridjit and decided against using it.<p>One of the drawbacks to working with these sorts of storage services is the temptation to create overly-abstract models too early. Abstraction and metadata-based schemas are great but can also add needless complexity (ironically) early on.
I love this article, kudos Ryan!<p>I was about to do similar research before committing our next project of 5TB data space with SimpleDB, ouch! It would have been a disaster. I will probably use the S3 just for mass storage nothing else.<p>We want more of these unbiased articles on HN.
i am trying to sum things up<p>a lot of his problems seem to suggest that SimpleDB require more coding to contol the data.<p>the counter argument is, that okay, this is true, but you also don't do any maintenance ... which seems like a fair trade-off, plus you can always add an abstraction layer in code that simplify the cooding<p>so my question becomes, can't one add an abstraction layer on top of simpledb that makes it act more like a traditional rdbms ?