Benchmarking BDB, CDB and Tokyo Cabinet on large datasets

Sorry, but 500MB DB size is a tiny dataset these days (anything < 1GB is tiny, < 4GB is small, < RAM on a single node (~8GB-64GB) is medium, < Disks on a single node (~128GB to a few TB) is large, huge dataset requires multiple nodes and typically above 128TBs.)

It would be nice if DJB would release CDB under the public domain like he has now done with most of his other software. However, there's a great public domain implementation by Michael Tokarev I'd recommend: <a href="http://www.corpit.ru/mjt/tinycdb.html" rel="nofollow">http://www.corpit.ru/mjt/tinycdb.html</a>

In my experience, the downside of BDB is simply the license. There is no way for me to use it in my program (and distribute my program) without opening up my source. I can't simply 'query' a BDB server - by using BDB at all I have to link to it and also open source my own code as part of the Sleepycat License (or so I've been told - IANAL).

I'm a bit surprised at the poor showing of Tokyo Cabinet in this test, given the amount of buzz it's gathered in the last few months, but the CDB results honestly don't surprise me in the least -- it's a special-purpose datastructure wrapped in a thin library that just <i>smokes</i> any other DB type if your workload allows for a complete rebuild of the database on each update.

It's a shame there is not Redis among the tested KV DBs. I guess the networking layer is not an option in this specific use case.