About the use of flat files to store the actual JSON: Each object is stored with multiple keys in general, and LevelDB has no native support for values with multiple keys. I considered duplicating the objects, or having a "primary" key (say the ID) that all other keys would resolve to, but finally settled on an external data file and storing the file reference and byte offset in LevelDB.