"While memory-mapping is used for the hashtable, values are read directly from disk, decoupling our I/O throughput from how much memory is available."<p>Whether you're mmap'ing or using read(), you're hitting the page cache before you hit disc, and potentially evicting the LRU page thereof. Glancing through the source it doesn't look like they're using actual "direct IO" (which, in order to be performant, would have to have its own caching layer).<p>That being the case, for lots of tiny reads & writes I'd expect mmap to be superior to read() and write().