There is so much negativity in these comments! This project is really cool!<p>I really appreciate the trend to rewrite C/C++ libraries in Go. It has always been really frustrating that hacky library wrappers for other languages leave performance and features on the table because they're either incomplete or just too hard to implement in the host language. For most of the languages out there, it is always better to have a native implemention.<p>There are now a number of great embedded key/value stores for Go, which make it really easy to create simple, high performance stateful services that don't need the features provided by SQL.
Now let somebody run a QuickCheck on it, like they did with LevelDB: <a href="http://htmlpreview.github.io/?https://raw.github.com/strangeloop/lambdajam2013/master/slides/Norton-QuickCheck.html" rel="nofollow">http://htmlpreview.github.io/?https://raw.github.com/strange...</a>
It's interesting that the main motivation for this was that the Cgo interface to RocksDB wasn't good enough. I hate to turn this into a language war, but it's a big difference between Rust/Nim/etc where you just call C code more or less directly, and Go/Java/etc, which need a shim layer to bridge between C code and their language runtime.<p>And possibly Go has the right approach! Is it better to make C integration as simple and smooth as possible, to leverage existing libraries, or is it better to encourage people to ditch all that unsafe C code and write everything in Go?
I am excited about the fact that the design is focused on SSD from day one (not just optimised for SSD). Wondering whether the authors have plan to optimise it for Intel Optane which has lower latency and much higher random IOPS at low QD. Currently I am using a cgo based RocksDB wrapper with the WAL stored on Intel Optane, the rest of the data is on a traditional SSD.<p>It will also be great to have some comparison on write performance with fsync enabled.<p>Overall, very interesting project, bookmarked, will definitely follow the development!
Range iteration latency is very important and might be limited by concurrency. I think you can only get 100K IOPS on Amazon’s i3.large when the disk Request queue is full.<p>fio [1] can easily do this because it spawn a number of threads<p>While working with Rocksdb we also found that Range iteration latency was very bad compared to a B+-tree and that RocksDB get good read performance mostly from random read because it's using bloomfilters.<p>Does anyone know if this got fixed somehow recently?<p>[1] <a href="https://linux.die.net/man/1/fio" rel="nofollow">https://linux.die.net/man/1/fio</a>
Silly question - if SSDs or even motherboard had persistent storage of (couple of) 4 KB blocks where you'd be able to fsync < 4 KB (unfinished/non-yet-full pages) of data fast (DRAM + battery) - would that setup speed up writes in databases?<p>It seems that databases often want to persist/flush data in unfinished (4KB or 8KB) pages when they're being built, once they are full, they don't change much - once full they could be normally persisted. Another kind of pages are those that change very frequently - ie. single "root" page which keeps counters or other stuff in single, root page.<p>It seems a bit wasteful that multiple "checkpoints" (flushes/fsyncs) on partial blocks are triggering on hardware whole block rewrites. Similarly with "root"/"meta" pages that keep track of just few bytes frequently changing are triggering similar whole page rewrites.<p>To be honest even some kind of PCI card with little battery and slot for DDR4 would probably do the trick, no? The rest could be implemented in software - as long as you'd have access to fast flushing with battery backed memory that survives hard crash - it should be fine.<p>Is this silly idea?
This could not have come at a better time, I've been looking for a fast, simple, pure Go key-value store.<p>Few questions:<p>- Does this have a log file? If so, what does it log and can it be disabled?<p>- How is the data stored? (Single file, multiple files, etc.)<p>- How is the RAM usage?
off topic: flipping through the dgraph code, I noticed their licensing switch from Apache 2 to AGPLv3, anyone involved around to comment? Adding a draconic open source license is an unwise decision for an early stage database product imo<p>(<a href="https://open.dgraph.io/licensing" rel="nofollow">https://open.dgraph.io/licensing</a> is a dead link)
Figures per minute? Same-process key-value should give millions <i>per second</i> operations (using one thread). And accessing via network, e.g. a la Redis, should be hundreds of thousands <i>per second</i> [1].<p>[1] <a href="https://redis.io/topics/benchmarks" rel="nofollow">https://redis.io/topics/benchmarks</a>
How do I use it? there is zero docs on how to even get started, what the API is like(is there even one?) This could be worth all the peanuts in the world, but if the only way to learn how to use it, is read the Go source, very few people will bother.