ACM Queue interview of Theo Schlossnagle, founder and CTO of Circonus, about IRONdb [1], an alternative time-series platform to Prometheus/InfluxDB. Some high-level architecture choices:<p><pre><code> - CRDT to avoid coordination service
- LMDB (B-Tree) for read optimized requests
- RocksDB (LSM-tree) for write optimized requests
- Modified consistent hashing ring split across two Availability Zones
- OpenZFS on Linux
- Flatbuffers for Serialization-Deserialization
- Information Lifecycle Management for historical data
- circllhist; HDR [high dynamic range] log-linear quantized histograms
</code></pre>
[1] <a href="https://docs.circonus.com/irondb/" rel="nofollow">https://docs.circonus.com/irondb/</a>
> But the real advantage of the CRDT approach is that, if you can limit your entire operation set to CRDTs, you can forego consensus algorithms such as Paxos, Multi-Paxos, Fast Paxos, Raft, Extended Virtual Synchrony, and anything else along those lines.<p>Things like this are said a lot, but I don't believe CRDTs provide the same consistency guarantees as Paxos/Raft.<p>Anyone have thoughts on why CRDTs might be favored over the internals of something like a well written eventually consistent datastore like Cassandra that doesn't use Paxos/Raft?<p>Is he just saying that your consensus algorithm then just becomes that math of CRDTs and not the careful complexity of a distributed consensus protocol?<p>(the interviewee then goes on to describe that CRDTs fit their use case well, so maybe that is all he is saying...)
> We've witnessed more than exponential growth in the volume and breadth of that data. In fact, by our estimate, we've seen an increase by a <i>factor of about 1x10^12</i> over the past decade.<p>If 10 years ago we had X data collected, now it is 1,000,000,000,000 * X data??? I find this highly surprising and wonder if it is a typo, and if so, what the real numbers are.