TL;DR: Pay lots of attention before choosing C* or scylla. only use the most basic stuff, don't work on near-realtime. More pifalls than the number of seconds those software have been out.<p>Every time I see people talking about cassandra or scylla I am reminded of the huge pains and pitfalls of what feels like hacked-together solutions that were built on a very shaky theory.<p>I worked only 2 years on systems that used cassandra and scylla (both on dev and ops side). About 7TB total.<p>I admit they are very fast, provided you do proper continuous maintenance and that your application goes out of the way to conform to the database (absolutely not the other way around)<p>The only safe use of these 2 software is basically:
1) add new unique rows (no updates)
2) <i>much later</i> (granted only after repairs, which take a real lot) you can read, but only up to a certain time in the past.
3) basically almost avoid deleting<p>Anything else, and you are in the realm of cases and special cases.<p>You can't trust errors, your operation might have been applied anyway. LWT terribly slow, limited and you should not run more than a very limited number of them.<p>It's full of features that are added, then the cassandra guys realize that it really does not work, in both theory and practice, and than they deprecate those a few releases later.<p>Compared to SQL, it feels worse than navigating blind on a minefield of special cases.<p>The CQL language does not help. Only use the very basics of the language, otherwise it's a constant game of:
This is the rule
/ With This exception
/ Unless you also have This
/ Except That
/ but it works again with X<p>and on and on.<p>It's like they had a very basic, specific use case, and went to build a ton of features that don't stack on top of it.<p>Like adding full SQL on top of redis, but worse.
I'm likely to bring this up every time I notice a Jepsen post on HN.. I still miss the flair from the old days Kyle :)<p>I guess I'm nostalgic for it. Your posts were during a time when I was still on a steep trajectory in my career growth and discovery. Just soaking everything up, and it was FUN. And others in my "generation" seemed to also be having FUN, and having FUN with it. Things are just a bit more bland these days :|
Companion blog on ScyllaDB's site: <a href="https://www.scylladb.com/2020/12/23/jepsen-and-scylla-putting-consistency-to-the-test/" rel="nofollow">https://www.scylladb.com/2020/12/23/jepsen-and-scylla-puttin...</a>
> Insert is a destructive operation by design. In fact, writing any collection literal (e.g. {} or (1, 2)) is internally implemented by writing a deletion tombstone followed by the new values.<p>These range tombstones are also evil. If you keep doing it to the same partition, even at a rather low QPS, you can easily bring down some nodes by keeping their CPU at 100%. Only tested this with Cassandra though, not Scylla.
I love Kyle's work with Jepsen. Has anyone done anything similar with filesystems? I'm imagining a filesystem mounted on a virtual device, and having the virtual device randomly suffer write faults and various other failures. And then a fuzzer asserting that all consistency requirements are still met. (All operations before a successful fsync should be stored, no lost writes or files, no unexpected garbage data, etc.)
The wolf also shall dwell with the lamb, The leopard shall lie down with the young goat, The calf and the young lion and the fatling together; And a little child shall lead them.<p>All popular databases shall routinely undergo Jepsen reviews; And none shall prohibit publication of test or benchmark results.<p>Isaiah 11:6-7 (New Database Translation)