The all-to-all dependency step between Calvin's sequencer layer and scheduler layer seems like it will be a problem as things scale, because it means that a single stalled sequencer [edit, orig: scheduler] blocks all writes in the system whether they conflict or not. This is the kind of dependence structure that magnifies outlier latencies and unavailability at scale.<p>In Spanner's design, on the other hand, a transaction can only be blocked by another one on which it actually conflicts. It will have worse average performance, but better tail latency as things scale.<p>Perhaps it is just a nit, but the blog post is somewhat inaccurate when it says that Spanner uses the commit timestamp to order transactions. It uses locks to order the transactions, then holds the locks for extra time to ensure that TrueTime order agrees with the lock-based order.
Tangentially, while FaunaDB and Spanner both seem great, and I love consuming the technical details in articles like these, neither I or my companies are ever going to commit to using something which isn't open source.<p>I have a real problem understanding why Google (and now FaunaDB) are so clueless and/or inconsistent about software licensing. Google seemed to finally get it with Kubernetes, which has grown into a real and vibrant ecosystem; I will (and do) happily use Google's hosted Kubernetes product, while I know, at the same time, that I am not locked into a single vendor. With Spanner (and FaunaDB), all I get is lock-in. (The cynic in me suspects that Google's open-sourcing of Kubernetes is a cunning strategy to dominate the cloud by commoditizing container orchestration, and thus undermining competitors such as Amazon. After all, it's not like Google is open-sourcing Cloud Storage or BigQuery either.)<p>The fact that I can't run these products locally on my dev machine is another major downside unrelated to the philosophical aspects of free software.<p>I'm sure Google and Fauna will make truckloads of money on enterprises that are willing to bet on them, of course. Personally, I am holding out for CockroachDB.
The Spanner design seems more resilient in the face of server failures. The initial Calvin papers call for taking the entire replica offline if a single server in the replica fails. Are there more advanced versions of Calvin that get around this?
> Before we get started, allow me to suggest the following: Ignore the CAP theorem in the context of this discussion. Just forget about it. It’s not relevant for the type of modern architectural deployments discussed in this post where network partitions are rare.<p>Anyone worried about that statement. Ok so they are rare, but what happens when they do occur? File corruption is rare as well but in our large deployment over thousands of servers I see it happen pretty often.
I would like to point out that the original implementation of Calvin is open source: <a href="https://github.com/yaledb/calvin" rel="nofollow">https://github.com/yaledb/calvin</a>
"It’s not relevant for the type of modern architectural deployments discussed in this post where network partitions are rare"<p>I think the post needs to be more precise about this. Modern networks are asynchronous and so they are essentially always partitioned. And this is important, because later, when snapshot and transactional read latencies are discussed, they are not exactly "very low", not on "AP" systems level. Because "CP" systems need consensus they can't just send requests to all of the replicas and wait for the first reply or even read one locally, "very low" there is more about talking to a remote node "low", with all of the problems associated with it, like poor tail latency, etc.
Abadi has an interesting paper here about why so many NoSQL databases are missing transactions. <a href="http://dbmsmusings.blogspot.com/2015/10/why-mongodb-cassandra-hbase-dynamodb_28.html" rel="nofollow">http://dbmsmusings.blogspot.com/2015/10/why-mongodb-cassandr...</a><p>The fairness, isolation, and throughput (FIT) tradeoffs seem more interesting to me than CAP.
I think it's often a better strategy to do sharding or micro-services and if possible keep the service so small that it's state can fit in a single machine. If you can have data boundaries, like for example one customer do not need to access the data of another customer, then you can separate customers's data and place them in different databases.