I love this work. I might claim it (and some of its antecedents) is the most important distributed systems work of the last decade.<p>Why? It addresses the central question in distributed (and multi-threaded!) system design: when do we need to coordinate between systems? This is important for exactly the reason that the James Hamilton quote says. Successful scalability requires avoiding coordination. Scalable systems (and efficient systems, and fast systems) are the ones that minimize scalability to the level absolutely required by the guarantees they want to offer.<p>As the authors say:<p>> As system builders, of course, we are interested in the complement of this space: what can be achieved, and, importantly, how can we achieve it while minimizing complexity and
cost? The CALM Theorem presents a positive result that delineates the frontier of the possible.<p>This tool for thinking about what possible systems we can build is one that's very understandable to most programmers:<p>> A program P is monotonic if for any input sets S,T where S ⊆ T, P(S) ⊆ P(T).<p>A program is monotonic if, when you run it on a subset of its inputs, you get a subset of its outputs. As you run it on more data, the set of true things may grow, but it never shrinks.<p>> A program has a consistent, coordination-free distributed implementation if and only if it is monotonic.<p>Now we have a useful roadmap to designing scalable distributed system, fault tolerant distributed systems, scalable parallel compute code, and fast multi-threaded code. Using the definition we can identify whether a program is monotonic, and if it is we know we can implement it without coordination. If it is not, we can decompose a program into monotonic and non-monotonic parts, and (if all goes well) take advantage of the non-monotonicity. In many cases, we can do tons of parallel work and only coordinate a couple times.<p>> Conflict-free replicated data types (CRDTs) provide an object-
oriented framework for monotonic programming<p>More conceptual clarity! CRDTs are widely used, and widely talked about. Why do they work? Because they provide ADTs for writing monotonic programs.
This is a cool result from a couple of famous database experts. Though, it seems that the only set of practical programs that satisfy the CALM property are set-based operations found in database systems. These are expressive, but not expressive enough to cover all possible computations. So, is it the case that all computations can be decomposed into a partial order of CALM programs each of which don’t need coordination except at the points of order. If so, is this approach analogous to Lamport’s CStructs in Generalized Consensus?
early 2019 fwiw.<p>CACM has a (large) 2020 blog post about it too: <a href="https://cacm.acm.org/research/keeping-calm/" rel="nofollow">https://cacm.acm.org/research/keeping-calm/</a><p>I remember seeing a blip of this when it came out, but I haven't heard anything since. Anyone know if it's known / used / ??, or has it been subsumed by some other thing?
I'm trying to think about service oriented architectures (or microservices - choose your term). Based on this..<p>- looking at the larger system, you are preventing scale by creating interconnects between these services<p>- looking at the services themselves, it makes the logical boundary smaller, thus giving opportunity to find monotonicity more easily<p>How do others think of this?