A Dismal Guide to Concurrency

Nice article.<p>While I have high respect for the exposition in this article, and by extension Mr. Vogels work that the author makes a reference to, I think it's a tad too early declare eventual consistency a clear winner. In fact, I think it's too early to declare that a there is a contest in the first place. While there seems to be some level of agreement that partition tolerance is required for real world systems consistency vs. availability is more of a continuum instead of two discrete choices. I find that in day to day work some data requires a paxos level of consistency (say a configuration file containing your inter-cluster topology) and some data can be highly inconsistent (like activity data such as 'read' status of emails).<p>As a side-note, I would argue that all eventual consistent systems rely on some domain knowledge about the data that makes it possible for the system to reliably merge conflicting data. Such algorithms usually have the property that they only move state in one direction, emails only move from unread to read for example.<p>What I'm trying hard to figure out is how Consistent, Available and Partition Tolerance map onto Intelligence, Emotional Stability and Good Looks. Do we always require Good Looks and thus end up choosing between Intelligence and Emotional Stability or ...

Great footnote. I'm undecided if I should tackle Clojure or Erlang next. Each one is excellent and interesting in its own way.<p>&#62; A lot of writeups repeat a "nine nines", ie 99.9999999% reliability claim for Erlang-based Ericsson telephone switches owned by British Telecoms. This works out to 31 milliseconds of downtime per year, which hovers near the edge of measurability, not to say plausibility. I was present at a talk Armstrong gave in early 2010 during which he was asked about this. There was a little foot shuffling as he qualified it: it was actually 6 or so seconds of downtime in one device during a code update. Since BT had X devices over Y years, they calculated it as 31ms of average downtime per device per year. Or something like that. Either way it's an impressive feat.

&#62; A Consistent/Available system means that reading and writing always works the way you expect, but if one node fails the whole system stops working.<p>I've seen this mistaken statement several times now and it really bugs me. A consistent/available system can't tolerate partitions, i.e. a split in the network. However, it can certainly tolerate a single failure and remain consistent and available. That's the whole point. A trivial system with these properties is one that replicates every update to all available nodes and requires reads to get matching results from at least half of the nodes. Paxos is another example.

Relevant complementary reading (Widefinder implemented in Clojure): <a href="http://meshy.org/2009/12/13/widefinder-2-with-clojure.html" rel="nofollow">http://meshy.org/2009/12/13/widefinder-2-with-clojure.html</a>

This author is really good at analogies.

Thanks for a nice overview. I have by and large managed to bypass concurrency issues so far, but these days an i7 CPU runs 8 threads and a run-of-the-mill dual socket Xeon could have 24 threads, so I suppose I'll just have to bite the bullet in order to get anywhere near the performance even a single box can supply.<p>Where would goroutines, from Google's Go (<a href="http://golang.org/" rel="nofollow">http://golang.org/</a> ), fit in here?<p>My understanding is that goroutines are pretty much the same as Erlang message passing, except that Erlang messages can work across multiple computers, while goroutines are restricted to one computer (same memory space). Is this correct?

Its not only languages, but data structures that need to change. C.f. this video (<a href="http://vimeo.com/6624203" rel="nofollow">http://vimeo.com/6624203</a>) from ICFP '09

Can someone mirror it outside Facebook?

Will this change with quantum computing and entanglement?<p><i>You can't avoid it by using electrons instead of street urchins. It's impossible for an event happening in one place (eg data changing inside one computer or process) to affect any other place (eg other computers or processes) until the information has had time to travel between them.</i>

tl;dr

Does...this article have e thesis? Or is this just sort of an overview of stuff...