Good post, but the tone of Joyent's posts so often irk me. Too much poking at Amazon while holding themselves on a pedestal.<p>They're a competitor to Amazon, of course they think they're superior... just so smug.<p>It seems like a bad practice, especially when you end up with pie in your face later. Not too long ago, they had an entire food fight thrown in their direction, so its not exactly like they're immune from issues.
This is one of the key insights to take away from this whole AWS mess: When things start to go wrong, your automatic recovery code will increase the load on your system, and commonly lead you into a spiral of death.<p>I'm delighted to now have a name for this: "Congestive collapse."<p>The first time I saw congestive collapse in a real-world system, it was an ugly surprise. And this is presumably one reason why Netflix runs at 30-60% capacity across 3 AZs: They want to be able to lose a zone without overloading key systems.
Joyent knows all about shared network drive failures.<p><a href="http://techcrunch.com/2008/01/15/joyent-suffers-major-downtime-due-to-zfs-bug/" rel="nofollow">http://techcrunch.com/2008/01/15/joyent-suffers-major-downti...</a><p>Which of course they solved by getting out of the business completely.