A part of the solution: robust sync.<p>Let's take the typical example of a photo editing app. Robust sync ameliorates a lot of these issues. You might lose your connection. Not so big a problem if you have your own local copy. One of the administrators is doing something that shuts off access to one copy of your data -- you still have another copy. The network is slow -- with synced data, you just have to send deltas.<p>Robust sync doesn't completely solve everything, but it takes these problems -- which are often edge cases -- and it reduces their impact.
One point seems to be missing: web applications that do have a web front are actually distributed not only on the servers, but also on the clients. This makes reasoning on reliability or security even harder.
A longer treatment of these issues that's still relevant today is "A Note on Distributed Computing" by Ann Wollrath, Geoff Wyant, Jim Waldo and Samuel C. Kendall:
<a href="http://labs.oracle.com/techrep/1994/abstract-29.html" rel="nofollow">http://labs.oracle.com/techrep/1994/abstract-29.html</a>