TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Distributed Systems and the End of the API

209 点作者 mcms大约 11 年前

18 条评论

ChuckMcM大约 11 年前
Fun stuff, amusing that the definition of a distributed system used; &quot;Where a computer that you never heard of can bring your system down.&quot; is actually one of Leslie Lamport&#x27;s more famous quotes.<p>When I joined Sun in &#x27;86 I thought it was the pinnacle of technological excellence to be a kernel programmer, and I joined the Systems Group, the notional center of the Sun universe, in 1987. However I discovered that the primary <i>reason</i> you had to be picky about kernel programmers what that their bogus pointer references crashed the machine (as they occurred in kernel mode with full privileges) but discovered that <i>network</i> programmers could crash the whole world with their bugs. So clearly they must be in a pantheon above kernel programmers. :-)<p>The author has come to discover that in the network world things can die anywhere, and this makes reasoning about such systems very complicated. Having been a part of the RPC and CORBA evolution I keenly felt the challenges of making APIs that &quot;looked&quot; like function calls to a programmer but took place across a network fabric and thus introduced error conditions that couldn&#x27;t exist in locally called routines. (like the inability to return from the function due to a network partition for a simple example).<p>Lamport&#x27;s work in this space is brilliant and inspired. Network systems can be analysed and reasoned about as physical systems when they exhibit discontinuities when considered as simple algorithms. The value here is to realize that a large number of physical systems tolerate a tremendous amount of randomness and continue to work as intended (windmills for example) while many algorithms only work consistently given a set of key invariants.<p>I gave a talk that was inspired by Dr. Lamports work titled &#x27;Java as Newtonian Physics&#x27; which was a call to action to create a set of invariants, in the spirit of physical laws, that would govern the behavior and <i>capabilities</i> of distributed systems. It was way early for its time (AOL dialup connections were still a thing) but much of the same inspiration (presumably from Lamport) made it into the Google Spanner project.<p>As with many things, at a surface level many people learn an API which does something under the covers across the network but having come up through their education thinking of everything as an API they don&#x27;t fundamentally grasp the notion of distributed computation. Then at some point in their experience there will be that &#x27;ah ha&#x27; moment when suddenly everything they know is wrong, which really means they suddenly see a bigger picture of things. It makes distributed systems questions in interviews an excellent litmus test for understanding where people are in their journey.
评论 #7736677 未加载
评论 #7737955 未加载
评论 #7737418 未加载
programminggeek大约 11 年前
At least the author is wise enough to see that REST&#x2F;RPC are not so different from each other.<p>I actually find it interesting that as I was learning about earlier networked &quot;objects&quot; type systems, programmers ran into problems where they were treating the networked objects as if they were local and that the network always works. Now, when we build REST api&#x27;s they always ship with client libraries that feel like local objects and completely abstract away most notions of network failure, etc.<p>I&#x27;m not saying we&#x27;ve made an unreasonable tradeoff, it&#x27;s just interesting that we seem to be making more refined versions of the same solutions with the same fundamental problems.<p>I guess the author was making a similar point.
评论 #7736007 未加载
评论 #7735900 未加载
jwingy大约 11 年前
CALM and CRDTs are interesting stuff.<p>That being said, I feel like the author is confusing a bit the specific implementations of modern APIs vs the concept of an API which I see as simply some (somewhat standardized) interface to a system which you don&#x27;t own. Those seem like two different problem domains to me, but perhaps I&#x27;m arguing over a different definition of APIs than from what the author is talking about....
评论 #7737197 未加载
评论 #7736000 未加载
cmeiklejohn大约 11 年前
I also covered a variety of similar issues when discussing that offline rich-web applications are perfect for CRDTs, because you are effectively building a distributed system, in my EmberConf 2014 talk [1][2] called Convergent&#x2F;Divergent.<p>[1] <a href="http://confreaks.com/videos/3311-emberconf2014-convergent-divergent" rel="nofollow">http:&#x2F;&#x2F;confreaks.com&#x2F;videos&#x2F;3311-emberconf2014-convergent-di...</a><p>[2] <a href="https://speakerdeck.com/cmeiklejohn/divergent" rel="nofollow">https:&#x2F;&#x2F;speakerdeck.com&#x2F;cmeiklejohn&#x2F;divergent</a><p>* edited to reformat list.
评论 #7735768 未加载
iadapter大约 11 年前
Of course APIs that serve as synchronous endpoints to distributed systems are a leaky abstraction. But its not the only one of its kind, there&#x27;s also Guaranteed Message Delivery [1].<p>I find the philosophy behind Akka in this context a better fit - embrace that networks are unreliable and build your app around this limitation accordingly [2]. The cost is that it results in more work for the developer just like with the usage of CRDTs.<p>[1] <a href="http://www.infoq.com/articles/no-reliable-messaging" rel="nofollow">http:&#x2F;&#x2F;www.infoq.com&#x2F;articles&#x2F;no-reliable-messaging</a><p>[2] <a href="http://doc.akka.io/docs/akka/2.1.0/general/message-delivery-guarantees.html" rel="nofollow">http:&#x2F;&#x2F;doc.akka.io&#x2F;docs&#x2F;akka&#x2F;2.1.0&#x2F;general&#x2F;message-delivery-...</a>
评论 #7735630 未加载
baldeagle大约 11 年前
TL;DR: APIs have issues with concurrency and latency, amongst others. Use Consistency As Logical Monotonicity (CALM) or Conflict-free Replicated Data Types (CRDTs) instead. Here is a little about how CRDTs work. btw: speech in NY on the 15th of May.
评论 #7735524 未加载
dgreensp大约 11 年前
As the author of EtherPad I&#x27;m familiar with CRDT, which is a cousin of OT. They don&#x27;t really replace APIs, unless you are using an API to synchronize data, which is only one of many things you might be trying to do.<p>In other words, if you&#x27;re building EtherPad or Wave, use a fancy data structure for the collaborative document. Otherwise, don&#x27;t. Meteor&#x27;s DDP provides a nice model, where the results of RPCs stream in asynchronously.
评论 #7737157 未加载
评论 #7737447 未加载
评论 #7737445 未加载
ddp大约 11 年前
I believe it was Leslie Lamport who said, &quot;A distributed system is one in which the failure of a computer you didn&#x27;t even know existed can render your own computer unusable.&quot;
评论 #7735715 未加载
josephschmoe大约 11 年前
Many API users are not knowledgeable to the intricacies of network programming. This could definitely use an executive summary at the top with the following info: &quot;What will these new libraries that replace APIs offer to single point API users?&quot;
评论 #7737267 未加载
anuraj大约 11 年前
This is a network programmers view. System, Network programmers concern themselves about systems and topologies. For an application programmer, both needs to be abstracted and only the business logic is important. Money is in the top of the pyramid now - hence the proliferation of APIs.
评论 #7738129 未加载
logn大约 11 年前
Rest is less platform dependent than SOAP&#x2F;RPC. I see that as the main benefit. JSON is easier to work with than XML. The whole idea of service oriented architectures is that users don&#x27;t need to care about the tech stack details of your service. Rest and JSON do a better job of realizing that vision than SOAP&#x2F;XML. I don&#x27;t think anyone&#x27;s claimed that Rest is a design pattern to end all woes. Maybe we haven&#x27;t given due thought to what new design patterns (or data structures, architectures, etc) are emerging these days, and in that light, the article presents a lot of interesting pointers.
评论 #7736371 未加载
kylebrown大约 11 年前
Distributed API&#x27;s are a big part of Ethereum. I think the Merkle tree of the bitcoin blockchain (and the Patricia tree of the Ethereum blockchain) might even qualify as a semilattice.<p>In fact, its by the physics of information theory that a cryptographic blockchain solves the consensus problem. Specifically, information theory emerges from the laws of thermodynamics: Maxwell&#x27;s demon is essentially what secures one&#x27;s private keys from brute-force cracking attempts.<p>I&#x27;d like to see a comparison of how the blockchain solves the CAP problem, alongside CRDT&#x27;s. Are they not both solutions to the same problem?
评论 #7736218 未加载
sagargv大约 11 年前
Joel Spolsky had written along similar lines and argued that it&#x27;s important to know what is happening beneath abstractions.<p><a href="http://www.joelonsoftware.com/articles/LeakyAbstractions.html" rel="nofollow">http:&#x2F;&#x2F;www.joelonsoftware.com&#x2F;articles&#x2F;LeakyAbstractions.htm...</a>
rooted大约 11 年前
I think a distributed system is better defined as a system where timing becomes an issue to the coordination of components in the system.
richm44大约 11 年前
I stopped at the point where he claimed that APIs were always synchronous, this wasn&#x27;t even true in the 80s. For example XLib is a rather well used API and is asynchronous (there are many others).
评论 #7735372 未加载
评论 #7735897 未加载
评论 #7735375 未加载
评论 #7735373 未加载
mantrax5大约 11 年前
So he favors exposing a standard set of distributed data models instead of having APIs.<p>What a horrible idea.<p>Exposing implementations is bad because implementations change.<p>Exposing implementations is bad because as you expose the intricacies of your data model to your client (which he claims is a benefit) you in turn obscure and hide the intricacies of your business domain, which will surely not allow you to patch a service&#x27;s distributed data tree in an arbitrary fashion.<p>It&#x27;s in essence like having SQL as your underlying data model, and replacing your API with an open read&#x2F;write&#x2F;delete access to your SQL server to the entire world, and hoping everyone will run the right queries and all will be all right.<p>It won&#x27;t be all right.<p>APIs will become more asynchronous and eventually all APIs will be seen as protocols, that don&#x27;t <i>necessarily</i> follow a simple request&#x2F;response pattern.<p>But they&#x27;ll remain in the form of abstract commands and queries modeled after the <i>business domain</i> of the application, and not the <i>underlying data model</i> of it.
评论 #7736013 未加载
评论 #7737475 未加载
评论 #7737275 未加载
dreamfactory2大约 11 年前
Hmm the article seems based on some false assumptions. I&#x27;d argue that the whole point of REST as an architectural style is to be stateless and async. Of course you would use an ESB of some kind rather than point-to-point if you want to protect yourself from failure of a solution component - REST lends itself well to that or to building error-handling in the client. And isn&#x27;t &#x27;turning operations into data&#x27; what we are doing by switching from a verb-based model to a noun-based one?
评论 #7737242 未加载
ryanobjc大约 11 年前
CRDTs are absolutely fascinating, but sometimes I really wonder. It seems like you throw words like &#x27;semi-lattice&#x27; around ...<p>Also there is one particular element to the eventual consistency that bothers me, it&#x27;s that all these eventually consistent algorithms aren&#x27;t how high powered neural nets will work. Our brain is highly eventually consistent, but it computes without ever needing these algorithms.
评论 #7736735 未加载