Getting hyper about hypermedia APIs

I wrote the spec for application/hal+json that got compared to WS-star , here's where I'm coming from:<p>JSON doesn’t have links. Establishing some basic conventions for that makes complete sense. Defining those conventions is called a spec. Giving a payload that follows conventions a name also makes sense. Establishing that is called registering a media type identifier.<p>It makes no sense to keep reinventing the linking wheel in every API . Pretending like a very minimal media type like hal+json is akin to WS-* is incredibly disingenuous and/or stupid.<p>Establishing a standard media type like hal+json with a bunch of conventions allows us to build generic tooling that can help with both serving and consuming payloads that contain links.<p>Being pragmatic is great, but misrepresenting a genuine effort to improve the status quo and improve the API ecosystem in a reasonable, non-complicated fashion as ‘hand-waving’ or 'unnecessary' is not very constructive.

DHH's API philosophy (send simple JSON serializations over the wire, mostly from the server to the browser) got him through the 2000's OK, but the longer he argues against hypermedia, the less relevant Rails becomes for API design.<p>These are not particularly good arguments he puts forth here. I believe they are in response to Mike Kelly (designer of the HAL+JSON hypermedia format) and his recent post: <a href="http://blog.stateless.co/post/38378679843/hypermedia-apis-on-rails-why-dhh-should-give-a-fk" rel="nofollow">http://blog.stateless.co/post/38378679843/hypermedia-apis-on...</a><p>DHH's comment that URLs instead of IDs are a good idea is true (but he even gets that part wrong in how he implements it, leaving a '.json' extension on the URL). And the rest of the article is trying to hand-wave away the value of hypermedia (links -- all hypermedia means is links, at its core).<p>And comparing HAL+JSON, a blessedly lightweight standard, to WS-* is just dirty.<p>This post didn't convince me of much other than that DHH may run out of steam on this issue before too long.

Bias alert: I'm the newest Rails committer and one of the bigger proponents of Hypermedia APIs in the Ruby world.<p>Anyway, many, many other companies _do_ find hypermedia principles to be useful. See Balanced Payments, for example:<p>&#62; Fun fact: our internal statistics show that client libraries that construct the uri receive roughly 2 orders of magnitude more 404 status codes from Balanced than clients which use the uri directly. &#62; &#62; <a href="http://www.theatlantic.com/magazine/archive/1999/03/the-market-as-god/306397/" rel="nofollow">http://www.theatlantic.com/magazine/archive/1999/03/the-mark...</a><p>GitHub is starting to add hypermedia stuff to their responses:<p><a href="http://developer.github.com/" rel="nofollow">http://developer.github.com/</a><p>Here's their main API guy talking about the advantages even this partial implementation has achieved: <a href="https://twitter.com/pengwynn/status/281849041707474944" rel="nofollow">https://twitter.com/pengwynn/status/281849041707474944</a> <a href="https://twitter.com/pengwynn/status/281849329243787265" rel="nofollow">https://twitter.com/pengwynn/status/281849329243787265</a><p>Twilio has always had elements of hypermedia in their API, and are considering moving further in that direction in the future. They had me speak at their conference for the last two years in a row about the topic specifically.<p>That said, if not doing hypermedia doesn't hurt you, don't change what you're doing! If you're interested in evolving your API over time while supporting old clients and behaviors in a simpler way, then consider checking it out. REST/hypermedia APIs are focused on long-term stability, evolvability, and massive scalability. If you don't need those things, you don't need hypermedia.<p>That said, I'd be happy to answer any questions on the topic, though I'm really busy today, so it might take a while to get back to you.

His questions about discoverability seem to assume that people are suggesting not writing any documentation at all.<p>"The idea that you can write one client to access multiple different APIs" is a straw man.<p>The connection he's making between HAL and WS-* is ridiculous.

hypermedia APIs mean that our API's basically become web pages and every client should act like a browser. That's great if you want... a browser app.<p>If you want to just expose data to let people build interesting things with it, hyperlinks in your api are a bit silly. They might be nice if you want someone to write a browser for your api, I don't see where else it would be awesome. Maybe if you want a web crawler to crawl your api.<p>Also, they add a layer of chattiness to your app. If you have well documented, unchanging url's people can write apps against those. If people have to go to a resource to find the url to another resource, you're going to have a lot of API requests just to look up URL's. People will realize that is a waste of bandwidth, and will hard code URL's anyway.<p>I rarely agree with DHH, but he's right.

<i>According to hypermedia lore, you will be able to willy nilly change your URLs without needing to update any clients. But that’s based on the huge assumption that every API call is going to go through the front door every time and navigate to the page they need. That’s just not how things work.</i><p>This is missing forest for the trees. The point is not that you, the API implementor, will be able to change URLs willy nilly. It's that I, the client, can support a <i>different</i> API implementation by just changing the entry point URL, without changing the application.<p>So, if my application supports the API that Flickr implements, and tomorrow someone creates Blinkr, which implements the same API, the user could just copy-paste the entry point URL and use it, just like I use my RSS reader for all the blogs and news sites out there.<p>Of course, this depends on using standard document formats and a restricted, standard set of methods. Rings any bells?<p><i>Thinking that we can meaningfully derive all that by just telling people to GET / and then fumble around to discover all the options on their own just doesn’t gel with me.</i><p>Well, am I glad that straw-man was burned to the ground!

This post seems hyper-defensive and hyper-reactionary. URLs are <i>fundamental</i> to HTTP web services - they're how the web works. Imagine if you visited a website in your browser and instead of link to the various pages, the page just printed IDs and you had to copy/paste them into the address bar after the domain name.

Sounds like a redefinition of HATEOAS. I don't put URLs as my ID's, but every resource has an "href" value which is the URL which makes it somewhat discoverable by tools such as <a href="https://github.com/jed/hyperspider" rel="nofollow">https://github.com/jed/hyperspider</a>.

I use <a href="http://www.remobjects.com/ro/" rel="nofollow">http://www.remobjects.com/ro/</a>, and have something similar, called RODL Files (<a href="http://wiki.remobjects.com/wiki/RODL_Files" rel="nofollow">http://wiki.remobjects.com/wiki/RODL_Files</a>), this is how look like:<a href="http://wiki.remobjects.com/wiki/RODL_Library_(Service_Builder)" rel="nofollow">http://wiki.remobjects.com/wiki/RODL_Library_(Service_Builde...</a>. This are like SOAP, but better. With them, I can parse the RODL file, and build a python client automatically in seconds, with a python script that output a python client for the remobjects server, with all the class, method calls and that stuff. RemObjects also generate automatically the clients for .net, delphi, js, obj-c, php (<a href="http://wiki.remobjects.com/wiki/Generating_Interface_Code_from_Your_RODL_(Xcode)" rel="nofollow">http://wiki.remobjects.com/wiki/Generating_Interface_Code_fr...</a>). Is something I miss very much when try to do a REST server without RO.<p>And the documentation is embebed in the RODL file, so is possible to output the client with the docs inline, and get it to show in the IDE when a call is made to the python client...<p>So, I think a meta-data about the service is VERY usefull. But the hypermedia is a poor attempt at that (IMHO).

I think these are good points for general public facing web APIs that are specifically designed for being mashed up or will be consumed by third-parties that will never talk directly to your company. I feel hypermedia lore has some really worthy ideas for APIs designed for business use where workflow and data integrity are under strict control but will change frequently and where several client apps will be made by the same company or companies working closely together.

Wow. Very surprising (in a bad way).<p>Basic Web architecture: 1. expose resources, 2. resources have names (URL), 3. allow basic actions on these (GET, DELETE , PUT, POST ) as needed. 4. Include URLs (as links/forms) in representations.<p>OK, <i>now</i> build your API . Please do NOT start w/the API and work back to basic web architecture. Servers should <i>always</i> provide URLs (url templates are fine), NOT the client (by way of snowflakey construction algorithm).

Hypermedia hype has always struck me as parallel to the "semantic web" nonsense from the late 90s.<p>"If we just use RDF triples to encode everything then machines can learn that apples are fruits and fruits are good for you, thus apples are good for you! Huzzah!"<p>Then, Microsoft invented SOAP.<p>"If we just have a WSDL that explains all of our API, then we can have automated methods to communicate between services! Programmers can just auto-generate code and life is great!"<p>Except it doesn't work that way. Most WSDL parsers auto-generate code that you then hand-edit and maintain over time. And, you, as the programmer have to <i>know</i> what methods to call. SOAP is just excessive ceremony transmitted over XML - another excessively rigid structure.<p>So then we got JSON REST APIs. Simple text structures. Reasonable defaults. Basic vernacular. Easily understood.<p>What concerns me about Hypermedia APIs is that folks are using the same sorts of grandiose, architecture astronaut-y stuff that we got out of the last two failed revolutions.<p>Hypermedia API proponents say that REST APIs are "highly coupled" (to the data model and versioning) and don't expose workflows.<p>Heck, that's why REST is so pervasive - they're super easy to write and consume/interact with. Enterprisey folks are so focused on long-term extensibility and maintainability that they overlook the cognitive overhead and inability to work with it on a daily basis. And that you in fact, tend to move <i>slower</i> in development because you can't comprehend or follow what is going on. And that all abstractions leak, leading to libraries and tools (SavonRB) that don't quite work if an API doesn't follow the spec exactly (and they never, EVER follow the spec 100%).<p>My attitude is the opposite: <i>Don't</i> design your APIs as if you expect them to be the next 1,000 year reich. If your API stays small and nimble enough, your consumers will also be able to be flexible to accomodate it. Yes, if you have a User REST API and you decide that your app no longer has users, well, then you have to get rid of it. But adding another layer or two of hierarchy and ceremony on top via Hypermedia wouldn't solve that either! Fundamental universe changes ought to break shit!<p>From what I've seen (I have a subscription to designinghypermediaapis.com which is very well written) Hypermedia APIs are an over-complicated solution to a problem that has a reasonable solution. Yes, links are nice. Want to propose a "Standard" to handle links? Okay, although that's REST! If you have an Object and you want to DELETE it, I don't need to know the URL. I have a convention via REST that allows me to derive it from my data. If that's different, OK, use this thing. But Hypermedia APIs are a <i>lot</i> more than just links (state machines, workflows, media types, etc.)

Aren't links also proper global IDs, allowing us to address objects cross-service ?

One neat trick that I think is worth adding to the discussion: URL templates.<p>Returning URL templates as part of your API response can give you the benefits of having a clean way to access sub-resources, without the headaches and bloat of having to enumerate every possible desired sub-URL.<p>For example, in DocumentCloud, a document's canonical representation has a unique URL for the content of every page as plain text, and as an image, in several different rendered sizes. Instead of doing something silly like this:<p><pre><code> resources: { text: [ "http://www.documentcloud.org/documents/1/pages/page-1.txt", "http://www.documentcloud.org/documents/1/pages/page-2.txt", "http://www.documentcloud.org/documents/1/pages/page-3.txt", ... ], largeImages: [ "http://www.documentcloud.org/documents/1/images/page-1-large.png", ... ], thumbnailImages: [ "http://www.documentcloud.org/documents/1/images/page-1-thumb.jpg", ... ] }, ... </code></pre> ... where you might have 5,000 pages in a document, you can imagine how unacceptably large that response might become. Instead, a single URL template can do the work. (<a href="http://tools.ietf.org/html/rfc6570" rel="nofollow">http://tools.ietf.org/html/rfc6570</a>) The spec has a whole bunch of goodies in it, but we just need the most basic interpolation feature for this case (real example, you may have to scroll sideways to see the complete URL):<p><pre><code> "pages": 5058, "resources": { "page": { "image": "http://s3.documentcloud.org/documents/21939/pages/sotomayor-s-senate-questionnaire-p{page}-{size}.gif", "text": "http://www.documentcloud.org/documents/21939/pages/sotomayor-s-senate-questionnaire-p{page}.txt" }, "pdf": "http://s3.documentcloud.org/documents/21939/sotomayor-s-senate-questionnaire.pdf", "published_url": "http://documents.nytimes.com/sotomayor-s-senate-questionnaire", "related_article": "http://www.nytimes.com/2009/06/05/us/politics/05court.html", "search": "http://www.documentcloud.org/documents/21939/search.json?q={query}", "text": "http://s3.documentcloud.org/documents/21939/sotomayor-s-senate-questionnaire.txt", "thumbnail": "http://s3.documentcloud.org/documents/21939/pages/sotomayor-s-senate-questionnaire-p1-thumbnail.gif" }, </code></pre> Basically, all of the simple URLs a Viewer might need to use in order to browse the document, search the text, and view related resources. In the past, when we've needed to change or expand the number of resources (adding HTTPs-only support, changing the URLs at which the page images are stored, or adding larger sizes of page images), it's been relatively easy to do, without breaking the viewers, or invaliding previously-valid JSON representations of the document. Here's the complete link to the above:<p><a href="http://www.documentcloud.org/documents/21939-sotomayor-s-senate-questionnaire.json" rel="nofollow">http://www.documentcloud.org/documents/21939-sotomayor-s-sen...</a>