The “API Mandate” memo at Amazon

As a current Amazon employee I can confirm this is no longer the case or only applies to such a select few pieces of software that it doesn't actually mean what you think it does. The amount of day to day stuff that is relied upon that runs on greasemonkey scripts and web scraping is insane. API's for a ton of things don't exist or are have heavy gatekeeping. (eg. contact x to get onboarded to our API and they just ghost you). Of course you can get a high level leader to tell you to use a proper API in a mailing list email thread where you are trying to get help to solve a problem but they will not actually help you get working API access, all talk no action. A perfect example is there is this basic CRUD app that I rely upon and I see the current maintainer is working on all these feature requests, the problem for me is that the site has a ton of AJAX and takes like 20 mouse clicks to get retrieve basic information. So I open a ticket / feature request just to get a basic REST API for the reads and they politely tell me I am an idiot and close the request. So much pointless work is created at Amazon from lack of API access.

How far does this principle scale? I&#x27;ve heard of this strategy and thought &quot;that&#x27;s kinda cool.&quot; Amazon is obviously successful in its domain, so it would be easy to assume some causality.<p>And yet, there&#x27;s another big tech company, fruit symbol I think, that has played the long game of continually fine tuning the interaction points to make the integration of their parts more than the whole. They get credit for attention to detail, leveraging their integration and their ability to move in ways they do because of their thorough integration across their whole hardware&#x2F;software stack. They&#x27;ve recently been in the press because of just how much they were able to tune their integration to the problem with their first desktop computing chip. Again, we credit their success with some causality due to this strategy, which (to me) is very different than the Amazon strategy.<p>Is it because Amazon is a cloud company and Apple is a hardware company that they have these different approaches to product development and each enjoy success? Or at the end of the day, are these &quot;do it this way&quot; less responsible than we&#x27;d like to believe?

I worked at a company that read the Amazon memo virtually straight after it was published and decided to copy it. The CIO at the time predicated every system&#x27;s teams&#x27; bonus on it.<p>No API, no bonus.<p>What followed was 18mths of people building enterprise messaging systems and &quot;buses&quot; of various types. Hundreds of pages of documentation on APIs was prepared and released.<p>I think 2 API&#x27;s with maybe 40 methods actually went live.<p>There were no bonuses<p>Many people left<p>The people who stayed were unhappy<p>It never happened, everyone quietly forgot about it after 18 mnths.<p>The CIO stayed for another 3 or 4 years - he only left when a new CEO came in due to internal promotion. The new CEO used to run one of the P&amp;L&#x27;s&#x2F;business units and hated the CIO with a passion.<p>So... it&#x27;s not the API mandate that made the difference at Amazon.

It&#x27;s interesting and often missed that this strategy implies a view of human collaboration and organization. Note that the memo says: &#x27;All teams... Teams must... ...another team’s data store...&#x27;. It seems that someone figured out that a (5-10 person?) team is the right human collective size to design and build useful stuff, but that collective shall expose what they build to other teams via APIs. It&#x27;s fine for the team to do things in a tightly coupled way - internally! In a way, tight collaboration between teammates can be reflected in tight coupling of internal components of whatever they build. But across teams the coupling shall be more formal and via APIs, reflecting looser and less powerful collaboration possible across teams.

I love this. As others have mentioned I wonder how much input Jeff had, and from who, before making the mandate.<p>I am experiencing the polar opposite of this mandate. The systems in my organization are always built to require human touch-points. What&#x27;s worse, our CTO mistakes these menial interactions as &quot;teamwork&quot; and &quot;collaboration&quot; when they are really just toil to compensate for the lack of platform-level thinking. I love how a CEO can put this so bluntly, upend everyones work for a couple of years and build a juggernaut because of it.<p>The idea runs parallel to one I have been championing throughout my career with SOAs which I call &quot;self serve architecture&quot;. I want others in the organization to be able to pick up use my services to their benefit with zero input or help from me or my team. I tell my team to design the API as if it were GitHub&#x27;s API.<p>Practically, that means - There are up-to-date and easy docs that cover what you need to know. - People can gain access on their own (via some existing workplace&#x2F;team based credential). At most we have to add them to a list somewhere. - The system will protect itself and inform users against problematic use (quotas, throttling, and visibility into this) - You have visibility into who your users are and what they are doing such that you can assess value, learn from usage, and communicate to consumers when necessary.

Does anyone know who else was involved in constructing this memo?<p>&quot;There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever&quot;<p>Was Bezos deeply enough involved in Amazon&#x27;s engineering to set those rules himself, or was the text of the memo influenced by a senior engineering group that he was working with?

I wish GitHub under Microsoft followed this philosophy. So much of their repository management can be done through their APIs, but you hit some painful brick walls around things like enterprise security where you could <i>really</i> use centralized management.<p>My business area uses around 200 repositories. APIs aren’t really optional at that scale.

Kind of lame that it doesn’t even mention Steve Yegge or his blog, which is almost certainly the source.

Is this the same doc as &quot;Distributed Computing Manifesto&quot; mentioned in Werner Vogels&#x27; blog? [1]<p>These legendary Amazon Memos haven&#x27;t leaked, unlike Billg&#x27;s various memos [2]. That is... journalistically unfortunate, I would say. I even wonder current Amazonian actually has the access to these docs.<p>[1] <a href="https:&#x2F;&#x2F;www.allthingsdistributed.com&#x2F;2019&#x2F;08&#x2F;modern-applications-at-aws.html" rel="nofollow">https:&#x2F;&#x2F;www.allthingsdistributed.com&#x2F;2019&#x2F;08&#x2F;modern-applicat...</a><p>[2] <a href="https:&#x2F;&#x2F;lettersofnote.com&#x2F;2011&#x2F;07&#x2F;22&#x2F;the-internet-tidal-wave&#x2F;" rel="nofollow">https:&#x2F;&#x2F;lettersofnote.com&#x2F;2011&#x2F;07&#x2F;22&#x2F;the-internet-tidal-wave...</a>

If every part of Amazon is really like that, then it&#x27;ll make the task of breaking them up much easier!

Always worth reading Yegge&#x27;s insider take on this - <a href="https:&#x2F;&#x2F;gist.github.com&#x2F;chitchcock&#x2F;1281611" rel="nofollow">https:&#x2F;&#x2F;gist.github.com&#x2F;chitchcock&#x2F;1281611</a>.<p><i>The Golden Rule of Platforms, &quot;Eat Your Own Dogfood&quot;, can be rephrased as &quot;Start with a Platform, and Then Use it for Everything.&quot; You can&#x27;t just bolt it on later. Certainly not easily at any rate -- ask anyone who worked on platformizing MS Office. Or anyone who worked on platformizing Amazon. If you delay it, it&#x27;ll be ten times as much work as just doing it correctly up front. You can&#x27;t cheat. You can&#x27;t have secret back doors for internal apps to get special priority access, not for ANY reason. You need to solve the hard problems up front.</i>

This really shows in the products Amazon offers to developers. Everything that isn’t already an entrenched business success is some random internal tooling that got productized because everything does.

&gt; All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.<p>I wonder how this is accomplished for event driven architectures built on shared event buses.

Things like these are taken as dogma or a religion -- and are applied to all things in an organization, for better or for worse.<p>When there&#x27;s a case for tighter coupling and less services, (and yes, there are cases for it), this memo gets brought up and microservices win the argument.

&gt;It doesn’t matter what technology they use. HTTP, Corba, Pubsub, custom protocols — doesn’t matter.<p>So how does Amazon manage this btw? I don&#x27;t find AWS SDKs to be all that consistent or inconsistent. They are usually good _enough_. Is that all it takes?<p>By contrast, Google seems to spend a lot of time on their single repo, build the world, approach. For the most part it seems beloved, or at least people try to recreate it outside google with things like Bazel.<p>I feel like Google&#x27;s approach is more popularized. Is that because its actually better, just advertised more, or simply consistent enough to explain?<p>Can anyone shed light on Amazon&#x27;s approach?

Is this actually still applied in 2021?<p>Most memos from two decades ago aren&#x27;t still followed.

&gt; It doesn’t matter what technology they use. HTTP, Corba, Pubsub, custom protocols — doesn’t matter.<p>CORBA is quite close to direct linking, with a network in between. The developer does not see it as a service or protocol, but a library call, which is rather the point. And it&#x27;s not very compatible with the next one:<p>&gt; All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.<p>CORBA&#x2F;COM never played well over the internet.

As much as anyone who uses AWS and its various client libraries can attest to the areas this manifesto didn&#x27;t solve, I think it&#x27;s still very much the right way to go, and I think the success of AWS was at least significantly attributable to it. I&#x27;ve been part of several projects whose goal was to build Good interfaces on top of services after the fact, and I&#x27;m absolutely convinced that the shittiest interface in the world built in from the start trumps whatever you think you&#x27;re going to be able to do later.

Does Amazon has some standards&#x2F;conventions for inter-team APIs, i.e. something like Google&#x27;s AIPs (API Improvement Proposals) [1,2]?<p>[1] <a href="https:&#x2F;&#x2F;google.aip.dev&#x2F;general" rel="nofollow">https:&#x2F;&#x2F;google.aip.dev&#x2F;general</a><p>[2] <a href="https:&#x2F;&#x2F;google.aip.dev&#x2F;" rel="nofollow">https:&#x2F;&#x2F;google.aip.dev&#x2F;</a>

I&#x27;ve worked at a place where this was cargo-culted and it worked horribly. All the worst aspects of &#x27;services-first&#x27; design.<p>I think this works much better when each team is working on what could be regarded as a complete end-to-end product e.g. a database. It works poorly when each team is working on part of a product.

<i>&quot;HTTP, Corba, Pubsub, custom protocols — doesn’t matter.&quot;</i><p>That could have backfired pretty badly, haha.

&gt; All teams will henceforth expose their data and functionality through service interfaces.<p>&gt; Teams must communicate with each other through these interfaces.<p>I read that as interfaces to the <i>team</i>, instead of interfaces to software the team is responsible for. Something like Mechanical Turk(<a href="https:&#x2F;&#x2F;www.mturk.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.mturk.com&#x2F;</a>) but for internal team communication. I&#x27;m not sure if that was what was meant but that sounds interesting.

I had always read this as a metaphorical memo. Not necessarily that every team must have a software service running somewhere in infrastructure serving RESTful routes but rather as a way of thinking about agreements between groups. Your team&#x27;s &quot;API&quot; could be documents describing: if you want us to do XYZ, send us a request via ABC, and you should expect a response in UVW format in x amount of time.

I wonder how an example from the article like the API to post new listings to Amazon works in practice with the requirement to be designed to be open to outside developers. It seems like that’d force some sort of review process (and I’m not really sure who can review all new listings) between API call and public availability that might not be there if you eg. had a private API for approved employees.

I spent 2020 at AWS and I can&#x27;t think of a single part of their toolchain that exposed APIs as first-class citizens. Not Pipelines, not Apollo, not CRUX, not Brazil, not MCM.<p>Its all command line applications and browser interfaces (the stuff first-year developers are most familiar with building, I suppose). I was familiar with this memo so it was quite a shock. Wish they&#x27;d taken their own advice.

This is the way I tend to work. I even write about it[0] (“Keeping Things Vague”).<p>APIs (and opaque modules, in general) are key to the way I develop software. It works well.<p>(0) <a href="https:&#x2F;&#x2F;littlegreenviper.com&#x2F;miscellany&#x2F;evolutionary-design-specification&#x2F;#vague" rel="nofollow">https:&#x2F;&#x2F;littlegreenviper.com&#x2F;miscellany&#x2F;evolutionary-design-...</a>

I don’t think Yegge is a very reliable source in general, even though he was there at the time. I’m not sure why his version of these events is taken as gospel everywhere. I never saw Bezos concern himself with technical matters while I was at Amazon (though I was in AWS, which he seemed happy to leave to ajassy).

&gt; 6. Anyone who doesn’t do this will be fired.<p>Sounds about right<p>Azon turned throwaway compute instances and one-click 2 day delivery ecom into huge cash cows.<p>They aren&#x27;t some golden example of how to properly design infrastructure, it just worked out for them. luck&#x2F;timing&#x2F;reinvestment had a lot to do with it.

It is exactly what author Cal Newport is recommending in his new book &quot;A World Without Email&quot; (March 2021): replacing the ping-pong of emails and meetings by standardized processes and requests via tools. It&#x27;s simply incredible to see that Bezos saw this coming 20 years ago!

I&#x27;ve always wondered how far this extends: &quot;All teams will henceforth expose their data and functionality through service interfaces.&quot;<p>Does it include e.g. the team writing the on-device text renderer for a Kindle? What would such a service interface look like?

&gt; no direct reads of another team’s data store<p>This reminds me of how App Store used to ban game emulators interpreting ROMs downloaded from the internet, while at the same time allowing XML parsing.<p>Does it really matter if the interface you&#x27;re talking to over the network is code written by another team, or data written by another team? I suppose implementation details are often hidden away in data stores, but does it have to be that way?

this is a post breaking down a pseudo-mythological memo from 2002 at a company where things have obviously changed and evolved, and there are multiple upvoted comments in here from amazon people saying this isn&#x27;t the case or things have changed etc.

Or: how to waste cycles and make your codebase run slow as shit on a networked computer cluster.

&gt; Anyone who doesn’t do this will be fired. &gt; Thank you; have a nice day!<p>What a colossal twat.

Like all the best myth making propaganda this has some sense and truth to it

Have this mandate in Google<p>You&#x27;ll have every engineer complaining and stop working to fight their <i>freedom</i> and finally the change has to be reverted. And 5 years later, oh my god, Amazon is doing that, we need to move to that direction as well...

Deleted as I misread the year in the article.

So Amazon has no teams that write simple libraries for other teams?<p>I imagine now that somewhere in amazon there is a &quot;qsort&quot; REST API that does all the sorting.