Yegge's post was very interesting reading, and I took similar learnings away from it. I was at Amazon at the time, however, and there were things that certainly weren't true any more:<p>>3) There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network.<p>API first... except if you want to be a number of certain new services that somehow managed to get away with not presenting an API, even though an API would make every service team's life easier.<p>> 5) All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.<p>Except, similar to above, where teams apparently decided they didn't want to think that way at all and management just let them.<p>> 6) Anyone who doesn’t do this will be fired.<p>Unless your exception is perceived providing value to the company. Then you'll get lauded, and everyone is told they'll need to use your js laden, web only interface, and to hell with any automation.<p>Mostly those exceptions just codified further in my mind about just how right the Bezos email Yegge paraphrased actually was.
This was not exactly a Jeff Bezos mandate but the result of an engineering brainstorm. The mandate came more out of a "how to scale Amazon for the next decade" discussion. In large companies, one where distributed/independent teams are as important as distributed systems this ended up being the only way to operate.<p>Initially, during the good old days of Amazon, there was what you'd call a single datawarehouse. It made sense initially for every system that processed an order to access the data by querying that data warehouse—this meant that the processes would be distributed (different services), while the data would be centralized. It also meant that any change to the way the data is stored in the datawarehouse meant deploying code to a hundred places.<p>The most important problem that this addressed was however different. A centralized datawarehouse meant that every customer request bubbled up into N queries to the datawarehouse (where N is the number of services that needed access to the data—billing, ordering, tracking...).<p>The mandate summarized in one line would be this—"the data is the one that should go to the services, not the other way round." Voila, microservices.
Hi! I’m a senior engineer at Amazon. Throwaway account but I’ll try to respond to questions if anyone cares to ask.<p>Yeah we use services heavily, but there’s plenty of teams dumping data to S3 or using a data lake.<p>There’s also the “we need to do this but management doesn’t see value so let’s dump it on the intern or SDE 1, who we won’t really mentor or guide and then blame, forcing them to switch teams as soon as they can.”<p>If you work at another company and think we have our stuff figured out at Amazon, we really don’t. We have brilliant people, many of who are straight up assholes who will throw you under the bus. We have people who are kind and will help you gain all kinds of engineering skills. We also have people who are scum of the Earth shit people who work at Amazon because I don’t think any other sane company or workplace would tolerate them. We have extremes on the garbage people end of the spectrum, unfortunately.<p>Sorry long rant - point being - it’s good to learn how we do things. The internal email on services is pretty unique. I learned about it when I was an SDE 1 back in the day. But - don’t take it as gospel. It doesn’t mean you need to build services.<p>I can think of any number of examples where we follow anti patterns because no one gives a shit about the pattern, whether it’s a service, a bucket, a queue, or a file attached to the system used for scrum tasks, or shit passed over email... we care about value at the end of the day. If you don’t provide sufficient value at Amazons bar, they have no problem tossing you out the window.
> While the third point makes all the difference in the world, what Amazon really did get right that Google didn’t was an internal communication system designed to make all the rest possible.<p>> Having teams acting like individual APIs and interacting with one another through interfaces over the network was the catalyst of a series of consequent actions that eventually made possible the realization of AWS in a way that couldn’t have been possible otherwise.<p>Google has worked this way since time immemorial. That’s what protocol buffers are for: to create services and pass data between them using well defined interfaces.
I worked at an organization that had a similar declaration. Here's how it played out:<p>1. Everyone is super excited for other teams to share their data<p>2. Everyone wants an exception from sharing their own data because it's too hard or too sensitive to share.<p>3. Eventually everything gets shared, but it takes 3-4 times longer than it really should.
Something like Conway’s Law was also recently cited by Elon Musk (jump to 3:30)
<a href="https://www.reddit.com/r/SpaceXLounge/comments/dbttaw/everyday_astronaut_a_conversation_with_elon_musk/" rel="nofollow">https://www.reddit.com/r/SpaceXLounge/comments/dbttaw/everyd...</a>
Found the original post from Yegge a really interesting and thought provoking read. Didn’t realise from the context that he originally accidentally posted it as a public rather than private Google+ post!<p>His follow up post explaining this, and with an interesting anecdote about presenting to Jeff Bezos, is archived here (seeing as G+ has, ironically (or not) given the context, shut down): <a href="https://gist.github.com/dexterous/1383377#file-the-post-retraction-message" rel="nofollow">https://gist.github.com/dexterous/1383377#file-the-post-retr...</a>
At least as of 3 years ago when I left, the software systems that drove the mandate towards SOA were still massive systems that communicated almost purely through a monolithic Oracle database. It was the software system(s) that was responsible for all automation and accounting at fulfillment centers. This is one of those rare times where I actually think a full rewrite from scratch would have been a better idea.
I wish the actual body of the email was available and published. I’ve only read Yegge’s account of the note and didn’t see it in any of Bezos’ books.<p>I suppose it’s nice that the email, or really any amazon emails, has not been leaked.
> 6) Anyone who doesn’t do this will be fired.<p>So I've never worked at a company over 150 people. Is this... a normal thing for an email? Maybe I'm just one of those softies but an email with that line would throw me off my day and cause a serious hit to my morale and confidence of working there.
The article mentions this as dog fooding, but does that really apply here? Did they do this with the idea in mind that they'd turn this stuff into a product? It struck me as Bezos wanting things built for the future, reducing technical debt, and the product-ification was an excellent byproduct, but perhaps not intentional.
Eat your own dogfood.<p>You can't sell to customers effectively if your flagship product only works because it has access to resources the customers will never have... and it is designed around that flagship's needs and not your customer's needs.
Yegge's article never says it was an email. What should the title be?<p>Edit: I've taken a rather lame crack at it and am open to improvements.
A lot of interesting thoughts here but the author doesn't really wrap them into a conclusion. A whole lot of words to say "they all work and it depends".
I love some of Amazon's executive policies. From what I've read, everyone has to write a multi-page paper before executive meetings, and everyone has to read it, so the meeting goes smoothly with everyone understanding the issues. I hate how no one reads anything in most organizations.
> what Amazon really did get right that Google didn’t was an internal communication system designed to make all the rest possible.<p>I'm not following what he means. What is the thing he is describing as "an internal communication system" here? That made all the rest possible? What is/was this internal communications system?
The issue if you are developing using such requirements is that the product will end-up quite expensive. A simple messaging or authentication feature becomes a fully fledged multi-tier service maybe with super admins, owners/admins and clients. Dev budget is not an issue for Amazon though...
A few things I'd add today:<p>* Every service must provide latency and error-rate metrics.<p>* Every service must be capable of generating and/or responding to backpressure when things become overloaded.<p>* Every service must be prepared to support multitenancy.
The thing to point out is Bezos is a real techie, and while any business guy would have built amazon on top of msft or google cloud, the fact that he knows about infrastructure made it possible for Amazon to build AWS
Why does the title of this keep getting flopped around? It's shifted three or four times today. I thought it was supposed to be the title, or the subtitle, and avoid paraphrasing.
> 6) Anyone who doesn’t do this will be fired.<p>I would have so much loved this approach in the last corporate job I had. It would have changed so many things in such a short time...
here's an article about how the idea of AWS came about. the main takeaway is that it evolved and the article has a lot of 'we' in it, not only 'jeff'<p><a href="https://techcrunch.com/2016/07/02/andy-jassys-brief-history-of-the-genesis-of-aws/" rel="nofollow">https://techcrunch.com/2016/07/02/andy-jassys-brief-history-...</a>
We have robotic baristas here in SF, but no one uses them. Why? People want to have their food prepared and served by a real human being, in most cases. The food tastes better when it's served to you by a real person.
If you use an adblocker like uBlock Origin, you can add the following rule:
news.ycombinator.com##.pagetop<p>Unfortunately it removes ALL of the top navbar but I've found it really useful to get around HN's damaging and useless gamification metric.