Nice, interesting article.<p>But I would stress less that "Seeing Like a State" -- that is a top-down, global solution -- was not the problem.<p>The problem was that "Tim" didn't really understand the problem he was trying to solve (well, none of us truly understand very much at all, but he didn't understand it better than many of the teams associated with the individual services).<p>"Tim"'s proposal probably solved some problems but created various other problems.<p>The best solution, though, (IMO) isn't that Tim should be smarted and better informed than everyone else combined, nor that every team should continue to create an independent solution. Instead "Tim" could propose a solution, and the 100 micro service teams would be tasked with responding constructively. Iterations would ensue. You still really, really need "Tim", though, because multiple teams, even sincere and proficient ones, will not arrive at a coherent solution without leadership/direction.<p>> A global solution, by necessity, has to ignore local conditions.<p>That's just flat wrong. A global solution can solve global concerns and also allow for local conditions.
For anyone interested in social systems that help avoid this top-down, centralized failure mode, I cannot recommend RFC 7282 enough:<p><a href="https://datatracker.ietf.org/doc/html/rfc7282" rel="nofollow">https://datatracker.ietf.org/doc/html/rfc7282</a><p>A whole lot of wisdom is captured in that document, including a deep understanding of the differences between unanimity, majority rule, and consensus.<p>If you're involved in standardization efforts in any way, whether it's deciding where your team will put braces in source code or running software architecture for a Fortune 100, it will well repay your reading time.
This seems to be hallmark of a “Middle” developer. Not so junior that they couldn’t build a working solution that they assume everyone should use, but not senior enough to think twice about whether they should be building it.<p>The “we should make a common framework” for this line is the dominant thought at this level. Never even a library. A framework. Everyone must do it this way.<p>The more senior people share concepts and maybe libraries, and allow the team to use them if they see fit.
This article does a good job describing one failure mode that's not understood well, but the opposite failure mode is much more common in my experience- having lots of ways to do the same thing can be very inefficient and brittle, even at small companies. The right answer is not "never unify systems" or "always unify systems", but develop judgement about when things should be unified.
Lots of resonant points here. It’s worth making it to the end.<p>I work at a company where there’s a number of different little less-than-one-man projects, and there’s a lot of variety, and so a couple of non-tech types, frustrated with resource allocation (having the right kind of skills at the right place at the right time in the right amount) wants to standardize and simplify.<p>What I’ve observed though is that when you tell your house painters they can only work with black paint, they can only give your customers black walls, and when your customer wants wood panel, or textured fuschia, then you can’t earn revenue from that market demand.
In general, "unity" is something software developers routinely pursue just for the sake of unity itself, failing to understand that unity comes with significant tradeoffs. It is much harder to build a unified solution than a localized, one-off solution. Divide-and-conquer is often a much better engineering strategy: DAC might create more work than unity, but the work is more likely to succeed instead of falling apart because we failed to anticipate all the use cases within the unified framework, especially when we lack experience in the domain.<p>Also refer to Jeff Atwood's Rule of Threes (which he borrowed from someone else) here.
I've noticed that ALL beginners seem to have a reinvented global solution phase.<p>Everyone who does electronics might say "Oh I'm going to use this one connector for everything". And it's either ok, if it's a standard connector, or a giant pile of crap that means they can't use a lot of existing stuff because they insisted on this insane DIY grand scheme.<p>Usually such things have an element of "I want to do Y, so I'll build a modular kit X and use that to Y". And then X becomes the real project and Y is never finished.<p>The insidious part is how the new product is often a tiny bit better than what's out there. But it doesn't matter. The mediocre standard solution is still way less trouble than the beautiful perfect custom thing. I'd rather have Just Works tech than tech that's Just Right. Anything that seems perfect and beautiful and simple, I don't trust, because it was probably made for one specific task, not to be a general standard you don't have to think about.<p>I think of the failures with global solutions are because someone did them on a small scale, or because they have to do with natural systems.<p>Fully top down planning of manmade things by a giant industry consortium is most of why tech is great. Otherwise we would have no USB C, and 12 different CPU architectures.<p>Sometimes design by comittee protocols suck, but usually because they didn't have <i>enough</i> control, and instead of a protocol, they deliver a description language for companies to make their own protocol, with every feature optional so that compliance does not necessarily mean compatibility.<p>When you do it internally it can suck because it's more effort than it's worth to replace all your existing stuff.
Counter example: Tom standardized a bunch of services... and it worked! Everything is easier and more efficient now.<p>I agree with the thrust of this post: Changing something that is not understood is a dubious undertaking. But the author fails to make a compelling connection between the above and software development. A poor solution <i>may</i> be a result of not understanding enough of the system as a whole, or it <i>may not</i>. We simply can't tell.<p>Standardization (i.e. simplification) is generally a good thing in software development. How would Tim's system look if they had opted for his approach from the start? How does the 3rd iteration of the system compare to the 1st iteration? Maybe Tim's solution is stepping-stone to something better. Impossible to tell.
People are similar to water: they will often the path of least resistance.<p>The trick is to find a solution, document its “shape”, make it easy to integrate and market the hell out of it internally. Then you let the market decide.<p>Building a big shared common library can be a mistake, but that’s not because it’s intrinsically the wrong choice, it’s impact is partly a function of how many resources you can dedicate to effectively designing and maintaining it. At a certain scale the economics of this suddenly flips.<p>The problem in the post seems to scream infrastructure rather than code. Identify the different types of queues services need, pick some off the shelf and preferably managed solutions, then make it take 5 minutes to get started.
As a whole the strategy of "Let's see what's common in all these systems" is a good start to understanding the systems. There is a limit to the complexity any single person can understand. Unification is simplification. It helps understanding. But I agree it is no good trying to make the landscape fit a simple map when reality is much more complex. There's no Silver Bullet in trying to combat complexity.<p>But rather than trying to unify everything think about micro-services. Each service can be its own isolated solution. Of course it needs to be optimized in terms of how well it works when all the other services are running as well. But I think isolation is the key to independently optimizing everything.<p>I like this sentence from the article: "Lots of it doesn’t matter, but some of it matters a lot".
The article seems a bit defeatist to me. When I create an Interface and have a few implementations for it, I've created a standardization, most people would agree it's a big improvement in code reuse and maintainability to leverage interfaces over just having a bunch of one time use concrete classes.<p>If you think of the United States, you might argue having a central government was better than each state being its own country and maybe that's the edge the US has over Europe.<p>Deciding to build roads everywhere top down definitely helped with overall car transportation.<p>Having HDMI as a common format for streaming video and audio sources is a big improvement over each TV using its own format.<p>There's plenty of counterexample to what the article talks about. So what gives? And on the example mentioned, how do you know if that's a great use of standardization or not?<p>I've been in many companies and I think they often fail to invest enough in frameworks and standards across the tech side. AWS was born out of Amazon's own effort to do top down standardization for example. And now it's their biggest cash cow and the entire industry is standardizing over its services.<p>Way too often I see people stuck on contract like work. Each locally scoped small problem needs its own locally scoped big project to be handled. That just simply doesn't scale.<p>The beauty of most engineering is exactly finding these top-down mechanism that do scale, even if it involves changing the business processes themselves. Think of a dishwasher for example, a top-down design that can wash most things but not all, eventually things that are not dishwasher safe became less and less popular and almost extinct, because people want to scale their efficiency.<p>Can top-down standards fail, ya sometimes, but when they succeed they take things to the next level of scale.<p>Don't shoehorn every problem in the same solution, but also, don't solve every solution independently of one another and reinvent the same wheel, or your engineering team of 10 will quickly become thousands while your business product will have barely grown.
I haven't read Seeing Like A State. I have to say, i am <i>extremely</i> skeptical about the author's fable about forestry. Commercial foresters today still mostly use monocultures with evenly-spaced planting patterns, and i simply don't believe that they would be doing that if there was a straightforwardly better way to grow trees, even it was less "legible". This has a powerful scent of Gladwell-esque insight porn - the sort of story we love because it's counter-intuitive and makes us feel cleverer than people who haven't heard it.<p>I don't suppose we have any foresters on this board who can comment?
Some good points. At the same time, sometimes standardization is good and necessary. Imagine if everyone had to reimplement TCP/IP analogues to communicate over a network; we'd never get anything done!
The first part of the article describes the common situation where we see similarities across projects and effort being duplicated, apparently unnecessarily.<p>To improve understanding and efficiency, we come up with proposals that involve some kind of standardization, such as a shared library abstracting a service.<p>My team has recently embarked on such an effort, and I was really hoping for a new take on how to avoid the various pitfalls involved in attempting this kind of standardization.<p>Obviously, central planning works quite well for certain things and, just as obviously, it hasn't worked well for other things. In most situations, it seems that a combination of planning + flexibility works best.<p>Further, what works or doesn't work for nation-states isn't particularly applicable to software architecture. The analogy feels quite tortured to me.<p>As to the example given in the article regarding trees, one could just as easily choose an example from modern agriculture where "central planning" seems to work quite well.<p>In the end, I feel like the article just boils down to: "Make sure you understand the problem domain and the cost of large scale change before you spend a lot of time and effort making said changes."
I disagree with the conclusion although not necessarily with the situation described.<p>A programmer only has so many hours in the day; if you want to be a more efficient programmer you either have to learn to type/think faster or you need to build frameworks, write libraries, and codify common practices.<p>There are situations where that doesn't work but if your job is to pump out code for an organization there's a good chance that most of your applications will have the same authentication, the same look and feel, etc. Putting effort into that core will pay dividends later. But you can't be a slave to your own code; if it doesn't fit in the box then don't force it.
While there is much to worry about over centralization, there are benefits to at least trying if you can consolidate n-m efforts when n is the total number efforts using a queue and m are the snowflakes. It all depends on organization size and whether some costs can be amortized.<p>Everyone having specialized everything is exceptionally expensive at scale. Here, the key is tower of babel. What Tim should have done is kick start the effort with one or two teams, then organically grown it by reducing operational burden for new teams.<p>The mistake here is trying to globally optimize rather than seeking different vertical consolidation.<p>By the way, this is why Amazon has so many services.
I expect "Seeing Like a State" goes into it, but top-down global abstraction and standardization often does work. e.g. mass production, mass media, and States themselves are unbelievably, fantastically successful.<p>The point is more than it doesn't <i>always</i> work, as standardization is a poor substitute for actual understanding.
This article is amazing and is perfectly timed for the news about Sri Lanka's mass move to organic farming - <a href="https://news.ycombinator.com/item?id=28443798" rel="nofollow">https://news.ycombinator.com/item?id=28443798</a>
I thought it was interesting that an article that talks about the dangers of generalizing (aka standardizing) based on a few data points is, itself, generalizing based on a few data points.
Great short read and book recc.<p>I find myself often trying to articulate this idea (sometimes to myself, often to a friend) - starting with the map and building a territory from it --> generally bad.<p>While reading this article I wasn't even thinking about software. I think of a lot of progressive and/or socialist politics. To assert that the world should look as you one sees fit is flawed, both because one is not representative but mostly because the complexity of a system beyond toy example cannot be accurately modeled enough when the risk is catastrophe.