Normally EVERYTHING has both economies and diseconomies of scale.<p>You model the price per unit as the sum of different curves.<p>Complexity not only increases on Software, but if you design a thermal engine, or a plane, or a car.<p>Working making something as simple as fiberglass, we had something like 100 components, like tensioactives. Most of them we had no idea what they were for, as they were added like decades ago by someone who new.<p>Nobody wanted to remove a given component and be responsible for the fiber breaking and stopping the line, incurring on tens of thousands of dollars in penalties, so new complexity was added, but not removed.<p>In my experience, software is the thing in which YOU CAN get the most economies of scale possible, because you do not depend of the physics of the world. But you need to control complexity as you develop.<p>In the real world, you create a box because it is the only way of doing something, and the box automatically encloses everything that is inside. You can't see inside, nor want to. It is a black box that abstracts your problems away.<p>In software you have to create the boxes. Most people don't do it, with nefarious consequences.
Bootloaders are small, but very important software.<p>k/q is small but a very useful interpreter.<p>There are so many examples, but it appears that to "the market" the most valued software development is large scale.<p>The sentiment is create and contribute to large projects or go home. Stupid, but true.<p>"Do one thing well" is more than just a UNIX philosophy. It is an essential truth. Most programs are lucky if they can do one thing "well". How many so-called "engineers" are afraid to write small, trivial programs lest they be laughed at?<p>Large programs often become liabilities. Can we say the same for small programs? If it happens, write a new one.<p>Maybe a user with an unmet need would rather have a program that does the one thing they want as opposed to one program that can allegedly do everything... whereby they are granted their wish through addition of "features". More internal complexity. And majority of users only using a fraction of the program's feature set. Waste.
Software has economies of scale in distribution. In fact the economies of scale of software are the key point of how software businesses are causing disruption. A single software program can be replicated infinitely at zero cost and allow anybody who has 1 liter of milk to have 1000 liters of milk at no additional cost. So in the author's example, software would be the same price for both 1 and 2 liters.<p>Complexity is something completely different and is well known in all products. I can design a calculator that adds numbers very easily. A calculator that does fractions is much harder to design and costs more. A car with a more complicated engine is much harder to build than a simple engine. This has nothing to do with the actual economies of scale of the calculator or car or you could say that cars have dis-economies of scale too - and obviously they don't. They're the poster child for economies of scale.<p>Building a truck that is 10km long is worse than building 100 trucks that are each 100m long, but this has nothing to do with 'diseconomies of scale' inherent in trucks.
In software you pay for complexity. Big software is more complex than small software (by definition!) so it's more expensive.<p>However, managing systems of small software also incurs complexity, the smaller the software components the harder you have to work to make them play together.<p>It's often not clear a priori whether it's worth to pay a lot more up front to get a monolithic solution or to try and glue together many simple tools.
As RyanZAG says, all production has diseconomies of scale in unit complexity. The production process has economies of scale, meaning that churning out more units of equivalent complexity reduces the marginal cost of churning out the next unit of equivalent complexity.<p>Software exhibits this same economy of scale in production. Take Google's machine learning platform. They allow multiple functional teams to churn out roughly-equivalently-complex machine learning-powered widgets in less and less time. Contrast that with a startup building a single machine learning-powered widget and the marginal cost to Google is significantly lower.
It's a difficult analogy to make, in particular when you forget to consider the ocean of milk underneath you from all the libraries and frameworks you are using. Then the difference between 4 small bottles or 1 big bottle seems less significant.
The best part of the article is the concrete image of the milk cartons. On first seeing the image, your mind is going to tend to think things ought to be one way. Then it comes out and says, "No, it is the opposite." That creates a bit of cognitive dissonance and makes one ask: "Wait, why?" This is good as far as software goes, because it is so abstract that often the brain is not fully engaged when talking about it. It is too easy to know something in the abstract, but then not know it enough to apply it in the concrete.
Every time software economies of scale come up, I can't help but be reminded of Jira's pricing model (<a href="https://www.atlassian.com/software/jira/pricing?tab=host-in-the-cloud" rel="nofollow">https://www.atlassian.com/software/jira/pricing?tab=host-in-...</a>):<p><pre><code> (Per month:)
Users Total Per user
----- ------- --------
1 $10 $10
5 $10 $2
10 $10 $1
15 $75 $5
25 $150 $6
50 $300 $6
100 $450 $5
500 $750 $2
2000 $1500 $1</code></pre>
Diseconomies of scale apply in wholesale finance too. Put a big order on a limit order book, and you'll exhaust the liquidity and move prices against yourself. Dealers usually offer wider spreads for large trades as they'll have to work off a large position afterwards, and they need compensating for taking the risk of being on the wrong side of a major price move while they have the position.
Diseconomies of scale certainly aren't unique to software, and the author sensibly notes "individual knowledge".<p>One of the main effects of protectionist and interventionist policies has been related to them. A domestic firm starts to rot, unemployment prospects are rising and a sense of national preservation starts to set in. Thus, in the short term, tariffs are levied, subsidies are made and some macro notion of "stability" or "optimality" is reached. The long term costs are the artificial delaying of the onsets of diseconomies of scale with state and business expansion leading to symbiotic interests. Then people complain about Big Business fucking them over.<p>(The fact that the author quote Keynes makes this all the more ironic. Keynes-the-man wasn't objectionable, but the neoclassical synthesis/"pop Keynesianism" of his disciples Paul Samuelson and John Hicks did influence government policy in a negative way, as noted in James M. Buchanan's <i>Democracy in Deficit</i>.)
People don't think software has economies of scale. The amount of articles I've seen about the "mythical man month" and such all talk about how hard software is to scale.<p>What people do think is that the marginal cost of reproducing software is basically zero, regardless of size. This means that choosing between two products, if product 1 has n amount of features, and product 2 has those same exact n features plus an additional feature, all consumers will rationally choose product 2 (lots of assumptions, i know).<p>This is why companies try to get bigger because if they can offer more features, than all the consumers will choose them and they get all the sales. One could argue that this is the reason why the "power law" effect thats been talked about on HN recently happens.
I get the point the article is trying to convey. Scale increases organizational complexity and overhead relatively more than the added manpower contributes. However, the analogy with the 'milk' is far off. First of all, since the 'duplication' cost of software is near 0, buying a single seat or license is more expensive than buying in bulk. With a few exceptions, this can be established by browsing any product or SaaS website. Second, but this is minor, in retail vendors often abuse this expectation pattern and have now started to charge more per volume for the larger packages.
The production side of software is more like R&D, and there you find the diminishing returns, as iconified in DeMarco's 'the mythical man-month'.
There are many kinds of scale.<p>Poor performance on military projects is often an issue of huge development costs spread out over a tiny number of units.<p>Apple spends as much to develop an iPhone as it costs to develop a new weapon system, except they sell millions of the phones so the unit cost works out ok.
It depends on your point of view.<p>The point of software is to deliver value to the business. There's overhead with supporting and integrating each system -- to borrow an analogy from the article, each milk carton needs cardboard, a date stamp, etc. Even if software development productivity drops 75% and delivery cost increases, having one big carton of milk may be more cost effective than supporting 50 smaller, more nimble cartons.<p>If you want evidence that this exists, consider that SAP and PeopleSoft exist and are thriving businesses. Or that the general ledger of most big financial institutions are running on mainframes with code that's been in production for 30 or more years.
If you had to build a windows GUI from assembly code, almost all software projects would be too expensive. Instead we reuse high level languages and frameworks to start with the basics a program needs.<p>To extend the metaphor to milk, what if the milk industry had to invent the glass industry in order to make the bottles which it comes delivered in? Consumers would have cows not refrigerators.<p>The dis-economies-of-scale-software are programs where normal glass simply can't be used to hold the milk. A whole new custom type of glass has to be developed. And this usually for a type of milk only like 1,000 people even drink it.
From a "Computer Science" perspective: "Economies of scale" is another word for "sublinear growth". Software is, fundamentally, a graph. And the number of edges in a connected graph grows quadratically.<p>Pretty much any strategy to improve making software at scale, whether code organization or organizational design, is finding ways to limit the complexity of the graph to a constant multiplier of the number of nodes, and keeping that constant small, rather than allowing things to grow quadratically.
"Much of our market economy operates on the assumption that when you buy/spend more you get more per unit of spending."<p>Supply and Demand says the opposite. The supply curve slopes upward, meaning that a higher per-unit price is required when the aggregate supply is higher.<p>Economies of scale apply in some situations, but people generally place way too much weight on them.
> And if you don’t know, the UK is a proudly bi-measurement country. Countries like Canada, The Netherlands and Switzerland teach their people to speak two languages. In the UK we teach our people to use two systems of measurement!<p>The Netherlands? We only speak Dutch here. :-)<p>I guess the author means Belgium, where they speak (at least) two languages: Vlaams and French.
Software has diseconomies of scale, but also has economies of scale.<p>For example, because of context switching: when a developer makes one change it can be pretty easy for them to add another change (everything is already "open").<p>Other comments here mention distribution and combining small simple tools for something larger.
>Finally, I increasingly wonder where else diseconomies of scale rule? They can’t be unique to software development. In my more fanciful moments I wonder if diseconomies of scale are the norm in all knowledge work.<p>The pop music industry seems to fit the bill.
Article seems to be conflating fixed costs with variable costs; development with production. Takes more labor to set up a plant to manufacture a million Model-Ts than building one horseless carriage in a barn.
It is actually a lot worse than that article suggests. There is a history of big government software projects which proved practically impossible to complete on time and on budget or to get them working at all.
> Four, get good at working in the small, optimise your processes, tool, approaches to do lots of small things rather than a few big things.<p>Why, I think I've heard that before...<p>"Do One Thing and Do It Well" from <a href="https://en.wikipedia.org/wiki/Unix_philosophy" rel="nofollow">https://en.wikipedia.org/wiki/Unix_philosophy</a><p>Edit: Strange post to be down voted on. It was an interesting connection to me.
Well I made a command-line Go program that processed a 10+GB XML file, made some computations, and put it in a JSON Redis structure. It had zero bugs, with months of development. I made it myself, so you can have zero bugs if you do things yourself. The point where you have to work with others results in bugs.
To be fair, the first few developers are often more than 1x multipliers. But you definitely reach a team size where additional developers have decreasing marginal value pretty quickly.
"Loose coupling" is one of the basic principles they tell you in beginner's courses. This is common knowledge to the point of being cliche. This encourages coming up with good boundaries, interfaces, whose both sides can independently vary.<p>Choosing these separation points is not trivial, by the way. It usually requires many iterations and a good gut feeling for what future requirements may arise. One example that is frequently mentioned is content vs form, for example website content and its presentation. It's useful, but still strongly depends on the actual uses. What works on screens doesn't necessarily work on paper. Not all forms can accommodate any kind of content. In other words the content also needs to fit the form. Or see how the layered model of the TPC/IP or the ISO-OSI protocols actually isn't that principled in practice. A high-level protocol may want to adjust its workings depending on what lower-level protocol it is running over.<p>So decoupling is not trivial. It often reduces performance too. Say you want to chain two tools sequentially. What if you end up using a specific combination 90% of the time, but there is some intermediate step in the first component that calculates something that is thrown away and the second component needs to recompute it because the general interface between the two doesn't include passing that data along? Then you can either accept it, or tightly couple them, OR <i>even worse</i>: you may try to find the real pure elegant reason for why this piece of data is actually logically required there and then you end up with abstractions that only have one instantiation, just for the sake of not "hard-coding" things.<p>But anyway, back to the core issue. There is not much to be surprised about.<p>Complexity of system = Complexity of individual components + Complexity of their interactions<p>This is analogous to the decomposition of the variance of a set of numbers: total variance = variance inside groups + variance between different groups. There is also some physical analogy for calculating energies, where you need to take couplings into account.<p>If you have a big system you need to take care of lots of interactions. If you choose to use small independent tools, then <i>you</i> need to write the code for their interaction.<p>Should the components come from the same vendor or different ones? There are arguments for both. Use same: they tested all sorts of interactions and fixed any bugs that they found (Apple supporters often point to this). Use different: each component only cares about its job, so less bugs in them, clear responsibilities etc. Certainly, if the same vendor develops all components of the system, they don't have to be so conceptually clean and they may make up for bugs in one subsystem by quirky code that tries to cancel out the bug's effects in some other subsystem, leading to fragile software.
The argument the author makes is really that software <i>development and maintenance</i> has diseconomies with the scale of projects and releases (basically that development and maintenance output needed scales superlinearly with complexity and output scales sublinearly with team size), which seem to be fairly widely.accepted observations in the field.<p>There some effort to portray this as unusual compared to other industries through a direct comparison to retail costs of larger grocery goods and manufacturing economies of scale, but that's somewhat missing the point. Product development and engineering probably faces similar diseconomies in non-software domains (the same complexity issues and human factors issues that effect software development are present) and, OTOH, actually delivering units of identical software (or services provided via software in the SaaS world) have similar (perhaps more extreme in some cases) economies of scale as are seen in many areas of manufacturing, as the marginal costs are low and more units means that the fixed costs divided by units sold goes down.
It's not a brilliant article.<p>Software is not like milk. That analogy is facile and stupid.<p>Software <i>should</i> be more like civil engineering, where it's normal to unleash a big team on a big infrastructure project and still have some hope that costs and deadlines stay under control. Or maybe like movie making where there's a cast of thousands, the time is huge, and the costs are epic, but some projects stay under control - while others don't.<p>It's maybe more interesting to wonder what's different about software than to look for enlightenment on supermarket shelves. Because the problems stated - multiple communication channels, mistakes in modelling and testing - are handled just fine in other industries.<p>The crippling issues are that you can't model software, and there's not much of a culture of formal specification.<p>So you can't test software until you build it, requirements may change iteratively, the latest technical "solutions" often turn out to be short-lived fads, and you're always balancing between Shiny New Thing and Tarpit of Technical Debt. <i>That's</i> why it's hard to build. You have to build your cathedral to see if it stays up when it rains. You can't simulate it first. And even if it stays up it may be the wrong shape, or in the wrong place.<p>It doesn't help that management often sees software as a cost centre instead of an engine room, and doesn't want to pay a realistic rate for quality, maintainability, and good internal documentation.<p>Having too many people on a project is not the problem. The problem is more usually having no idea what you're doing, why you're doing it, or how you want it done - but believing that you can throw Agile or Six Sigma (etc) at it to make it work anyway, because Management Theory.