> In my career, I have taken a particular approach based on building new systems from scratch (before they succumb to the three laws), but this is a lot harder than it sounds<p>This seems like an infeasible solution in most if not all real world cases I've seen. Especially if you have a system with 15 years of legacy and some thousands of man-years of effort put into them.<p>Now, the obvious "don't let it get to that point" is a smug IT nerd answer I often see regarding this, but this seems like the kind of green-field startup thinking that doesn't apply to 90+% of software development. We have an existing system. This needs to be maintained and extended because it's our core business. We cannot just say "oh it's become too complex. Let's go and spend the next few years creating a new product, so we'll have a multi-year feature freeze".<p>The <i>only</i> feasible way I see to do anything even close to this is to split up the software into domain-like silos slowly and incrementally. At that point you can start <i>considering</i> rewriting silos as they become too complex/cumbersome/big. However, at that point you're already doing complex, lengthy and generally risky refactoring, and speaking of it being a write from scratch is very obviously not true.
These are definitely not laws but consequences of bad engineering culture.<p>I’ve worked on well designed systems that stayed well designed. For example one I remember that was very solid: How? The team had enough support and autonomy from management to:<p>1) allow us to hire the right people (no secret sauce - just a mix of not too fast growth and a healthy mix of seniors and juniors)<p>2) not rush new features, and spend enough time on design and tech debt.<p>It probably helped that the system was a core e-commerce platform. If it descended into chaos it could have had a material negative impact on the top line.<p>When reading articles like this positing “laws” remember many people live in a bubble and may have only worked in dysfunctional orgs. There are tons of counterfactuals out there when the system and complexity was critical to the business. (And also tons of examples where despite that it still descended into chaos- but that isn’t a law)
> In my career, I have taken a particular approach based on building new systems from scratch<p>= Software consultants and free lancers. I don't like them because I am jealous that they get to do greenfield projects and make all the architectural decisions but never actually have to maintain their own creations (and therefore never really learn how to design a system for changeability and maintainability).<p>> What can we do about this state of affairs?<p>1st: be aware of these laws and that keeping a 20 year old system in production is a non-trivial task and to be proud of every additional year I can keep it running.<p>2nd: Instead of seeing it as a 'DoS attack on myself by previous developers', I try to see it as a neverending Sudoku that challenges and stimulates and pays me every day.
I think it is important to discuss the notion of accidental complexity and essential complexity here. If your organization's strength is that you have world class engineering essential complexity is your friend: a problem domain with big essential complexity is really a moat: it keeps the barrier to entry into the market high. If there were less essential complexity in the world there would be much less money in software engineering and much less software engineer jobs would exist. Case in point: markets where barrier to entry regarding technical complexity become too low degrade into a race for the bottom. (like the flood of indie games that do not make money.) On the other hand accidental complexity is not our friend: if you maintain a system with too much accidental complexity there is a great risk that a smarter competitor could create something at least as good with less resources.
> a well-designed system is an unstable, ephemeral state; whereas a badly designed system is a stable, persistent state<p>It's kind of entropy: I wouldn't say that a badly designed system is "stable", it is just as unstable as a well-designed system and keep evolving, but there are many more ways to be "badly designed" than to be "well designed" so without actively fighting entropy, it'll be unstable but consistently "bad"
I have a counter-take on the first law that I've seen two examples of within my (20 year) career:<p>> <i>A system that inevitably degrades (to a significant degree) over time was badly architected. Most systems aren't badly architected but most systems that *succeed* are. Good architecture requires time (usually an unprofitable amount) & most successful systems prioritise velocity.</i><p>In summary: the demands of profit necessitate bad system design.<p>Of the two examples I've seen in my career: one was by a larger-than-needed/accidentally over-resourced team in a very large corp, who were operating on an undervalued background product with no real OKRs. The second was a government-funded contract (these often skew badly in the opposite direction due to corrupt tender processes but barring that there's often breathing room for quality if there's a will).
I understand software better from words and diagrams than reading the code.<p>Complexity is aided by having sturdy mental models - what you intuitively understand. And to see the truth clearly.<p>To have humility: my code is probably ugly to other people, but I understand my code faster than reading yours.<p>I can't pretend to know that the system I build would be better than yours.<p>Be wary of pride.
First law implies that it is impossible to change a badly designed system into a well-designed system through incremental changes.<p>I disagree with this, it is certainly possible to improve the state of some system without starting from scratch.
Very interesting article. I feel like these are more laws of culture than laws of software per-se. Said in other ways:<p>"It only takes a match to burn down a forest"<p>"One bad apple spoils the bunch"<p>Keeping a system well designed requires good engineering culture (as other people have said), and a great alignment across engineering, management, sales... the whole company.<p>In control systems, you typically handle these situations with some sort of feedback mechanism. Said another way: you need to "feel" the problem if you want to be able to adjust quickly enough.<p>It'd be interesting to know if this kind of "feedback loop" exists for human societies / companies. Or it is just what it is: it exists, but it has a very large delay, which causes the system to eventually collapse and restart...
I think this was more comprehensively described by The (Eight) Laws of Software Evolution: <a href="https://en.wikipedia.org/wiki/Lehman%27s_laws_of_software_evolution" rel="nofollow">https://en.wikipedia.org/wiki/Lehman%27s_laws_of_software_ev...</a><p>This subject always reminds me of something someone said, possibly Professor Alain April, "software is the only system where maintenance tends to degrade it."
I disagree with "a well-designed system is one that is easy to change over time".<p>To me this is the axis of "generic" vs "specialized". As systems get increasingly optimized (for anything: resource usage, bandwidth, latency, cost, etc.) they increasingly need to make choices that limit other potential choices.<p>This has nothing to do with being badly or well designed. A well designed system is one that does it job well, which in part includes being able to evolve it over time (or not). This might includes staying more generic, or becoming more specialized.
17 years ago I built a system to automatically consolidate and pre-processed data from various sources. It saved 20 man-hours a day of effort and downtime. Put in 12 hours of support over the 2 years it stayed under my management, while I took many hits about how "badly" it was designed and how other teams could improve it. Moved on from that role.<p>6 years after left the role, I got a call from a sysadmin. He told me the machine that hosted the system went down but they managed to recover the HDD and restore it into a VM. I was surprised because I thought they had decommissioned it long ago, when the vultures who wanted to pad their KPIs tried to write a better version. He said multiple efforts all failed and no one was able to replicate what that system does. The reason he gave me a call is because after they loaded the HDD into a VM and turned everything back on, the system came back online and was working perfectly without any effort on his part. He was astounded and just wanted to talk to the person who was able to build a system that literally wouldn't die.<p>It wasn't the most elegant or best designed system, but I designed it not to die and that's exactly what I got!
> In my career, I have taken a particular approach based on building new systems from scratch<p>This is often what I see junior programmers do. The author has some experience, much in research, and I’m curious what they practice here and how effective it has been.<p>I don’t see poor design as inevitable myself. It can be hard to battle against the tide of developers with good intentions and wrong incentives. All too often we are pushed to accept fast and cheap over good and well done.<p><i>Update</i>: spelling/grammar.
This is a bizarre piece riddled with odd metaphors and attempts at sounding smarter than it is, e.g. through mathematical parlance.<p>>Let’s say that a system X is well-designed. Someone comes along and changes it – by definition, quickly and easily – to a different system X’. Now X’ either continues to be well-designed; in which case it can be quickly and easily modified again to a different system X’’; or it will enter a badly designed state and hence be difficult to modify.<p>???
An example of a large code base that has survived decades and undergone huge scope changes without running into these problems is the Linux kernel.<p>Initially it only ran on X86 PCs and most of the hardware it supports today didn't exist. Today it runs on many architectures from small embededded systems to super computers.<p>Some of the initial code was pretty rough but the modern stuff is pretty good and cleanups and refactors are done all the time. Indeed long term maintainability is one of the major considerations for maintainers deciding to accept or reject patches.<p>Why does this work for Linux? Because companies are used for funding (the majority of kernel developers now do it as part of their paid jobs) without giving them technical control of quality. The companies paying people to work on the kernel get a say in <i>what</i> is worked on (presumably things important to them) but not about <i>how</i> it is done.
These "laws" are poorly framed. Software "complexity" is a function of the natural or artificial intelligences observing and altering the software system, a corollary of- with enough eyes, all bugs are shallow.<p>The classification of "complexity" is a function of the observer, not the observee.
I liked this article. These follow from the basic law that all systems that grow or change eventually decay and die. Death is an inevitable part of life whether that be for animals, societies or even software.<p>This is good. Rebirth brings vibrancy and you cannot have birth if you don't have death.
If these are the first three "fundamental" laws of software complexity, where does Gall's law fit? The zeroth law maybe?<p>Also, the second law: "complexity is a moat", seems to be contradicted by "worse is better".
I read this piece with great relief. It's easy for anyone to feel offended/frustrated when coworkers, outsourced teams or even customers twist and turn your system's original clean design, making it badly designed over time. I enjoyed reading the three laws because they make me feel part of an universal problem. Software systems are no different from physical matter: it decays. It's the damn entropy. Who of us would dare to fight against it?
Re: first law ... one thing I have been thinking a lot lately is just how much like gardening software is. But in gardening we are not afraid to trim the plants and put them in shape as we go. In software we end up just adding stuff on top, without regard for the overall design. There is this bias towards always adding rather than removing things, and the key to keep things in order is to remove things, to say no, etc.
> <i>When systems compete with each other for market share, delicacy goes out the window and designers often give the application everything it wants.</i><p>This rings false for established systems that are locked to their APIs.<p>The vendor of a large, complex set of APIs that large, complex applications depend on is not going to give the application users everything in a new API for fear they will migrate to another set of APIs.
How's about the 1 law of software complexity: software developers are not taught how to professionally communicate, so they over talk and ignore, misinform and mislead one another into a clusterfuck of complexity, over and over again. Never dawning on their minds that better communications would solve their repeating nightmare of complex weakly composed mud balls called their work.
I think the main thing we have control over is if we are part of a green field app making damn sure it is as dead simple as possible. Im currently working on a redesign of a medium sized app and the powers that be decided they wanted feature flags, which means keeping the old and new versions in the same app, and a giant separate branch for the in progress redesign. One of these would have been sufficient because they actually don’t intend to ever look at the old designs again. Not to mention three different models for the same data in the same domain that I immediately consolidated. Also, I hate feature flags and think they are the worst bandaid for design indecision and business people incompetence ever devised
At the root of all this is a philosophical problem that encompasses all we do: unsustainable growth. We're culturally obsessed with the concept of "more". More value. More money. More features. More, more, more. This is where it gets us: over the verge of ecological catastrophe. Unsustainable systems prone to breakage. Enshittification of everything. Planned obsolescence and yearly phone upgrades, natural resources be damned!<p>If we are to survive, we need to slow down and instead of making "more" make "better". Follow Unix philosophy. Embrace "good enough" and learn to stop.<p>Who am I kidding, we're doomed.
I have a law I figured out a few years ago.<p>Almost all programming jobs will be on too complex systems. If a system is well designed and not too complex, then they wont need to hire you!
I always find myself sitting down to read <i>Out of the Tar Pit</i>[0] at least a couple times per year. It has been—and continues to be—one of the most seminal texts in my career. I still remember the first time I read the following passage on complexity, and how it just turned on all the mental light bulbs:<p>>> Essential Complexity is inherent in, and the essence of, the <i>problem</i> (as seen by the <i>users</i>).<p>>> Accidental Complexity is all the rest — complexity with which the development team would not have to deal in the ideal world (e.g. complexity arising from performance issues and from suboptimal language and infrastructure).<p>>> Note that the definition of <i>essential</i> is deliberately more strict than common usage. Specifically when we use the term <i>essential</i> we will mean strictly essential <i>to the users’ problem</i> (as opposed to — perhaps — essential <i>to some specific, implemented, system</i>, or even — essential <i>to software in general</i>).<p>The best skill I've learned, and continued to practice and improve, is the ability to strip how we talk about problems we want to solve with software down to what's truly <i>essential to the problem</i>. Making a habit of doing so helps clarify the contours of the problem itself, and improves discussions around solving because the boundaries of what's truly essential become clear—and then everyone involved knows that every choice we make from that point is additional, accidental complexity <i>we are adding to the problem ourselves</i>.<p>Far too often I have seen even greenfield software quickly ratchet up the overall complexity because the people making choices don't take the time to really isolate <i>the problem</i> from <i>the software</i>—but instead frame the problem within the context of languages, frameworks, architecture, infrastructure, and so on, and then just start slinging code at the problem.<p>If you haven't yet read <i>Into the Tar Pit</i>, it truly changed the way I look at and think about software and problem complexity. You may find value in it, as well.<p>[0]: <a href="https://curtclifton.net/papers/MoseleyMarks06a.pdf" rel="nofollow">https://curtclifton.net/papers/MoseleyMarks06a.pdf</a>
The solution is probably going to involve dropping our dangerous utopian ideals about how complexity and deviation from perfection is problems that must be solved by any means necessary.<p>The world is a complex place where nearly nothing fit into an simplistic vision of simplicity and virtually no other engineering discipline shy away from gradual improvements and complexity management the way the IT sector does.<p>There is plenty of examples of real world road, water and sewage infrastructure where the system as a whole have continuity dating back centuries where every problem occurring was fixed in place without anyone ever redesigning the system by wiping and redesigning, and this is a source of pride not shame for the people working with those infrastructures.<p>The sooner we go away from the idea that just one more redesign using X tools in just the right way width the right team will finally crate an system that don't need constant maintenance and refactoring to keep serve the needs of it's users.
That hurts a little too deep.<p>Elegant code and clean architecture are no match for the chaos of the human mind.<p>Note how even the author thinks that nonsensically complex systems like Kubernetes and Zookeeper are the best possible designs.
When I read the title, I was expecting to find something akin to 3 metrics of software complexity from Ousterhout's Software Design Book.<p>the actual article was kinda disappointing.