Isolating complexity is the essence of successful abstractions

The author&#x27;s assertion is true - complexity has to live somewhere. The nuance, though, is that all places complexity can live are not created equal.<p>Let&#x27;s take the example of memory management: by pushing that complexity into the type system, Rust forces the programmer to deal with it and design around it. At the expense of some performance, we could instead push this complexity into a runtime garbage collection system. Since the runtime system understands things about the runtime characteristics of the program that can&#x27;t be proven via static analysis, it can also handle more things without the programmer having to intervene, thus reducing the difficulty of the programming language. For most programmers this is a positive tradeoff (since most programmers are not writing software where every microsecond matters).<p>Similar tradeoffs exist in many different areas of software engineering. One monolith, where all the information is in one place, is easier to write code in than two microservices, which keep having to ask each other questions via API call. Yet, sometimes we need microservices. Rendering your web application entirely on the frontend in React, or entirely on the backend with templates, where all the logic lives in one place, is much easier than doing server-sided rendering then hydrating on the frontend. Yet, sometimes we need server-sided rendering and hydration.<p>Complexity is an irreducible constant, yes, but cognitive load is not. Cognitive load can increase or decrease depending on where you choose to push your complexity.

I do not think of complexity as one thing. Abstractions are about both hiding and exposing complexity at the same time. Different levels of abstractions can expose or isolate different part of complexity. Exposing parts of it in a way that they become amenable to your tools is as important as isolating other parts somewhere in the background. Essentially, this has to do with how well a given abstraction choice maps into the structure of the problem-space and the relationships there. The choice of which parts of complexity you isolate and which you expose is important. You probably do not want to deal with everything at once, but also usually you cannot avoid dealing with something.<p>The way I primarily see (and often like) type systems wrt complexity is as choosing which parts of complexity are important and exposing them (and rest being still there to deal with). There is a cognitive aspect to abstractions and complexity, irrespective even of IDEs, debuggers, compilers etc. I personally want my abstractions to make at least some sense in my head or a piece of paper in the way I think about the problem before even I start writing code. If the abstractions do not help me actually cognise about (some part of) the problem, they probably solve other problems, not mine.

That&#x27;s why Typescript&#x2F;Python optional typing hit the best balance for me. Coding in duck-typed language is generally fine when your test suite is as fast and frequent as a type checker. That also explains why TDD is more popular in say Ruby or Python vs. Java. Speaking of Java, the problem with types is when you try to reify every single problem you encounter in your codebase. By the way, python has structured types since 3.8, and I hope they get more popular in Python code: <a href="https:&#x2F;&#x2F;docs.python.org&#x2F;3&#x2F;library&#x2F;typing.html#typing.Protocol" rel="nofollow">https:&#x2F;&#x2F;docs.python.org&#x2F;3&#x2F;library&#x2F;typing.html#typing.Protoco...</a>

I don’t think I agree that either typescript nor rust successfully hide the complexity in their type systems.<p>By the nature of type systems, they are tightly coupled with the code written around them.<p>Rust has rich features to handle this coupling (traits and derives), but typescript does not.

Isolating complexity, I would say, is a consequence of using good abstractions... not necessarily the <i>essence</i> of abstraction however. The essence of abstractions are in <i>semantics</i>. I define a type and an algebra of relations on that type which gives me theorems I can use. That is the essence. The consequence is that I can now think in terms of the theorems and definitions I&#x27;ve established rather than all of the details at the lower-level.<p>However, sometimes it&#x27;s a bit over-rated when all that&#x27;s needed is some information hiding and indirection, which is what this article appears to be discussing. These tools are the ones that are &quot;leaky&quot; in the sense that the complexity they attempt to hide often escapes the confines of their interface. It tends to give &quot;abstraction&quot; a bad reputation among programmers who have to deal with such systems.<p>Essential complexity does have to live somewhere. Best to be upfront about it.

I find this topic particularly interesting. I&#x27;ve often said to others that software, in itself, is a general abstraction of one or more complex tasks. The whole point of software is to hide complexity and make possible, in a hopefully simpler manner, doing things that would otherwise be very difficult or impossible. Despite what users may experience, the complexity remains but becomes hidden.

Python showed what relaxed types could do. And we could go a long way as it turns out without types. But there are use cases for types, and even python admitted such when they added type annotations.<p>However, when I was a kid a would put a firecracker next to an object. I didn&#x27;t bother running the scenario through a compiler to see if the object was of type Explodable() and had an explode() method that would be called.

One of the problems with abstraction is that while it hides complexity, it makes changes that must reach into that complexity difficult. Abstraction is great if the code is never going to change again. If the people using the code want new features, then abstraction is a barrier. Getting around the abstraction barrier makes the code more complex. You have to think about the entire life cycle of the code, not just what looks pretty when you first write it. Most developers have no idea what their code will be used for 5 or 10 years into the future. As an example people have been trying to abstract away the complexity of network connections for decades, without a lot of success in keeping the complexity hidden. Someone always needs direct intervention in a layer in the network stack to make their product work right.

I don’t think abstractions are inherently tied to hiding complexity. The purpose of an abstraction is to abstract over variations of a thing (think polymorphism), where each variation by itself might still be simple, or to separate essential features (e.g. parameters you have to pass) from accidental features (e.g. implementation details), where again there is no inherent implication of complexity on either side.<p>In slightly different words, an abstraction separates what client code needs to reason about from what it should be able to ignore. Of course, if an abstraction isolates client code from certain complexities, that will contribute to the success of the abstraction. But it’s not the essence of what an abstraction does, or a necessary condition for it to count as successful.

I have always felt that it&#x27;s better to &quot;concentrate&quot; complexity into one key component and make the rest of the codebase simple than to distribute complexity evenly everywhere in some kind of open-coded swamp.

&gt; Complexity has to live somewhere. If you are lucky, it lives in well-defined places.<p>This whole section makes me think of construction which has similar abstraction and hidden complexity problems. It strikes me that they solve it by having design be entirely separate from implementation. Which is usually the corner where all our luck as software developers inevitably runs out.<p>Our methods are still rather &quot;cowboy.&quot; We have cool &quot;modernized cowboy&quot; languages that make it hard to shoot your foot off, but at the end of the day, we&#x27;re still just riding old horses and hoping for the best.

&gt;<i>The question is first of all whether we have written them down anywhere</i><p>The only hard thing in software: papers please (easily accessible documentation)

Does complexity mean a long block of code with many levels of nested conditionals which are messed with cross-block mutable variables ?

complexity has to live somewhere, code anxiety was a real thing for me

&quot;Parameterizing complexity&quot; is probably a better way to say it. There&#x27;s no isolation when it comes to software.

This is such a simplistic view on the matter.<p>The author talks about complexity like it&#x27;s always an intrinsic thing out there (essential) and the job of the abstraction is to deal with it. It misses the point that a great deal of the complexity on our plates are created by abstractions themselves (accidental). Not only that, sometimes great abstractions are precisely the ones that decide to not isolate some complexity and allow the user to be a &#x27;power user&#x27;.

I think &quot;types&quot; is the solution of two completely different problems:<p>1. how to specify memory layout for faster execution<p>2. how to give hint when I press . in IDEs<p>if you use typing outside these two scopes you&#x27;d probably find many troubles.