Bloat is still software's biggest vulnerability (2024)

I am beginning to think that the terrible situation with dependency management in traditional C and C++ is a good thing.<p>Now, with systems like npm, maven or cargo, all you need to do to get a package is to add a line in a configuration file, and it fetches all the dependencies you need automatically from a central repository. Very convenient, however, you can quickly find yourself with 100+ packages from who knows where and 100s of MB of code.<p>In C, traditionally, every library you include requires some consideration. There is no auto-download, and the library the user has may be a different version from the one you worked with, and you have to accommodate it, and so does the library publisher. Or you may have to ship is with your own code. Anyways, it is so messy that the simplest solution is often not to use a library at all and write the thing yourself, or even better, realize that you don&#x27;t need the feature you would have used that library for.<p>Bad reason, and reinventing the wheel comes with its own set of problems, but at least, the resulting code is of a manageable size.

People often think &quot;speed&quot; when they read &quot;bloat&quot;. But bloat often means layers upon layers of indirection. You want to change the color of the button in one dialog. You find the dialog code, change the color and nothing. You dig deeper and find that some modules use different colors for common button, so you find the module setting, change the color and nothing. You dig deeper and find that global themes can change colors. You find the global theme, change the color and nothing. You start searching entire codebase and find that over 17 files change the color of that particular button and one of those files does it in a timer loop because your predecessor couldn&#x27;t find out why the button color changed 16 times on startup so he just constantly change it to brown once a second. That is bloat. Trivial change will take you half a day. And PM is breathing on your neck asking why changing button color takes so long.

At the library level, I dislike how coarse grained most things are. Sadly becomes easier to reimplement things to avoid huge dependency chains.<p>Want a simple web server ? Well, you’re going to get something with a JSON parser, PAM authentication, SSL, QUIC, websockets, an async framework, database for https auth, etc.<p>Ever look at “curl”? The number protocols is dizzing — one could easily think that HTTP is only a minor feature.<p>At the distro level, it is ridiculous that so long after Alpine Linux, the chasm between them and Debian&#x2F;RHEL remains. A minimal Linux install shouldn’t be 1GB…<p>We used to boot Linux from a 1.44mb floppy disk. A modern Grub installation would require a sizable stack of floppies! (Grub and Windows 3.0 are similar in size!)

In my last job, just to run the software on my local machine, I had to launch 6 different microservices running in a containerized, Linux virtualized environment on Windows and had to launch them in a particular order and had to keep each one in a separate console for debugging purposes. It took about 20 minutes to launch the software to be able to test it locally. The launch couldn&#x27;t be automated easily because each service was using a mix of containers and plain Node.js servers with different versions and it was Windows so I would probably have to write some unfamiliar code for Windows to automate opening all the necessary git bash tabs...<p>The services usually persisted except for automatic updates so I only had to restart all the services a few times per week so it didn&#x27;t make sense to invest time to automate.

&gt;Even companies with near-infinite resources (like Apple and Google) made trivial “worst practice” security mistakes that put their customers in danger. Yet we continue to rely on all these products.<p>I am at a big tech company and have seen some wildly insecure code make it into the codebase. I will forever maintain that we should consider checking if candidates actually understand software engineering rather than spending 4 or 5 hours seeing if they can solve brainteasers.

When it comes to building software for money, I prefer to put all of my eggs into one really big basket.<p>The fewer 3rd parties you involve in your product, the more likely you will encounter a comprehensive resolution to whatever vulnerability as soon as a response is mounted. If it takes 40+ vendors to get pixels to your customers eyeballs, the chances of a comprehensive resolution rocket toward zero.<p>If every component is essential, does it matter that we have diversified the vendor base? Break one thing and nothing works. There is no gradient or portfolio of options. It is crystalline in every instance I&#x27;ve ever encountered.

Yet if you deliver a system without a modern bloated framework or a massive cloud stack and you are &quot;old fashioned&quot; and &quot;out of touch&quot; - been there done that, got the tee-shirt.

Personally I see Docker as a problem more than a solution.<p>Back when I had slow ADSL (like 2 Mbps) I couldn&#x27;t use Docker at all at home because the repository server had low timeouts. I was downloading 20GB games with Steam not to mention Freebase data dumps and other things that large because I had reliable tools to do the downloads, which Docker didn&#x27;t use so downloading 5GB of images was not &quot;wait for it&quot; but rather &quot;you can&#x27;t do it.&quot;<p>By accelerating the rate at which you can attach random dependencies you can run into problems because you are using 6 different versions of <i>libc</i> for Christ&#x27;s sake. Rather than getting Python from some reputable source like conda or deadsnakes, Docker gives data scientists superpowers to get Pythons with random strange build options and character encodings. A 20 megabyte patch requires 2 GB of disk IO once it goes through the Docker IO multiplier. A 5 minute build becomes a 20 minutes build. Docker is fast from the viewpoint of &quot;ops&quot; but is slow from the viewpoint of &quot;dev&quot;; where people use Docker they are always taking forever to do the simplest things and facing extreme burnout.

Discussed at the time:<p><i>A 2024 plea for lean software</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=39315585">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=39315585</a> - Feb 2024 (240 comments)

This hits hard for me because I&#x27;ve run into way too much extra code getting piled on for no real reason. Stuff just gets harder to handle over time and gets in the way. Kinda makes me ask myself- you think folks are just chasing easy installs or is it more about looking busy than keeping things actually simple?

A big issue is the speed at which teams are expected to deliver. If every sprint is expected to deliver value to the user, there is isn’t enough slack in the system to go back and prune the code to remove cruft. People end up cutting corners to meet deadlines set by management. The corners that get cut are the things that are invisible in the demo. Security, documentation, and all the chewing gum holding it all together.

No argument from me, I also believe bloat is a very large problem.<p>A get of my lawn section :)<p>I remember when GUIs started becoming a thing, I dreaded the move from Text to GUIs due to complexity. I also remember most programs I wrote when I started on minis were 64k code and 64k text. They were rather powerful even by today&#x27;s standards, they did one thing and people had to learn which one to use to perform a task.<p>Now we have all in one where in some cases you need to page through endless menus or buttons to find an obscure function. In some cases you just give up looking and move on. Progress I guess.

They talk about the imessage vulnerability (1), but is it really an example of bloat to accidentally allow pdfs to be parsed with an extension of .gif? I guess it&#x27;s an example of an unnecessary functionality, but Apple would sell a lot less iPhones if they didn&#x27;t add all these UI gimmicks.<p>(1) <a href="https:&#x2F;&#x2F;googleprojectzero.blogspot.com&#x2F;2021&#x2F;12&#x2F;a-deep-dive-into-nso-zero-click.html" rel="nofollow">https:&#x2F;&#x2F;googleprojectzero.blogspot.com&#x2F;2021&#x2F;12&#x2F;a-deep-dive-i...</a>

To me, the root cause of this problem is the externalizing of knowledge. The number of tools used in building software has exploded. Each such tool, while purporting to make the job of the developer easy, hides what it really takes to make software. In turn, the developer unwittingly grows reliant on the tools, thereby externalizing the essential knowledge of what it really takes to build software, or what the real cost of adding a dependency is. Everything turns into, &quot;pff, I&#x27;ll just click that button on my IDE--job done!&quot;.<p>Every software component follows the same pattern. Software, thus made from these components, ends up being intractably complex. Nobody knows what a thing is, nor how things work.<p>This is where we are right now, before we add AI. Add AI and &quot;vibe coding&quot; to the mix, and we&#x27;re in for a treat. But don&#x27;t worry - there&#x27;ll be another tool that&#x27;ll make this problem, too, easy!<p>I&#x27;m hereby coining the term &#x27;cognitive sovereignty&#x27;.

The article makes using dependencies look bad, while the actual issue rather is &quot;quality controlling&quot; the code in dependencies, as dead code elimination (or &quot;tree shaking&quot;) removes the bloat from the final artifact. Because dependency as a concept itself is good, because going the opposite way and reinvent the wheel you&#x27;ll get an even worse kind of bloat - bloat you have to maintain yourself.

Perhaps a better title would be &quot;Supply chain vulnerabilities and attacks are software&#x27;s biggest vulnerability&quot;.

I&#x27;d expect LLMs to continue making bloat significantly worse. When the cost of a thing craters, in this case generated lines of code, then you&#x27;ll inevitably get way more of that thing.<p>Also my observations so far of &quot;vibe coding&quot; include a lot of copy and pasting errors that are then fixed with installing yet more libraries until you have a massive, glued-together mess.

This IEEE Spectrum article on software bloat and security provides a good summary of the problems plaguing much software these days but I see no indication that we will find solutions anytime soon.<p>There&#x27;s just too much invested in the building of software to dismantle current arrangements or change methodologies quickly, it would take years to do so. Commercial interests depend on bloat for income, so do programmers and support industries.<p>For example, take Microsoft Windows, these days it&#x27;s so huge it will not even fit onto a DVD, that&#x27;s petty outrageous really. I recall Windows expert Mark Russinovich saying that the core&#x2F;essential components of Windows only take up about 50MB.<p>But there&#x27;s no incentive for Microsoft to make Windows smaller and thus have a smaller footprint for hackers to attack. Why? As that bloatware makes Microsoft money!<p>Rather than dispense with all that bloatware Microsoft has build a huge security edifice around it, there are never-ending security updates, secure Windows boot&#x2F;UEFI, it&#x27;s even had to resort to a hardware security processor—Pluton. And much of this infrastructure is nothing but a damn nuisance and inconvienience for end users&#x2F;consumers.<p>Microsoft doesn&#x27;t just stop there, it then makes matters worse by unnecessarily changing the Windows GUI with every new version. Moreover, it&#x27;s not alone, every Linux distribution is different. What this means is that there&#x27;s less time to perfect code as its features keep changing.<p>Now take the huge numbers of programming languages out there. There are so many that many programmers have to learn multiple languages thus cannot become truly proficient in all of them. That lack of expertise alone is problematic. Surely it would be better to concentrate on fewer languages and make those more adaptable. But we know that&#x27;s not going to happen for all the usual reasons.<p>Same goes for Web browsers and Web bloat. Every time I complain on HN about browser bloat, the abuse of JS by websites and the never-ending number of Web protocols that keep appearing, I&#x27;m voted down. That&#x27;s understandable of course because programmers and others have a financial vested interest in them. Also, programmers have taken much time to learn all this tech and don&#x27;t want to see their efforts wasted by its obsolescence.<p>And I&#x27;ve not yet mentioned the huge and unnecessary proliferation of video, sound codecs, image and audio formats not to mention the many document formats. Programs that use all these formats are thus bigger and more bloated and more prone to bugs and security vulnerabilities. In a more organized world only faction that number would be necessary. Again, we know it&#x27;s not just technological improvements that have brought such numbers into existence but also commercial and vested interests. Simply, there&#x27;s money in introducing this tech even if it&#x27;s only slightly different to the existing stuff.<p>I&#x27;ve hardly touched this subject and said almost nothing about the economic structure of the industry, but even at first glance it&#x27;s obvious we can&#x27;t fix any of this in the near future, except perhaps by tiny incremental steps which will hardly make much impact.

FWIW, I started learning Elixir and OTP to overcome architectural bloat in future projects.<p>The Erlang ecosystem has many useful abstractions at just about the right level that allow you to build the simplest possible custom solution instead of reaching for a 3rd party solution for a component of your (distributed) system.<p>Building just the right wheel for the task at hand does not mean you have to reinvent it first.

Considering the disdain for software which does not have a thousand external dependencies in form libraries and framework is it any surprise?

Is there an OS that embodies this sentiment?

Unfortunately, this is not something that programmers, let alone security ops, can fix. In many companies, the management is too brain-dead to even conceive of the possibility of doing something that does not immediately translate into (short-term) profit. Companies that treat its software as a quality artifact are rare. At best, you have to go out of your way as a programmer to fix shit and maintain even a baseline of quality before shit hits the fan. The only way to get the bulk companies aligned with this goal is to make it so that failure to do so costs them money, AKA fining them for security breaches, accidents, etc.

Interesting! I&#x27;m not an expert but an aging amateur and *nix&#x2F;foss enthusiast. I see some parallels to what I&#x27;ve thought before that may, or may not be erroneous. First, it seems to point toward the original *nix philosophy of do one thing.<p>From a user&#x2F;fanboy&#x2F;paranoid point of view, I don&#x27;t like systemd. I&#x27;ve good development arguments for it&#x27;s improved coding for usb device drivers. Still, when I have to reboot, because my system is frozen. It&#x27;s more complex to use than say runit. Lastly, I&#x27;m nervous, that if a company took it over, it&#x27;s the one piece that might help destroy most distros. Please no hate. This is only my personal point of view, as an amateur e.g. there are people on both sides that have a much better understanding of this.<p>Seems to favor the microkernel? I&#x27;ve been hoping we one day get daily driver micro-kernel distro. I asked about this but didn&#x27;t get a lot of answers, except for those that mentioned projects that aren&#x27;t there yet e.g. I would love to try Redox, but from my understanding, after 10yrs it&#x27;s still not there yet.<p>It also brings me to a point that has confused me for years. As, an amateur how to I decide what is better for what level of virtualization from program images like appimage&#x2F;flatpacks, containers, to VMs. So far, I&#x27;ve hated snaps&#x2F;flatpacks because, they make a mess of other basic admin commands, and because there seems to be missing functionality. and&#x2F;or configuration. It may be better now; I haven&#x27;t tried in a while. Personally, I&#x27;ve enjoyed portage systems in the past, and they are so fast now (to compile). A lot of forums, forget that there are home enthusiast and basically talk about it from an enterprise perspective. Is there a good article or book that might explain when to choose what. Much of what I&#x27;ve read are just &quot;how to&quot; or &quot;how it works&quot;. I guess, I would prefer someone who acknowledges we need something for the hardware to run on and when it makes more since to use a regular install vs an image (appimage&#x2F;flatpack&#x2F;snap).<p>Anyway, thanks so much for the article. I do believe you are right, a lot of companies just put out fires because none want to invest in the future. I mean even the CEO usually only is there a few years, historically comparatively; so why would they care? Also, I think H1-B is a security risk in and of itself because, at least in TX, most IT is Indian H1-B. I mean they want a better life, and don&#x27;t have as many family ties here. If they were to &quot;fall into&quot; a large sum...they could live like Kings in India, or elsewhere.

In other news, bad software is bad. Heh, excuse the sarcasm. Tangential: I&#x27;ve come to think of the major software problems (like bloat, or closed-source, or lack of interop) as an effect of our generally chaotic and self-contradictory culture: &quot;money is the root of all evil&quot;, &quot;how much do you make&quot;, &quot;it has to be good&quot;, &quot;it has to be done fast&quot;, &quot;do what you think is best&quot;, &quot;do what&#x27;s expected&quot;, &quot;be conservative&quot;, &quot;be innovative&quot;. It&#x27;s hard to navigate through all that, and if you do, and you have something to show for it, you have my applause, irrespective of how good it is, for whatever problematic definition of good I happen to use today.