AI makes tech debt more expensive

&gt; Companies with relatively young, high-quality codebases benefit the most from generative AI tools, while companies with gnarly, legacy codebases will struggle to adopt them. In other words, the penalty for having a ‘high-debt’ codebase is now larger than ever.<p>This mirrors my experience using LLMs on personal projects. They can provide good advice only to the extent that your project stays within the bounds of well-known patterns. As soon as your codebase gets a little bit &quot;weird&quot; (ie trying to do anything novel and interesting), the model chokes, starts hallucinating, and makes your job considerably harder.<p>Put another way, LLMs make the easy stuff easier, but royally screws up the hard stuff. The gap does appear to be widening, not shrinking. They work best where we need them the least.

&gt; However, in ‘high-debt’ environments with subtle control flow, long-range dependencies, and unexpected patterns, they struggle to generate a useful response<p>I&#x27;d argue that a lot of this is not &quot;tech debt&quot; but just signs of maturity in a codebase. Real world business requirements don&#x27;t often map cleanly onto any given pattern. Over time codebases develop these &quot;scars&quot;, little patches of weirdness. It&#x27;s often tempting for the younger, less experienced engineer to declare this as tech debt or cruft or whatever, and that a full re-write is needed. Only to re-learn the lessons those scars taught in the first place.

&gt; There is an emerging belief that AI will make tech debt less relevant.<p>Wow. It&#x27;s hard to believe that people are earnestly supposing this. From everything we have evidence of so far, AI generated code is destined to be a prolific font of tech debt. It&#x27;s irregular, inconsistent, highly sensitive to specific prompting and context inputs, and generally produces &quot;make do&quot; code at best. It can be extremely &quot;cheap&quot; vs traditional contributions, but gets to where it&#x27;s going by the shortest path rather than the most forward-looking or comprehensive.<p>And so it does indeed work best with young projects where the prevailing tech debt load remains low enough that the project can absorb large additions of new debt and incoherence, but that&#x27;s not to the advantage of young projects. It&#x27;s setting those projects up to be young <i>and</i> debt-swamped much sooner than they would otherwise be.<p>If mature projects can&#x27;t use generative AI as extensively, that&#x27;s going to be to their advantage, not their detriment -- at least in terms of tech debt. They&#x27;ll be forced to continue plodding along at their lumbering pace while competitors bloom and burst in cycles of rapid initial development followed by premature seizure&#x2F;collapse.<p>And to be clear: AI generated code can have real value, but the framing of this article is bonkers.

&gt;Instead of trying to force genAI tools to tackle thorny issues in legacy codebases, human experts should do the work of refactoring legacy code until genAI can operate on it smoothly<p>Instead of genAI doing the rubbish, boring, low status part of the job, you should do the bits of the job no one will reward you for, and then watch as your boss waxes lyrical about how genAI is amazing once you&#x27;ve done all the hard work for it?<p>It just feels like if you&#x27;re re-directing your efforts to help the AI, because the AI isn&#x27;t very good at actual complex coding tasks then... what&#x27;s the benefit of AI in the first place? It&#x27;s nice that it helps you with the easy bit, but the easy bit shouldn&#x27;t be that much of your actual work and at the end of the day... it&#x27;s easy?<p>This gives very similar vibes to: &quot;I wanted machines to do all the soul crushing monotonous jobs so we would be free to go and paint and write books and fulfill our creative passions but instead we&#x27;ve created a machine to trivially create any art work but can&#x27;t work a till&quot;

Evergreen: <a href="https:&#x2F;&#x2F;static.googleusercontent.com&#x2F;media&#x2F;research.google.com&#x2F;en&#x2F;&#x2F;pubs&#x2F;archive&#x2F;43146.pdf" rel="nofollow">https:&#x2F;&#x2F;static.googleusercontent.com&#x2F;media&#x2F;research.google.c...</a><p>Machine learning is the high interest credit card of technical debt.

It is a self-reinforcing pattern: the easier it is to generate code, the more code is generated. The more code is generated, the bigger the cost of maintenance is (and the relationship is super-linear).<p>So every time we generate the same boilerplate we really do copy&#x2F;paste adding to maintenance costs.<p>We are amazed looking at the code generation capabilities of LLMs forgetting the goal is to have <i>less</i> code - not more.

This is just taking the advice to make code sane so that humans could undertand and modify it, and then justifying it as &quot;AI should be able to understand and modify it&quot;. I mean, the same developer efficiency improvements apply to both humans and AI. The only difference is that currently humans working in a space eventually learn the gotchas, while current AIs don&#x27;t really have that ability to learn the nuances of a particular space over time.

I love the way our SWE jobs are evolving. AI eating the simple stuff, generating more code but with harder to detect bugs... I&#x27;m serious, it feels that we can move faster with these tools but perhaps have to operate differently.<p>We are a long ways from automating our jobs away, instead our expertise evolves.<p>I suspect doctors go through a similar evolution as surgical methods are updated.<p>I would love to read or participate in the discussion of how to be strategic in this new world. Specifically, how to best utilize code generating tools as a SWE. I suppose I can wait a couple of years for new school SWEs to teach me, unless anyone is aware of content on this?

On one hand I agree with this conceptually, but on the other hand I&#x27;ve also been able to use AI to rapidly clean up and better structure a bunch of my existing code.<p>The blind copy-paste has generally been a bad idea though. Still need to read the code spit out, ask for explanations, do some iterating.

&gt; Not only does a complex codebase make it harder for the model to generate a coherent response, it also makes it harder for the developer to formulate a coherent request.<p>&gt; This experience has lead most developers to “watch and wait” for the tools to improve until they can handle ‘production-level’ complexity in software.<p>You will be waiting until the heat death of the universe.<p>If you are unable to articulate the exact nature of your problem, it won&#x27;t ever matter how powerful the model is. Even a nuclear weapon will fail to have effect on target if you can&#x27;t approximate its location.<p>Ideas like dumpstering all of the codebase into a gigantic context window seem insufficient, since the reason you are involved in the first place is because that heap is not doing what the customer wants it to do. It is currently a representation of where you <i>don&#x27;t</i> want to be.

AI has a different &quot;tech debt&quot; issue.<p>Because with AI you can turn any problem into a black box. You build a model, and call it &quot;solved&quot;. But then reality hits ...

&quot;Companies with relatively young, high-quality codebases&quot;<p>I thought that at the beginning the code might be a bit messy because there is the need to iterate fast and quality comes with time, what&#x27;s the experience of the crowd on this?

&quot;Companies with relatively young, high-quality codebases benefit the most from generative AI tools&quot; - this is not true<p>The codebases that use the MOST COMMONLY USED LIBRARIES benefit the most from generative AI tools

I asked the AI to write me some code to get a list of all the objects in an S3 bucket. It returned some code that worked, it would no doubt be approved by most developers. But on further inspection I noticed that it would cause a bug if the bucket had more than 1000 objects because S3 only delivers 1000 max objects per request, and the API is paged, and the AI had no ability to understand this. So the AI&#x27;s code would be buggy should the bucket contain more than 1000 objects, which is really, really easy to do with an S3 bucket.

It&#x27;s funny that his recommendations - organize code in modules etc. - are nothing AI-specific, it&#x27;s what you&#x27;d do if you had to handover your project to an external team, or simply make it maintainable in the long term. So the best strategy for collaborating with AI turns out to be the same as for collaborating with humans.<p>I completely agree. That&#x27;s why my stance is to wait and see, and in the meanwhile get our shit together, as in make our code maintainable by any intelligent being, human or not.

Speaking personally, I&#x27;ve found this tech much more helpful in existing codebases than new ones.<p>Missing test? Great, I&#x27;ll get help identifying what the code <i>should</i> be doing, then use AI to write a boatload of tests in service towards those goals. Then I&#x27;ll use it to help refactor some of the code.<p>But unlike the article, this requires actively engaging with the tool rather than, as they say a &quot;sit and wait&quot; (i.e., lazy) approach to developing.

It is not just the code produced with code generation tools but also business logic using gen AI.<p>For example a RAG pipeline. People are rushing things to market that are not built to last. The likes of LangChain etc. offer little software engineering polishing. I wish there were a more mature enterprise framework. Spring AI is still in the making and Go is lagging behind.

I find AI most helpful with very specific, narrow commands (add a new variable to the logger, which means typescript and a bunch of other things need to be updated) and it can go off and do that. While it&#x27;s doing that I&#x27;ll be thinking about the next thing to be fixed already.<p>Asking it for higher level planning &#x2F; architecture is just asking for pain

I keep waiting for the pairing of coding LLMs with a programming language created specifically to be coupled with a coding LLM.

I agree with a lot of the assertions made in TFA but not so much the conclusion. AI increasing the velocity of simpler code doesn’t make tech debt more expensive, it just means it won’t benefit as much &#x2F; be made cheaper.<p>OTOH if devs are getting the simpler stuff done faster maybe they have more time to work on debt.

While this primarily focuses on the software development side of things, I’d like to chime in that this applies to the IT side of the equation as well.<p>LLMs can’t understand why your firewall rules have strange forwards for ancient enterprise systems, nor can they “automate” Operations on legacy systems or custom implementations. The only way to fix those issues is to throw money and political will behind addressing technical debt in a permanent sense, which no organization seemingly wants to do.<p>These things aren’t silver bullets, and throwing more technology at an inherently political problem (tech debt) won’t ever solve it.

&gt; In essence, the goal should be to unblock your AI tools as much as possible. One reliable way to do this is to spend time breaking your system down into cohesive and coherent modules, each interacting through an explicit interface.<p>I find this works because its much easier to debug a subtle GPT bug in a well validated interface than the same bug buried in a nested for loop somewhere.

I recently started playing with OpenSCAD and CadQuery -- tried a variety of the commercial LLMs, they all fall on their face so hard, teeth go flying.<p>This is for tiny code snippets, hello-world size, stringing together some primitives to render relatively simple objects.<p>Turns out, if the codebase &#x2F; framework is a bit obscure and poorly documented, even the genie can&#x27;t help.

&gt; <i>Companies with relatively young, high-quality codebases benefit the most from generative AI tools, while companies with gnarly, legacy codebases will struggle to adopt them.</i><p>So you say, but {citation needed}. Stuff like this is simply not known yet.<p>AI can easily be applied in legacy codebases, like to help with time-consuming refactoring.

A hard choice: Tune your code to unique customer requirements or keep it generic to please your AI.

&gt; Instead of trying to force genAI tools to tackle thorny issues in legacy codebases, human experts should do the work of refactoring legacy code until genAI can operate on it smoothly. When direct refactoring is still too risky, teams can adjust their development strategy with approaches like strangler fig to build greenfield modules which can benefit immediately from genAI tooling.<p>Or, y&#x27;know, just not bother with any of this bullshit. &quot;We must rewrite everything so that CoPilot will sometimes give correct answers!&quot; I mean, is this worth the effort? Why? This seems bonkers, on the face of it.

Today&#x27;s job is finishing up and testing some rather gnarly haproxy configuration. There&#x27;s already a fairly high chance I&#x27;m going to stuff something up with it. There is no chance that I&#x27;m giving some other entity that chance as well.

The title of this article made me think that paying down traditional tech debt due to bugs or whatever is straightforward. Software with tech debt and&#x2F;or bugs that incorporates AI isn’t a straightforward rewrite, but takes ML skills to pay down.

Haven&#x27;t read the article, don&#x27;t need to read the article: this is so, SO, so painfully obvious! If someone needs this spelled out for them they shouldn&#x27;t be making technical decisions of any kind. Sad that this needs to be said.

&gt; <i>AI makes tech debt more expensive</i><p>This isn&#x27;t AI doing.<p>It&#x27;s the doing of adding any new feature to a product with existing tech debt.<p>And since AI for most companies is a <i>feature</i>, like any feature, it only makes the tech debt worse.

&quot;A product should be owned by a lean team of experts, focused primarily on the architecture of their code rather than the implementation details.&quot;<p>Sheesh! The Lizard People walk among us.

AI is a tool and nothing more. You give it too much and it will fumble, humans fumble but we can self correct where instead AI hallucinates. Crazy nightmare AI dreams.

Coding with AI could easily be a new form of early software&#x2F;developer tech debt. Taking leaps that are too big, or too small, can be unexpected.

Not sure its tech debt as such, its the hidden cost of having to maintain AI tech. its not a static state.. and its got an ongoing maint cost.

This type of analysis is a mirror of the early days of chess &quot;AI&quot;. All kinds of commentary explaining the weaknesses of the engines, and extolling the impossible-to-reproduce capabilities of human players. But while they may have been correct in the moment, they didn&#x27;t really appreciate the march toward utter dominance and supremacy of the machines over human players.<p>While there is no guarantee that the same trajectory is true for programming, we need to heed how emotionally attached we can be to denying the possibility.

Click bait headline. It is an opinion piece; it may be true (or not) but there is not references or clear justifications.

I enjoy reading these articles and reading comments from people who clearly have no idea how to use AI or it’s abilities.

True if you’re using AI the wrong way. AI means dramatically less code, most of which is generated.<p>Creating react pages is the new COBOL

AI code is just a more available SO code. You don&#x27;t use the code handed to you, you learn from it.

Is this based on a study or something? I just see a graph with no references. What am I missing here?

This isn&#x27;t tech debt this is ignorece debt and lazyness debt by hiring incomptence

&gt; human experts should do the work of refactoring legacy code until genAI can operate on it smoothly<p>How does one determine if that&#x27;s even possible, much less estimate the work involved to get there?<p>After all, &#x27;subtle control flow, long-range dependencies, and unexpected patterns&#x27; do not always indicate tech-debt.

I cannot wait for the inevitable top-down backlash banning any use of AI tools.

Don&#x27;t make me tap the sign.<p>&quot;GARBAGE IN -- GARBAGE OUT!!&quot;

Good for us I guess?

Code is not really lossy zipped text.

The author starts with a straw man argument, of someone who thinks that AI is great at dealing with technical debt. He makes little attempt to steel man their argument. Then the author argues the opposite without much supporting evidence. I think the author is right that some people were quick to assume that AI is much better for brownfield projects, but I think the author was also quick to assume the opposite.

... until it won&#x27;t. A mature code-base also has (or should have) strong test coverage, both in unit-testing and comprehensive integration testing. With proper ci&#x2F;cd pipelines, you can have a small team update and upgrade stuff at a fraction of the usual cost (see amazon going from old java to newer versions) and &quot;pay off&quot; some of that debt.<p>The tooling for this will only improve.

Yeah, this is a total click-bait article. The claim put forth by the title is not at all supported by the article contents, which basically states &quot;old codebases riddled with tech-debt do not benefit very much from GenAI, while newer cleaner codebases will see more benefit.&quot; That is so completely far off from &quot;AI will make your tech debt worse.&quot;

Microservices are back on the menu, boys.

are LLMs even auditable?

Bah this article is a bunch of nonsense. You&#x27;re saying that a technology that has been around for a grand 2 years is not yet mature? Color me shocked.<p>I&#x27;m sure nothing will change in the future either.

And well duh

LLM code gen tools are really freaking good...at making the exact same react boilerplate app that everyone else has.<p>The moment you need to do something novel or complicated they choke up.<p>This is why I&#x27;m not very confident that tools like Vercel&#x27;s v0 (<a href="https:&#x2F;&#x2F;v0.dev&#x2F;" rel="nofollow">https:&#x2F;&#x2F;v0.dev&#x2F;</a>) are useful for more than just playing around. It seems very impressive at first glance - but it&#x27;s a mile wide and only an inch deep.