Developer inequality and the technical debt crisis

This feels astonishingly wrongheaded to me. &quot;Imagine an Internet HyperCard that allowed regular people to easily build web apps, as easily as using a spreadsheet.&quot; and then JUST TWO SENTENCES LATER, &quot;There are many app-in-a-can tools that generate highly stereotypical apps but to be truly disruptive we need to match the broad generality of frameworks like Rails.&quot;<p>In other words, we need to make it possible for people to build simple things without much coding. Oh wait, people already can build simple things, but....that&#x27;s not what you mean?<p>The Web is actually a phenomenally easy platform to learn how to write code for, compared to basically every platform I&#x27;ve ever seen that came before it. Yeah, it has a shitload of quirks and stupidities and kludges, but that&#x27;s what happens when you&#x27;re building a system that gets used by a very large percentage of the global human population.<p>The author writes as though he&#x27;s never actually considered that doing anything at scale - not programming, anything; running a train system, or setting up a government, or selling falafels to people at lunchtime - inevitably involves hacks, compromises, kludges and half-measures. &quot;Technical debt&quot; is just another term for &quot;doing things that involve humans&quot;.

I think the reason there&#x27;s so much technical debt is largely because the amount it would cost to actually build quality software... is too high. We could not afford it. Like, as a society. Our society is built on crappy software.<p>I think it&#x27;s just a utopian fantasy to think that if only the right hypercard-like tool could be created, then the cost of building quality software would go down.<p>Or at any rate, actually: Let&#x27;s agree that the web is built on an enormous stack of kludges upon kludges. (These kludges are both in code frameworks that people use to build things on the web, and in the fundamental protocols of the web itself). The reason it is this way is, again, because by the time it is recognized what a problem this is, it would simply be too expensive to rebuild the web from scratch. We can&#x27;t afford it.<p>To build this utopian hypercard-like stack which allow just anyone to build web things, and be so high-quality that it just worked without having to understand the things it&#x27;s abstracted on top of, and to maintain it as web technology and desires continue to evolve, etc.... would be such an expensive undertaking, with such a high risk of failure, that it has no way to succeed in that fantasy of making the web all around cheaper.<p>We see posts like this come up here from time to time, written by non-programmers who have some kind of belief that programmers _like_ complexity, that programmers are _opposed_ to making things easy and simple. I totally don&#x27;t see &quot;modern programmer culture fetish[izing] complexity&quot; -- rather, on HN, I think it&#x27;s pretty clear that modern programmer culture fetishizes simplicity. It&#x27;s just that simplicity is _hard_. (And people chasing simplicity often end up over-abstracting, and just winding up with an even worse form of complexity). Succesful software that is powerful and reliable <i>and</i> simple takes skill and it takes time. And skill and time cost money.<p>We&#x27;ve built an economy and a society that is entirely based on crappy software, because the economy could not bear the cost of as much quality software as we have crappy software, and the crappy software provides short-term efficiencies that make people money. (and i&#x27;m not talking about programmers,I&#x27;m talking about the &#x27;domain&#x27; businesses which could not afford to run without software &#x27;automation&#x27; anymore, even though it&#x27;s all crappy)

I&#x27;ll go ahead and say it: this is a strawman.<p>A simple web page is as simple as it ever was:<p><pre><code> &lt;html&gt; &lt;head&gt; &lt;title&gt;Hi&lt;&#x2F;title&gt; &lt;&#x2F;head&gt; &lt;body&gt; &lt;h1&gt;Hello, world!&lt;&#x2F;h1&gt; &lt;&#x2F;body&gt; &lt;&#x2F;html&gt; </code></pre> The <i>problem</i> is that everybody wants something so flashy and pretty that they can&#x27;t settle for the functional-but-ugly barebones interfaces app programmers used to make. The <i>problem</i> is that we&#x27;ve trained users that if a product isn&#x27;t pretty it&#x27;s useless, and in pursuit of that all these other things happen.<p>You need to have a single-page app, because UX. Well, that in turn requires all these other things, in turn requiring still more things. You need to have a pretty web page that also looks good on mobile, in turn requiring more clever responsive CSS, and so on and so forth.<p><i>That&#x27;s</i> the root of the issue.

Agreed on the &quot;let&#x27;s make programming more accessible&quot; point, but meh on the &quot;good old days&quot; view. I&#x27;ve been writing software since 1976 so I have some perspective on the good old days. Programming was always arcane, and it always required specialized knowledge and the patience to work at a level of detail few people find enjoyable. Yes, there were things like hypercard that opened up programming at a certain level to a certain semi-skilled person. Excel falls into the same category. But in the end it is not the symbology, or the tools, or the environment that makes programming difficult: it&#x27;s the mental process of building up complex behavior from little pieces, analogous sometimes to being handed a bag of atoms and instructed to build a toaster. Some of us find building up these intricate abstract models fun and rewarding, but believe me we&#x27;re a tiny masochistic fraction of humanity.

One particular point of developer inequality and technical debt that I see get ignored is people with the social credit or track that lets them only create new things as opposed to those that have to do maintenance.<p>I find the latter often appreciate the place and purpose of types more clearly than the former, even if the former would benefit as well from having concise encodings of intent in the design and creation phase.<p>This bias is prevalent in dyn-lang&#x2F;consulting-oriented communities, often because they are churning out projects and dumping them on their clients&#x2F;employers.

I often watch other developers look to make their lives more complex by trying to solve problems which aren&#x27;t there. Why? Because the obvious solution is simple and boring. In any other culture that would be crazy. In programming culture, it seems to be the norm.

&quot;Our goal was to allow regular people without extensive training to easily and quickly build useful software. This was the spirit of languages like COBOL, Visual Basic, and HyperCard.&quot;<p>I think that this slightly overstates the matter. Having peered into the guts of some payroll code, I question whether it was easily built. Could we say<p>&quot;To allow application developers to concentrate on the complexities of the problem domain rather than on those of the computing environment.&quot;?<p>[edit: rephrased my first sentence.]

&gt; Attempting to simplify and democratize programming will attract only scorn and derision (as did COBOL and Visual Basic).<p>The reason COBOL and Visual Basic attracted scorn and derision is that they were awful. The problem with them is quite simple: You can use them to create something improperly in 1000 hours that you could create properly using other tools in 2000 hours. That thing will then work until you have enough concurrent users to expose the race conditions, or the Access database backing it reaches the 2GB limit, or the wind blows too hard and the hamsters powering it become frightened. At which point you&#x27;ll spend 10,000 hours of overtime trying to keep it from collapsing in production while the users all burn you in effigy.<p>And the likes of Javascript don&#x27;t suck because they aren&#x27;t like COBOL and Visual Basic, they suck because they are.

<i>The civilized platforms controlled by large companies who invested in developer tools are all gone, strangled by the Darwinian jungle of the web. It is hard for programmers who have only known the web to realize how incredibly awful it is compared to past platforms. The web is just an enormous stack of kluges upon hacks upon misbegotten designs.</i><p>This passage makes me remember that famous quote by Alan Kay:<p><i>The Internet was done so well that most people think of it as a natural resource like the Pacific Ocean, rather than something that was man-made. When was the last time a technology with a scale like that was so error-free? The Web, in comparison, is a joke. The Web was done by amateurs.</i>

One thing this article completely ignores (as do many of the comments on the site) is that expectations have also drastically changed. Yes, Hypercard was dead simple to get something useful actually working, but nobody would accept a web app that looked and behaved like a Hypercard app.<p>I think the author is right: there is a none-to-healthy culture of increasing complexity surrounding web development. But there is also a large amount of real complexity around it. Many of the complexities we deal with today are the result of attempting to put an abstraction layer over deeper, larger complexities. Unfortunately, few abstraction layers work without leaks, and those leaks add their own complexity.<p>With powerful browsers, APIs-as-a-service and single-page apps, we are almost to a point where the web itself has been abstracted. But, even with that, expectations are still incredibly high. Nobody wants to look at blocky, course web sites; they want beautiful, smoothly animated sites. And that takes complexity.

Love this post. Brings up the question of are we solving problems are merely difficult puzzles? &quot;Modern programmer culture fetishizes complexity as a status marker or tribal initiation ritual&quot;

The web won because it has the least barriers to entry for a new developer and solves a lot of difficult problems that the old-school application stack had. This is all from my history and point of view so I&#x27;m sure a lot of people will disagree.<p>Cost:<p>The days when big companies developed tooling and platforms cfor developers where great IF you worked at a company that could afford to pay for them. They were expensive. When I first started these suites were tens of thousands of dollars expensive, and later just thousands of dollars. Sure they did a lot of stuff, but man there was also A LOT of overhead in managing the suite. For young people and very small companies these things were unaffordable, and frankly unmaintainable. A lot of times you would here about companies who paid 10-20K for a suite only to not use it due to the training required for everyone, the installation cost, and the need to change your development process to fit the platform&#x27;s model. So anyone starting out as a developer or starting up as company would find the cheapest&#x2F;easiest options available. And since there are far more small development teams than big companies there was far more demand and community support for free&#x2F;cheap systems that while not great could get the job done. Linux was free, mysql was free,Php was free, Perl was free, Javascript was free, and there were communities that would support a new developer through getting these up and running (no 2500 training class required). The reason the LAMP stack became so popular was because it was free, could be installed on cheap hardware, and every element had a community that would support it.<p>Cross Platform Support and Application Distribution:<p>With C++&#x2F;C you had to compile for every target machine. What a nightmare, library flags, macros, ugh. Testing had to have special labs so they could find bugs that would popup in one OS, but not another. Then it had to be cloned on disks and shipped out. Java came around with write once run everywhere which was better (MS came out with CLI and .net). The problem with these was making sure your customers had the base platform installed (the JRE or .net). B2B companies was a little easier because they had standardized equipment and IT departments that knew what to do, but if you were targeting B2C or small business then getting them to install the base software (again JRE&#x2F;.NEt) was a pain because they would more than likely mess it up or just not know what to do so it required manual intervention. As soon as this got easier you wound up with the conflict of people having TOO much installed on their computer and just not wanting to install anything else unless they HAD to. Fast forward to now and we&#x27;ve got a built in client that adhere&#x27;s to common standards. If they want to use your app they bookmark it, and when they are sick of it they just delete the book mark. Sure there&#x27;s cross browser issues, but they are far easier to deal with than both having the customer install some platform and dealing with cross OS issues (plus you dealing with them doesn&#x27;t require the customer to do anything but hit the refresh button).<p>Developer Inequality:<p>I don&#x27;t know how you can say there&#x27;s a higher barrier to entry now than previous. In the 90&#x27;s almost everyone you met who was a developer was a CS&#x2F;EE major or started on computers at the age of 12. The arcane level of knowledge required for using platform to build an application in the 90&#x27;s dwarfs what&#x27;s required now. I still remember struggling with bugs where you&#x27;d go hunting for &quot;the guy&quot; who new everything, and then you&#x27;d spend a couple of days trying to figure where the issue was only to find some undocumented outcome to a flag passed to a lib. To the developer who was working for XXXX who wrote the library it this was a completely obvious and logical result. Compare that to now. StackExchange. Free Video tutorials, Websites dedicated to teaching you. I constantly meet graphic designers, marketers, accountants, etc. who decided to become developers. I know who guy who has built a decent business with no official development education at all. He learned rails and shoved up to heroku. He learned javascript to improve the client side, and now he makes a decent living.<p>You can start from NOTHING and in a few weeks have a base product built and being used by an alpha customer. That never would have happened 20 years ago. I feel freer and more productive now as a developer than anytime in my career.

I agree 99% with the point of the author. I would like to add my two cents how complexity and diversity of tools &amp; ways costs software developpers :<p>Yesterday I attended a conference by a young but experienced Phd Java engineer about his experience in a research project using NodeJS (server side web programming in javascript).<p>Basically he explained he needed 6 months of project time to get up to speed with the language and the ways of the platform (asynchronous callbacks...) and that is with mentoring of a more experienced dev in JS. For me this learning curve and the mentoring time is a lot of $$$$ and opportunities for suboptimal work.<p>The profile of this dev, and is ability to perform a fine talk makes me thing his capabilities are not subpar. Do we really expect doctors, architects others high profile professions to work during many months suboptimally, requiring special mentoring, just to use new tools ? It seems to me that their initial training is supposed to be all what they need for a decade or more.<p>What would you think if your surgeon needed to work 6 months with a tutor because his hospital bought some new chirgurgical tool ? Basically were are shooting ourselves in both feets by using tools evolving too fast for us to master in any sensible way.<p>Turing told us all languages can do the work. Yet we are still inventing new ones. Maybe we should focus our attention on something else to improve our output.

The author is kind of all over the place, and it&#x27;s hard to read it without thinking of counterexamples to a lot of his points.<p>If his general point is to bring computing to more people, I think a modern, web-based version of MS Access would be a huge step. Everyone needs databases. Everyone needs forms to access them in a way that makes sense for their business. Right now you have to actually program to make that happen.

<i>&quot;In the old days there was a respected profession of application programming. There was a minority of elite system programmers who built infrastructure and tools that empowered the majority of application programmers. Our goal was to allow regular people without extensive training to easily and quickly build useful software.&quot;</i><p>There are lots of people writing software today (probably the vast majority) who are not &quot;elite programmers&quot;. Here are just a few examples:<p>1. All over the academic world, you&#x27;ll find grad students in physics, biochemistry, etc. hacking together research software in Python (using numpy&#x2F;scipy), R, and many other languages.<p>2. All over the business world, you&#x27;ll find non-programmers writing programs in VBA and other end-user oriented languages to slice and dice data from databases.<p>3. The average CRUD code that powers today&#x27;s web startups usually doesn&#x27;t require anywhere near an elite programmer to create. Think of all the articles on HN written by a &quot;non-technical founder who learned to program in two months and created a site that makes thousands of dollars&quot;.

I&#x27;ve been working on a startup that addresses this.<p>It is best described as &quot;a front-end as a service&quot;: use our interface and API to build a cross-platform native mobile app, with a matching mobile-friendly web app.<p>Targeting companies who need a cross-platform whitelabel app to reach their customers or employees, but whose core competency is not flashy front-end software. Although I haven&#x27;t done in-depth market research on the topic, I figure that most companies want some sort of custom app, but not every company wants to try and manage a team of web and mobile developers to do that.<p>Let me know if anyone wants to meet up around SF and chat about this; I&#x27;ve been casually looking for a co-founder while I prototype and look for first customers.

Because it&#x27;s there. The only reason people use all these web tools is because they are there, documented and ready. The classic example is the Apache web server - it was there and worked. It was written when the web was small and the number of users to a site was low. Competing web servers are here now with performance way higher, so people are switching. Nobody wanted to write a better web server so nobody did for a long time - and better in that case was easily measured. Programming paradigms? How do you evaluate them? I agree there are fads (FP anyone?) but what objective criteria do we have to evaluate them, or design something from scratch to meet those criteria?

What is his actual suggestion? It sounds like he is just saying make programming easier. That would be awsome if he actually does it, but I see just the lone idea of making programming easier. How are you going to do that?

&quot;This Archaeology of Errors is no place for the application programmers of old: it takes a skilled programmer with years of experience just to build simple applications on today’s web. &quot;<p>I&#x27;m not sure HTML + JavaScript is any more or less easy than HyperCard + HyperTalk. I agree that nobody would design the web to be what it is right now if they had the chance to start over, but there&#x27;s also something to be said for the force of Darwinism in selecting out decent technologies. And every major web technology right now is just that - decent. Not great - decent.

The proliferation of shitty Javascript is crushing the web. J<p>Javascript is designed from the ground up to enable bad developers to write shitty code. It&#x27;s fine for alpha work, terrible for production.<p>It is possible to write beautiful, elegant, maintainable Javascript- but I&#x27;ve never seen a &quot;designer&quot; or even a &quot;JavaScript programmer&quot; write code like that. I have a job as a code mechanic because so many companies are built on woeful Javascript.

No one mentions the real reason, also because it is the large elephant in the corner of the room we ignore.<p>The truth is - most developers write crappy code. Even right now, here, on this site, plenty of people like their code is fine, it is the code of others that hurts the eyes.<p>We can say that programming is hard, the languages and tools maybe not perfect, but, that&#x27;s not quite it. A lot of developers just want to ship it, get it compiled and out of the door. It passes some tests, so it must be done.<p>If we built bridges like we built software, well, we know how that ends....

I think a lot of people are misinterpreting what the article is saying (and maybe it wasn&#x27;t expressed that well). The author is not saying that we need a tool that lets literally anyone be a developer (&quot;So easy even a manager can use it!&quot;), but rather that the current technology stack is so hacky that it gets in the way of progress, and demands levels of specialized knowledge that are tangential to the task of building web applications: that is, a higher-level set of development tools. Just as (to pick a high-level, desktop-oriented language at semi-random) C# is easier to work with than x86 assembly language, because it hides the messy details, I think the author would like a world where building a web application is as easy as building a desktop application: a world where issues like data serialization, browser-specific JavaScript hacks (and the hacky frameworks meant to solve them), incompatible databases, constant vigilance against easily preventable attacks (cross-site scripting, SQL injection, etc), and the necessity of juggling at least 3 different languages (JavaScript, HTML&#x2F;CSS, and whatever you have on the backend) do not occupy valuable developer brainspace.<p>The issue is that the current web technology stack was never meant for building applications. HTML was designed for static documents. JavaScript&#x27;s early development was mainly for making terrible mouseover-effects on links. XMLHttpRequest is a historical accident. So, yeah, it all works together, somehow, but it wasn&#x27;t designed to be used as it is today, and it shows. And it&#x27;s the developers who carry the burden of integrating these hacks. I, like the author of this article, wonder if it wouldn&#x27;t be nice to design a system to provide the advantages of the web (distributed client-server applications with highly-customizable, visually attractive front-ends delivered on-demand) with a slightly more human development process. If we can&#x27;t replace the web outright, maybe we can build better tools ontop of it.<p>I agree that current web technology is a horrible pile of kludges. Let&#x27;s try to build something better. Some attempts are already being made. Please check out the amazing Ur&#x2F;Web language [1] that lets you write front-end code, back-end code (compiled to native), and database code in a single, type-safe language that statically guards against errors that are rife in dynamically-typed web scripting languages. It abstracts the database, without wrapping your data in an annoying object layer; it lets you write front-end and back-end code in one language that communicate transparently. It is far from a finished product but it is an amazing glimpse and what could be with intelligent develop tools. There are some great examples on-line that show how easy it could be. Easy for a developer, of course, not for a layperson.<p>[1] <a href="http://www.impredicative.com/ur/" rel="nofollow">http:&#x2F;&#x2F;www.impredicative.com&#x2F;ur&#x2F;</a>

If this article&#x27;s entire content was replaced with &quot;Rabble, rabble, rabble&quot; it would have the same amount of substance.