The demise of the low level Programmer.

I have conflicting feelings about this. The low-level programmer is disappearing because the abstraction layers are getting better, which means the complexity we deal in is "how do I get a 30 machine cluster to map-reduce efficiently" instead of "how do I optimize these instructions so I don't blow out my L1 cache".<p>But...all abstractions in computing are leaky, and not having an understanding of lower levels of abstraction <i>will</i> come back to bite you.<p>The simplest example is floating point: how many pennies have been mis-appropriated because a developer used a float instead of a fixed-decimal number. If you understand how the underlying floating point abstraction leaks, you would never use an IEEE float for anything requiring exact decimal values; if not, you feel comfortable using the native type that has the ability to represent decimal values. You don't need to actually know how the float is represented to achieve that.<p>The real trick is understand how much of each layer of abstraction you need to know. A generalist needs to know enough about computer architecure, the network stack, security, etc., that he knows when he either needs to consult an expert (Google often counts ;) or when he is ok to rely on that abstraction, but if you try to learn too much, well, you aren't a generalist anymore.<p>Great generalists are, by definition, great at figuring that out. As a result, they are incredibly valuable because they are really good at just getting stuff done.

I don't think that knowledge of low-level systems programming is ever going away. The order in which we learn it is merely changing. In the past, everyone had to understand the absolute basics <i>first</i> because the higher level abstractions were very leaky. Over time the abstractions have gotten better, performance has become less of a problem for most cases, and low-level systems are no longer something everybody <i>needs</i> to know in order to get by at the higher layers of abstraction.<p>However, that doesn't mean that people aren't learning it anymore. They're just starting with a broad set of high-level knowledge and <i>drilling down</i> into the parts that are important to them. For instance, recently I've been toying with compact string representations of big integers (using bit shifting: how topical).<p>The end result is that you end up with developers who have broad knowledge of the higher layers of abstraction but incomplete and specialized knowledge of the lower layers. That seems perfect to me.

The low level developer hasn't gone. They're just a smaller proportion of a much larger population of developers.<p>That makes them harder to find, but there still there. Wouldn't be surprised to find out there are actually more of them than there were 20 years back.<p>The border of where "low level" begins is also shifting. I, for one, am glad I no longer have to hand assemble my own multiplication code :-)

One book I'd recommend for this is Michael Abrash's Graphics Programming Black Book. Of course it's a bit dated but there's a lot of knowledge to be learned from it and it has good discussions about why and when to optimize...<p>Plus some of the more dated chapter can be seen as a good history lesson :-)<p><a href="http://www.gamedev.net/page/resources/_/technical/graphics-programming-and-theory/graphics-programming-black-book-r1698" rel="nofollow">http://www.gamedev.net/page/resources/_/technical/graphics-p...</a>

In every technology stack, there's always some layer below which is unnecessary to understand and above which is essential to writing good code. Where you think that cutoff belongs is usually the layer at which you, yourself, understand it.<p>I mean, we live in a world where we have programming languages that COMPILE TO JAVASCRIPT.<p>That said, I applaud this guy's OCD fascination with low level optimization. It's only because of people like him that higher level programmers can even exist.

It's possible to descend to lower levels, if given enough curiosity. Some people just want to know how things work under the hood.<p>I can remember starting to program our brand new PET-2001 in BASIC (at 13 years old or so). After a while I became bored with the sluggishness of BASIC, and the cramped memory space. Only very small BASIC programs would fit. But there was a new challenge! I wanted to discover what made the PET really tick -- how did it understand BASIC? Then I learnt about ROMs, RAMs and the 6502 CPU, and how to program it in machine language. It was similar to programming programmable calculators, which I'd done since I was 9. But the 6502 worked much more faster, and had a rich instruction set, compared to those old calculators; and a much better screen, with graphics!<p>Since I was a kid, I always wanted to look into the inside of toys how they worked, for example, my 'Brummkreisel'[1] spintop. At 6, I wanted to open my hamster to look inside, but my mother wouldn't allow it... (Otherwise maybe I'd be a surgeon nowadays ;-)<p>[1] <a href="http://en.wikipedia.org/wiki/File:Brummkreisel_BW_2011-08-12_20-37-51.JPG" rel="nofollow">http://en.wikipedia.org/wiki/File:Brummkreisel_BW_2011-08-12...</a>

I graduated in 2010 with a ECE degree and learned all of this. Essentially all of that material is covered in CMU's 15-213 "Introduction to Computer Systems", which is taken by a huge proportion of the undergrad CS &#38; ECE program.<p>It looks like book from the course, which covers all of that material, is pretty widely used, too: <a href="http://csapp.cs.cmu.edu/public/adoptions.html" rel="nofollow">http://csapp.cs.cmu.edu/public/adoptions.html</a><p>However, my first job out of college was doing low-level optimization at a cellular silicon provider, but now I work on the web since it seems like most hardware companies are bigger and have more suburban locations (neither of which is my preference), and if you're working hardware you can't work remotely. Hardware companies also aren't necessarily the best software organizations.

There's a certain irony that the author laments the loss of low-level developers, yet the site doesn't work without JavaScript.<p>I like that he provides some links for further reading, and I'll bookmark the page for that reason alone.<p>Besides those links, his diatribe seems to be misplaced. For many applications, it's not necessary to know the difference between floating and fixed-point.<p>Perhaps one day, it will no longer be necessary to care about the difference between decimal/integer/floating-point, as the compiler/interpreter will know how and when to use the "right" one. And I will welcome this. I've seen inexperienced developers use float(val) when counting money; and no amount of training will stop the next generation of new programmers making the same mistakes.

Because programmers no longer grow up on 8-bit systems programming in assembly language, they jump straight in to Java or whatever - they've never gotten closer than 10 layers away from the actual CPU, it's all a magic black box to them.

The low level programmer is obviously not gone, as other people have already mentioned. Rather, what I think most of us who've been through a few decades of development can see, is that the hobbyist has become a different kind of person. Years ago, all the hobbyists would go "Oooh! I <i>finally</i> managed to print ASCII on the screen. Ok, now let's spend another day trying to get it working in color!" In 2012, there is (in most cases) very little point in doing this, and the 'hobbyists' we see now are mostly building websites 'n stuff.

There are lots of jobs for low level programmers outside of games too. I've worked for ITA Software (mentioned by PG in Great Hackers), Endeca (recently bought by Oracle for $1B, and we had nothing to do with photo sharing), have been wooed by robotics companies, and now work on file system caches with a peer-to-peer component.

I looked at the title and thought "good riddance", assuming it meant mediocre maintenence-only programmers were being replaced by automated-programming/systems, just as low-level lawyers can be replaced with document scanner/databases.<p>But systems programming seems to be an essential role that uptime-concerned operations will always want on retainer...

The low-level programmer isn't gone, but over the past decade we've seen the rise of the meteoric rise of the high-level programmer.<p>For instance working on web apps, when you an extra server costs $400/month (4 developer hours or less), what value is there in spending days hand-optimizing an algorithm (into a form that'll be harder to debug later)?<p>Another example: writing mobile apps. On a phone today you have 1 GHz of power, but still a very limited interaction model. It seems basically only games actually need to optimize for performance.

I've always felt attracted by low-level systems and I'm currently one of many students in Comp. Sci. For the moment all by love goes to all kinds of bare-metal stuff, optimization tricks etc. I'd like to go all the way through it but being about more abstracted development (although the term looks unproper to me, since being able to grasp a machine's ways is a valuable effort of abstraction itself) seems like what is and will be expected of me.<p>So I'm asking : career-wise, are the optimization ghettos (video games, embedded systems, critical systems etc.) all that one could expected when refusing to code and manage projects around buzz-words or is there actual innovation / change and a shiny future to come for bit nerds ?

I started life mostly doing low level stuff - and still do as needed. But, it has been about 4 years since needing to do any assembler (which was for windows mobile). Embedded systems still need some of this.<p>That said, my day to day life lives more in the realm of scripting/interpreted languages and occasionally diving into C/C++. Even in C++, though, frameworks like Boost make C++ seem higher level than it is.<p>Today, we see a need for people that can work on Cobol and companies providing Cobol=&#62;Java bridges. I suspect the need for the lower level programmer may occupy a similar nitch in a few years...

Your low level is my high level.<p>Even assembly code appears high level when you are looking into how the electrons run through silicon and copper. But it's not realistic to cover all the stack down to the atoms. Then which layer you can stop digging at?<p>For a programmer, the processor architecture (including the instruction set architecture and the bus architecture) level would be fine. It is the strongest abstraction barrier and you rarely need to look below it.<p>That said, if you are working on embedded systems, knowledge about digital electronics is almost a must, and some analog bits really helps.

Wise rule says "Don't tweak efficiency until needed". Since we have fast CPUs/GPUs and faster are made very often tweaking efficiency is less needed. I think this rule is sometimes overused, but what can I do? :P Anyway, there are still people that know much about internals even if they don't use it at work. :) Currently I code in Perl and can tell you something even about pageing internals. By the way we had quite nice class on it on university. :)

Meanwhile, half of Hacker News is implementing DCPU-16 compilers. The low level programmer is still flourishing, there's just a lot more people programming.

It looks like several people have interest in this as a career path. Personally, I'm tired of it and am trying to move up the stack. But for those interested in terms of a career path: <a href="http://www.eetimes.com/design/embedded" rel="nofollow">http://www.eetimes.com/design/embedded</a>

I'm surprised this article is still circulating after the hits it received last august. If anyone would like to discuss the subject further or needs some help breaking into a role like this then please feel free to ping me directly through linkedin/facebook<p>- andy firth (the author)

Low level programming <i>of PCs</i> perhaps.<p>There is more embedded code and applications out there than ever before, and tons of mobile devices that need similarly low level code.

I'm surprised no one has mentioned the story of Mel yet: <a href="http://www.pbm.com/~lindahl/mel.html" rel="nofollow">http://www.pbm.com/~lindahl/mel.html</a><p>Low-level programmers have been dying ever since the first assembler was invented.

Come, come<p>Up and coming low-level programmers still exist. I just turned 20 this past January and I've been doing x86 at least once a week for 5 years now, in fact, my meal ticket -- IDA Pro, sits as I type this whirring away at calculating the relocations of some PIC in our favorite browser.<p>While it may be true that that the majority of people with assembly knowledge are slaving on heap sprays more than swizzling pointers or tackling alignment issues these days (or any other "real" low level work). All and all, auditing isn't half bad and I suppose the markets agree.<p>I guess to that end, There's going to be no shortage of low level junkies as long as kids still think hackers are cool. Kids are going to be attracted to the cyberpunk glam of hackers (or, naturally, Angelina Jolie's smoking body in 1995). Of course, as it goes, If you want to be any good at that sort of thing, you have to write memory corruption bugs, and if you're writing memory corruption bugs then low level is a necessity.<p>So while it may be true that producing anything save VX crypters in assembly is a dead end, the deep knowledge is immensely rewarding and while it's true you might not be able to erect castles, you can perform some deep machine magicks that will make a few people's heads turn.<p>You unlock many things that many programmers simply dream of -- The ability to reverse applications, to tweak some 'interesting', decidedly-nonoptimal optimizations (LEA is always faster!) and find cleverly exploitable bugs in applications that you don't have to source to.<p>You can optimize in ways that other programmers, no matter how skilled in any other language, simply cannot. A recent and related anecdote is my database ( zv.github.com/artifact ) which, once upon a time, was extremely slow during the mkey exchange when it diffs the hashes against it's own, it was copying every hash and comparing it to every other hash it had received thus far (For when quadratic time is too fast). Normally this would have been a few hours long job to rewrite some pithy external library, write tests and somehow implement speedy string manipulation in Erlang. Nope, HiPE allows you to treat an object file as a module as long as it has the correct export interface. My humble database is now an order of magnitude faster on convergence (This may speak more towards the performance of string processing in Erlang than any personal prowess, but still).<p>So, in conclusion Mr. Andy Firth, it's probably true that the vast majority of people who you are interviewing aren't very good, because hey, that's the central limit theorem at work!<p>However, I assure you, somewhere out there, there's a boy and his linker, learning low level the hard way. If I were you, I'd hit up security and reversing/crackme forums, you're practically guaranteed to find a wizard or two.<p>That's the Circle of life :)

For what it's worth these topics are taught in university to a certain extent.<p>I took a class last semester where we had to write an assembler and virtual machine for MIPS (in C), had exams on branch prediction, instruction pipelining, program-performance as impacted by different types of caches.<p>But really, there is a lot of information to absorb. Intel's been doing heavy research on processors for decades. It's one thing to be familiar with the concept of branch prediction and instruction pipelining, it's another thing to know enough to actually have a [positive] impact on the performance of a program.

Previous discussion: <a href="http://news.ycombinator.com/item?id=2857422" rel="nofollow">http://news.ycombinator.com/item?id=2857422</a>