John Carmack discusses the art and science of software engineering

<i>With the NASA style devel­op­ment process, they can deliver very very low bug rates, but it’s at a very very low pro­duc­tiv­ity rate</i><p>I wonder how many non-developers understand this. I, along with the rest of my team, am trained in PSP (<a href="http://www.sei.cmu.edu/library/abstracts/reports/00tr022.cfm" rel="nofollow">http://www.sei.cmu.edu/library/abstracts/reports/00tr022.cfm</a>) and TSP (<a href="http://www.sei.cmu.edu/tsp/" rel="nofollow">http://www.sei.cmu.edu/tsp/</a>) and we use it in our day-to-day development.<p>It definitely helps us keep our defect rate below one bug/kLOC but it's an <i>expensive</i> process that results in very low LOC/day productivity. If very low shipped bug counts are very important to your organization, great. But most businesses these days seem to care more about having a <i>usable</i> product than they do a <i>perfect</i> (or close to it) product. Especially if it's on the Web where you can do multiple releases per day.<p>As an industry, we really need to bear in mind that different business domains need radically different approaches to software engineering.

I've gotten to know John a little bit, and I have to say it's a strange feeling to have a conversation with a person of his off-the-charts intelligence. I consider myself to be pretty smart-- was known as "the math whiz" in high school, went to top engineering universities and did very well there, have a couple of (minor) entrepreneurial accomplishments, etc. And yet talking to Carmack I feel like I'm talking to someone who is a full two standard deviations beyond me in raw intelligence horsepower. It's a pretty sobering and humbling experience.<p>Part of it is that he really does spend 8+ hours per day coding, every weekday, and has done so for 20 years. You'd think his experience level there is about as high as you can get, so it's always cool to hear him talk about the new things he's still learning at his work. I have to wonder if there's anyone else in the world that has both his raw ability <i>and</i> all those man-years of programming experience. It seems like most successful technical people end up doing management and business.<p>There's a couple of things people probably don't know about Carmack. For one, he can talk intelligently on a lot of different topics. A lot of nitty-gritty aerospace engineering, as well as the history of the space program and NASA for example. He's also up to speed on the latest across a wide range of technology, including things like cleantech.<p>Second, he has a pretty good sense of humor and can be quite funny. Which is surprising I think just because he spends so little time (effectively zero) out being traditionally social, which you'd think would be necessary to getting good at making people laugh. But in conversation he has a pretty sense of comedy and timing.<p>An example from his twitter feed that I clipped a while back:<p><a href="https://twitter.com/ID_AA_Carmack/status/167739644853747712" rel="nofollow">https://twitter.com/ID_AA_Carmack/status/167739644853747712</a> "Adding film grain, chromatic aberration, and rendering at 24 hz for film look is like putting horse shit in a car for the buggy experience."

He is definitely right about the social aspect of software engineering. But I think he sells a lot of the tools short. For example, he brings up things like Monads, Lambda Calculus, and whatnot - but then immediately dismisses them as not affecting what one truly does.<p>But I think this really misses the point. In our industry it is really easy to disguise oneself as a professional (or even just someone who knows what they are doing), without really knowing much of anything. Meaning, our focus as an industry has been on making the simple things as simple as possible (i.e. scripting/dynamic languages, code generation, frameworks).<p>But what I see happening in the Haskell space for example (and even further in languages such as Agda) are attempts to distill things down to their elements. To find the true semantics behind a problem. This not only helps by producing cleaner and more readable code, but it also helps with communication.<p>I really do believe software is a scientific (and mathematical) exercise. The problem is most of industry does not treat it as such, and hence we end up in the mess we are in.

I program mostly in Objective-C nowadays, but I started professionally with PHP. When I first started Objective-C, I found it really constricting in comparison. In PHP you can do things a thousand ways, most of the terrible. In Objective-C, specifically with Cocoa, things are a lot more rigid and prescribed. I found this frustrating, but love it now. It's made my PHP better too when I do occasionally go back. It forced me to think more about architecture. I also understand why my CS dept taught us Scheme first, not Java.<p>To link that back to the post, this is the type of constriction he's talking about to make better programmers. Cocoa and Objective-C restricted me to only writing at least halfway decent code. With PHP, because of it's flexibility, you're free to get things done quickly, but in a terrible way. Sure with PHP you can do things right too, but it takes a lot more self-discipline and also a priori knowledge.<p>Sorry to post yet another rag on PHP.

I would suggest watching his entire talk on youtube: <a href="http://www.youtube.com/watch?v=wt-iVFxgFWk" rel="nofollow">http://www.youtube.com/watch?v=wt-iVFxgFWk</a>

Running your code through static analysis can be eye-opening. And just like when you opened your eyes for the first time... you'll probably cry.

The quest for perfection may be futile.<p>DNA is also code, and it's full of bugs. That code lives for hundred of thousands of years, if not millions.<p>Biological processes offer the suggestion that your system can be functional in the face of constant failures and random variations in behavior.<p>Biology can even offer a very high reliability rate. While we get sick all the time, and people are born with all sorts of genetically disadvantageous traits, many key processes are mind-bogglingly reliable. (No sight v No sense of touch: Compare the rates of blindness to the rates of congenital analgesia type 2)<p>While the math behind CS offers tantalizing guarantees of reliability the reality of software development and developers deliver a reliability far lower.<p>I think it is a fascinating thought experiment to imagine a development process where instead of writing any code, all you're writing is tests (or feature descriptions) and let the code adapt to the environment you've defined.

One of the things I've been persuaded of is that software writing is fundamentally a non-scientific activity. Outside of the time and space constraints for a given subsystem, the craft of software is almost entirely subjective and limited only by mental capacities and flaws.

Richard Feynman wrote[1]:<p>"We could, of course, use any notation we want; do not laugh at notations; invent them, they are powerful. In fact, mathematics is, to a large extent, invention of better notations. The whole idea of a four-vector, in fact, is an improvement in notation so that the transformations can be remembered easily."<p>What he said about mathematics, I think it applies even more to programming.<p>[1] The Feynman Lecture on Physics, Volume 1, Chapter 17

Actually, I think one of his strongest points is about making mistakes -- everyone does them and everyone makes the most amateur of them. If you haven't tried running SCA on your code, try it sometime. I've got a ridiculous ego, but when you get hundreds of warnings on your code, you realize just how imperfect you can be and how impossible it is to be cognizant of all things at all times. This is why teams are almost always better, and I'd prefer to review and be reviewed by someone else -- I'm my own biggest blind spot.

If he wrote a book about programming, I'd buy it in a heartbeat. Too bad that even he probably is still figuring everything out.

I look forward to the QuakeCon keynote every year. Anyone who engineers software should really listen to Carmack.<p>Also, this article's kind of silly... there's almost no discussion? Just watch the video.<p>EDIT: I guess it's not so much an article as a sharing of info. Still. Watch the video.

As much as I respect John Carmack, I have to say that I'm a little disappointed that he is rehashing this meme of software development not being a science... OK, wonderful, the state of the art in his shop doesn't rise to the level of being a consistently reproducible, measurable process, but that doesn't mean that this is a permanent condition or that its an insurmountable one.

Interesting comments on Mac and Linux platforms [1]:<p><i>"Other interesting sort of PC-ish platforms, we have... the Mac still remains a viable platform for us. The Mac has never required any charity from id, all of those ports have carried their own weight there; they've been viable business platforms.<p>I actually think that the Mac is going to become a little bit more important for us. Interestingly, we have a ton of people that use, like Macbooks at the office, but we don't have any really rabid, OS X fanboys at the company that drive us to go ahead and get the native ports out early.<p>But, one of my pushes on the greater use of static analysis and verification technologies, is I pretty strongly suspect that the Clang LLVM sort of ecosystem that's living on OS X is going to be, I hope, fertile ground for a whole lot of analysis tools and we'll wind up benefiting by moving more of our platform continuously onto OS X just for that ability to take advantage of additional tools there.<p>Linux is an issue that's taken a lot more currency with Valve announcing Steam for Linux, and that does change, factor, you know, changes things a bit, but we've made two forays into the Linux commercial market, most recently with Quake Live client, and, you know, that platform just hasn't carried its weight compared to the Mac on there. It's great that people are enthusiastic about it, but there's just not nearly as many people that are interested in paying for a game on the platform, and that just seems to be the reality. Valve will probably pull a bunch more people there. I know absolutely nothing about any Valve plans for console, Steam-box stuff on there; I can speculate without violating anything.<p>One thing that also speaks to the favor of Linux and potential open source things is that the integrated graphics cards are getting better and better, and they really are good enough now. Intel's latest integrated graphics cards are good. The drivers still have issues. They're still certainly not going to blow away somebody's top of the line SLI system, but they are completely competent parts that are delivering pretty good performance.<p>And one of the wonderful things is that Intel has been completely supportive of open source driver efforts, that they have chipset docs out there, and they work openly with community to develop that, and that's pretty wonderful. I mean, anybody that's a graphics guy, if you program to a graphics API, use D3D or OpenGL, you owe it to yourself at some point to go download the Intel chipset docs. There's hundreds of pages of them, but you really should read through and see what happens at the hardware level. It's not the same architecture that Invida and AMD have on there, but there's a lot of commonalities there. You'll grow as a graphics developer to know what happens down at the bit level.<p>Another one of those things, if I had more time, if I could go ahead and clone myself a few times, I would love to be involved in working on optimizing the Intel open source drivers there.<p>So, it's enticing, the thought there that you might have a well-supported, completely open platform that you could deliver content through the Steam ecosystem there. It's a tough sell on there, but Valve gets huge kudos for having the vision for what they did with Steam, sticking through all of it. It's funny talking about Doom 3, where we can remember back in the days when they're like, 'Well, should you ship Doom 3 on Steam, go out there, make a splash?' ... I'm like, 'You're kidding, right?' That made no sense at all at that time, but you know Valve stuck with it and they're in a really enviable position from all of that now.<p>It still seems, probably crazy to me that they would be doing anything like that, you know, but, it's something that's not technically impossible, but would be really difficult from a market, sort of ecosystems standpoint."</i><p>[1] <a href="https://www.youtube.com/watch?v=wt-iVFxgFWk#t=44m28s" rel="nofollow">https://www.youtube.com/watch?v=wt-iVFxgFWk#t=44m28s</a>

Software development is usually about creating some sort of competitive advantage. And a fundamental tenant of competitive advantage is differentiation.<p>Standard implementations/algorithms/patterns are commodities and are purposely so.

The talk itself is well worth the 3 hours.

i just finished the devbootcamp program so as a person who was thrown into oo programming with no real cs underpinnings, it is interesting to hear him talk about the social component . it is also pretty interesting to here him talk about making mistakes and the need to just get things done versus optimization.

I don't want to be an ass, but this seems like a lot of vague contradictory rambling. Maybe I missed some big ideas in the parts I skimmed?