These "down the hole" investigations can sometimes be used as the basis for open-ended interview questions.<p>I always ask candidates to tell me everything that happens between the time that they see an interesting link on the screen, click on it, and see the result. I've had great answers in the space of 2 minutes, and amazing ones that covered 15 or more. Some of the more spectacular answers referenced neurons, muscles, round-robin DNS, firewalls, load balancers, application servers, and more. I can really determine if they know how the web works or not.
Nice article; it reminds me of taking OS at CMU. There's something incredible about learning what happens all the way down when you call, say, printf. Suddenly the computer transforms from an incomprehensible black box into an fascinating marvel of engineering - but unlike a magician's act, the magic only gets better when you see how it works.
It is worth noting this whole series is incredible. I used to work with the guy who started it (@jordansissel), and he was an incredible sysadmin/software engineer. If you go through the archives on the right side, you will find lots more incredible posts (although he doesn't write them all anymore, they are all of excellent quality).
Note, for those following along, that Mac users can use <i>dtruss</i> (which is a shell script that makes DTrace easier to use and more like strace).<p>What a neat article! Makes me want to explore some other commands and write something similar up.
Wow.
Really informative.<p>I have used strace to try to debug unpredictable or unresponsive programs, but it never occured to me to just run it against the commands I rely on that actually work.<p>I did a combination gasp/chuckle when I saw the 'write(l, "bar\n" ...' at the end of the ls dump. There is something both humbling and obvious about seeing under the hood of the platform that I just take for granted.
THE AMULET OF YENDOR: INSIDE THE KERNEL<p>JUST ONE MORE TURN...<p><a href="http://en.wikipedia.org/wiki/NetHack" rel="nofollow">http://en.wikipedia.org/wiki/NetHack</a>