I don't agree with this article. I think DNS is something few people take the time to learn, but it's not actually hard to learn. One of the great things about DNS is that the system itself will tell you about it's internal state in response to queries. It's very easy to inspect a DNS server for a known zone and understand how it works, and there's very good tooling that's free and widely available to do this (like dig).<p>It's always been a big surprise to me that my DNS expertise is what seems to be most memorable for a lot of folks I've worked with through my career, when I don't believe I know anything mystical or special. DNS is extremely well standardized, the most common server and client implementations rigorously follow the standard, and it's very easy to inspect with free tooling. It just takes some effort and time to learn, but it's not really hard.
I think the article hits the nail on the head. DNS itself isn't hard, but learning real-world DNS is, due to the (as described) hidden nature of many parts between intending to do a lookup and getting the expected result.<p>Some of this also stems from the divergence of what used to be a 'default' way to be connected to the internet (one interface, one gateway, one DNS server provider) and what is now far more common like having to figure out how to actually do the lookup in the first place (connected to multiple WANs, sometimes concurrently like LTE and WiFi) and as a user, not actually being able to tell which resolving path was used (was it the browser? the standard interface in the system C lib? was it some local resolver or recursor that sits in between the lookup? do any of them have a local cache? do any of them add special options by default?).<p>Even if everything works, you'd still not be able to blindly trust if the query and response used the same path in one application as it did in another. Three browsers might all use something different, and the OS itself might use something different yet again, only to have mDNS add a fifth option.
Because the only three hard problems in computer science are cache invalidation and naming things.<p>And DNS is a caching system for names of things.<p><a href="https://reddit.com/comments/15c2ul2/comment/jtty9dy" rel="nofollow noreferrer">https://reddit.com/comments/15c2ul2/comment/jtty9dy</a>
It's one of those things where there is a mismatch between how easy it seems to be, and how hard it turns out to be.<p>We all use DNS every day, and it <i>seems</i> really easy. The everyday language of DNS is: domain names, lookups, IP addresses. This language is exposed in browsers for all to see, and through this exposure we develop a mental model of how we think it works.<p>But under the covers there is a whole new language: zones, resolvers, delegated authority, that weird dot <i>after</i> the top-level domain...
I went through this a few years ago, deciding that I had only a piecemeal understanding of DNS based on the specific things I had run into. I knew about `dig(1)` and BIND and a CS101 idea of how recursive DNS resolution works, etc. But I was missing the working knowledge needed for designing and implementing anything non-trivial or for debugging non-working systems. So I read "DNS and BIND" (not quite cover-to-cover, but close, skipping over the details on some of the more exotic features). I set up a real BIND server for some unimportant personal web sites. None of it was hard, but it did involve a bunch of (time) investment. To be clear, BIND is not the right thing for many use cases, but a lot of DNS ideas and terminology still come from BIND and I found that stuff very valuable.<p>I think books are underappreciated for learning stuff like this. Most resources you find on the web are high-level theory ("here's a block diagram of how recursive lookup works"), task-oriented ("how do I get `dig` to do a recursive lookup"), or otherwise low-level (e.g., reading the source of your local DNS client to understand its retry policy). To understand the pieces and how they fit together, from the theory (what they're trying to achieve) down to implementation (what caches exist where), I find there's no substitute for a holistic approach that you usually find in books, but rarely (not never) find on the web.
It's probably a good idea for all IT people to have a working knowledge of how to debug DNS issues.<p>DNS has historically been a vector for <i>significant</i> security holes and it's likely that this will continue to be true for the indefinite future. These holes also lead to other vectors in nearly every other protocol like SMTP. Even the CA system used for HTTPS is highly dependent on a basically insecure protocol. (Would you notice if your bank bought a DV certificate instead of OV? likely not)<p>So, perhaps it's not such a bad thing that it <i>seems</i> hard to learn to those who don't have enough interest, since even now we see people building DNS things without taking the time to really understand the history of things like port randomization, cache poisoning, AXFR, etc.
Shameless side project plug: they mention a "debug" mode for dns resolving being nice to have. ComfyDNS has this in its web UI :3<p><a href="https://comfydns.com/" rel="nofollow noreferrer">https://comfydns.com/</a><p>It's the picture that says "TRACE google.com A IN" at the top.<p>ComfyDNS is partly scratching a personal itch - I was tired of hand modifying bind9 zone files. And also I was curious as to how DNS works - I knew surface level stuff but no details. So I implemented the RFC from "scratch" (I used netty but no DNS libs). It was a lot of fun.<p>(Also if/when the site goes down from hugging, forgive me, it is a rails app running on the oracle cloud free tier lol)
Another one of these articles. We learnt very easily back in the 1990's when the Internet was smaller, and the computers were much much slower and less capable.<p>DNS, LDAP, SMTP, IMAP etc were the bread and butter of ISP's back then and people actually referred to the official documentation (RFC's etc). You had to learn them if you wanted to run servers on the Internet at all, and with a bit of an investment of time (ie. your paid time on your job) you learned it.<p>This generation of developers and devops people don't have the patience or initiative and expect to be spoon fed and just cut and paste crap from StackOverflow and various low value blogs. Rather than learn the infrastructure that the Internet is built on, they grab the latest fashionable wrapper tool of the week, follow some shitty blog instructions, and then cry foul when it all falls apart and they cost their company lots of money. Just because they didn't take the time to learn the foundations of how things <i>actually</i> work on the Internet.<p>I've seen it time and time again. It's not actually that hard kids. You just need to do your homework.
It’s not. It’s one of the few things that hasn’t changed much and it’s operation is fairly straightforward.<p>dig is a little confusing. It’s more capable but less straightforward than good old nslookup (which still works fine BTW).<p>I think partly DNS and the core protocols may seem confusing to younger people in the industry because so much stuff “just works” now.<p>For example, today wifi routers “just work” right out of the box. In the early 2000s it would have taken a network engineer with knowledge of DNS, IP, Ethernet, RFC1918, actual routing protocols and whole bunch of other stuff to set something like that up and they’d have well known how it worked and why it was configured the way it was.<p>If you think DNS from a client can perspective is confusing, try configuring BIND ;-)<p>/OldNeckBeardRant
I found out about <a href="https://www.nslookup.io/learning/" rel="nofollow noreferrer">https://www.nslookup.io/learning/</a> recently, which greatly increased my knowledge of DNS. If you look at the list of DNS record types [0], you might be surprised at how many their are. Knowing how to use those can be a bit much.<p>[0] <a href="https://www.nslookup.io/learning/dns-record-types/" rel="nofollow noreferrer">https://www.nslookup.io/learning/dns-record-types/</a>
DNS is easy in the same way that chess is. The game mechanics are straightforward, and it gets more complex from there.<p>DNS bears the burden of delivering you to complex IT systems. It’s abused in various ways to enforce geographic restrictions, service levels, etc. It generally works, so long as everyone upstream knows how to configure things so that downstream things they don’t know exist work well.<p>When things don’t work… that’s not easy.
I feel like DNS is one of the more straightforward protocols, especially on a practical level, and especially given that most interfaces are a dropdown and two text boxes.<p>I have noticed a lot of developers shy away from it, probably because they don't use it much or it's not their job (rather than it being hard).
One of the best, but also strangest explanation of DNS I've seen is from "A Cat Explains DNS". It's wonderful.<p><a href="https://www.youtube.com/watch?v=4ZtFk2dtqv0">https://www.youtube.com/watch?v=4ZtFk2dtqv0</a>
As a nslookup user since before dig, i find the default dig output to be less than ideal. The top->bottom flow, the "important" information is always the third option on a line, etc.<p>Basically how do I put this, uh, its obviously written by someone for themselves and people who use it everyday.<p>So, for most things I continue to use nslookup, it gives me what I want 99% of the time, and for me its easier to parse.<p>So, various other people have said something similar but it reminds me of a couple utilities in AIX that an office mate of mine wrote. The thing was a very sharp tool but the 3 cases that comprise pretty much 100% of its usage required three+ mode selection parameters, etc just to print the equivilant of "its working" or "its not working". The result was 50 characters of typing that quickly turned into a shell script with two parameters. I sorta have the same thing to say about the "ip" command which showed up at the same approx time as dig. Yup it can do everything, but its a terrible UI too for all the same reasons. The defaults are ugly/hard and the hard stuff still takes 10 mins reading the man page for the once every two years you need to do it.<p>So worst of both worlds.<p>PS: I sorta feel the same way about bind, its written for people who run big public DNS servers, the rest of us are better served with "unbound" or the half dozen other utilities that are more oriented towards users looking to run an in-house split dns/caching resolver setup.
I remember being stymied by the BIND zone files (which are plain text) being called .db files (they may actually be called anything -- some people used the .db extension). I had just learned about sendmail, and I knew it kept alias tables in Berkeley DB files (which are binary), which in sendmail parlance were also called .db files.<p>As Alan Kay is fond of quoting Bob Barton... "Systems programmers are high priests of a low art."
I'm not a "developer" and I learned DNS without any problems. Therefore agree with other commenter that DNS is not actually difficult to learn. I like the output from DNS utilities such as BIND and tinydns format.<p>DNS is worth learning for any internet user, IMHO. I've written primitive utilities that when used together can do stuff none of the popular DNS utilities can do. I use these every day.<p>Here's DNS challenge for readers. Try to look up the domain "bootable.com". Use whatever is the preferred method.<p>People writing about DNS often compare it to a telephone book. IMO the way most people use DNS is more like "directory assistance".<p>IP addresses do change but by and large most stay the same for long periods. Old zone files, publicly available scans^1 and other bulk DNS data collected from web crawls and public DNS caches comprise the "phone book" for me. Absolutely essential.<p>1. Sadly, in recent some of these sources have changed to non-public. No phone book for you! Call directory assistance.
For me, the issue isn't so much DNS itself, but more that there are more things that try to resolve names on your PC, like /etc/hosts and all the stuff you put into resolv.conf.<p>Then you read online that you should try dig or so and those tools don't really match up with what your tools do(=resolving via libc), because dig really only does the DNS part.
One of the gotchas I encountered is that DNS is asynchronous, with possibly a long delay before reply. C apis make it look synchronous - which I think makes it harder to work with. There's also the detail that order of replies can be any.
(I found too many developers expected synchronous and instant replies)
This was an outstanding writeup, and I have felt the same frustrations in many other contexts. I enjoy going the "make your own, friendlier tools" route.
sometimes it's hard to even know WHERE to change the settings. Last week a friend was trying to setup heroku with api.foo.com and he needed to add a CNAME to the domain so heroku would make the cert and turn it on.<p>I used dig, i used host, I used whois, I got invited to their aws route 53 and saw all sorts of stuff in there but each change had no affect. Finally I noticed from whois that the name servers weren't even aws they were google.<p>So they gave me access to the google account but no domains in there.<p>Finally I asked, have the CEO log in to his personal google account and sure enough, that's where the change could be made.
Here's what's cool about the article:<p>- Presents some nice theories which make things hard to learn (infrequent use, poor tools...)<p>- Describes how DNS tools could be improved.<p>- Gives you a few gotchas for how one may shoot themselves in a foot with DNS.<p>Here's what's a bit (not much) less cool:<p>- I really have no clue if those things ACTUALLY make things hard to learn (because it's not a research paper on learning).<p>- It's a plug for other content on the side which actually describes the DNS protocol. I'll admit the sold content looks cool. I haven't purchased and can't vouch for the actual quality.
At one point, I had to make and maintain my own DNS zone text file, knowing only a little. Today, cheap providers give me various Web forms interfaces, but that doesn't mean it will always "just work", so knowing a little can still help.<p>I still had to break out `dig` the other day, when a DNS provider started answering records days past TTL, breaking email and Web.<p>To know when if/when that happened again, I looked around for a monitoring service, but didn't quickly find one that did what I wanted (and was overwhelmed by SEO, and very aggressive robo-sales emails from one of them), so I wrote a script that essentially runs `dig` and `diff`: <a href="https://www.neilvandyke.org/check-my-dns/" rel="nofollow noreferrer">https://www.neilvandyke.org/check-my-dns/</a><p>The monitoring script discovered one of a provider's DNS servers had ongoing problems, so I sent them a monitoring report, saying I wasn't a DNS expert, but perhaps, if there's a problem, the report would be helpful to their DNS experts. This seemed to immediately get past any "have you tried rebooting Windows" front-line flowchart response that I wouldn't have been surprised to hear, and they said they're working on it.
'Still'? There's a famous talk[1] in the network community about how DNS complexity is growing so fast that effectively <i>nobody</i> can keep up with it. There's so many competing, overlapping, and overriding RFCs that it's hard to make sense of it anymore.<p>[1] <a href="https://blog.apnic.net/2018/03/29/the-dns-camel/" rel="nofollow noreferrer">https://blog.apnic.net/2018/03/29/the-dns-camel/</a>
In a kind of corrollary of Dunning-Kruger, there is often a chasm between what people think DNS is and all of the things it can be. The article points some of that out (stub resolvers, even different implementations of libc); recursive versus authoritative responses; the recursion process; recursive versus authoritative servers.<p>A lot of deployment implementations are "bottom of the barrel" and aren't correct to begin with, although they work for the intended purpose. There is no checklist for server implementations which I am aware of (I've asked where people should know!). There's a lot of folklore which persists because if it works it's presumed correct.<p>There's "DNS" and then there's "The DNS", the "one true root" with arbitrary restrictions on the contents of labels. There's political interference with implementations in terms of the "one true root" doctrine, which interferes with marrying a resolver which e.g. queries a control plane and serves an application plane (where they have different roots): there are e.g. notions of forwarded zones, but there is no notion I am aware of of a "always recurse and lie that you're authoritative" zone (you can hack source code to accomplish this of course).<p>Even MITRE ATT&CK doesn't always get it right. They had it listed that DNSSEC "traffic" had to be examined with SSL tools, until I pointed it out (I didn't get credit). DNSSEC related records are ordinary DNS records, nothing is being encrypted.<p>I could go on: the experts were wrong on (UDP) frags, anycast...
DNS is complicated because it's old, and has been continuously revised in flight, to uplift to TCP, DoH, DoT, add DNSSEC, EDNS0, TSIG, notify, ECS, EDE. It continues in one ietf WG having outlasted another, it has cross area review. It's political, and technical, and has its black helicopter cabals.<p>I predate the DNS, and I continue to befuddle myself about details of the DNS on a daily basis despite a life embedded in DNS.<p>If you love DNS join DNS OARC.
djbdns is simple, easy to understand, easy to configure; it embodies a clear understanding of how DNS works.<p>Unlike BIND and dig, it was designed after DNS had been in use for a while.<p>Like sendmail, BIND suffers from being designed before anyone knew what it would need to do.
"(like, what if you could run dig +debug google.com and it gave you a bunch of extra debugging information?)"<p>The flag they're looking for is +trace and it provides exactly what they're looking for which is a path of how resolution occurs from root nameservers all the way down to the domain.
tl;dr: suggestion: DNS is really hard to unlearn.<p>I do not agree with the implied sentiment of the article.
And thus with some of the comments (from "both" sides) of the arguments presented here in the comments.<p>Hard is not defined, but is implied to be "harder than you believe it is."<p>That is a general characteristic of perhaps most topics worth pursuing in our field: computer science/engineering. (Defining it in this closed field space on purpose for brevity.) And especially in distributed systems, which DNS in some way falls under.<p>Further you have business interests, which have affected and will affect DNS -- we like it or not. And have added on top of it.<p>A number of individuals then use hard as "it is intractable" [to learn].<p>Is DNS harder to learn than BGP routing: I think not.<p>Does one need to spend a decade focusing on using and debugging DNS. I would argue not.<p>Does one need more time to dive into BGP than DNS to do state of the art work: I would argue yes.<p>DNS is simply not the hard and hot scientific problem people go around tackling from a research perspective.<p>It is a federated business (and political perhaps) problem at this point.<p>I do agree there is a lot of caching involved, which creates a ton of coherence and invalidation issues. That does not make the topic hard, it makes DNS problematic. (The other word I would use, and that I think Ms. Evans [the author] implies perhaps.) I agree totally with that.<p>I also think it is a point of approach the learning path to it for the average scientist here. A lot of people are introduced to DNS (or email to that extend) as: here is the sketch idea, we make sure it works, trusts us, and there is this single deified entity that makes sure it works, and it is not patched with other branches "constantly."
All of the aforementioned sentence of course is a lie. But a lot of engineers learn that.
(I was lucky to not have that introduction.)
And then it comes time to debug a DNS issue -- when they actually realize it is a DNS issue.<p>So I would recommend a revised blog title to: DNS is really hard to unlearn.
I love the flamewar that this post has started<p>There's the "it IS hard!" gang<p>and the "it's not hard, you're just dumb"
gang<p>and the "it's not hard, you just haven't spent the time required to properly learn it" gang
Because complexity is easy. It practically generates itself. It has a natural economic advantage.<p>Making sense of complexity, otoh, takes much diligent effort.<p>So complexity tends to win and everybody tends to befuddlement.
I used to specialized in these service enable services. DHCP/DNS/AAA/LDAP and etc; low level stuff tend to get take for granted. It's not difficult but there are much nuances. Not typical visible to layer7/Front end development per se. Embrace the "and". If you are a FE/BE dev AND understand system/dns/network. You just set yourself apart from the next person.
Great article. Thank you!<p>I used to develop and maintain one of the World largest DNS services and summarised my experience with DNS protocol in this blog post: <a href="https://pavel.network/please-stop-using-dns-protocol-for-your-application/" rel="nofollow noreferrer">https://pavel.network/please-stop-using-dns-protocol-for-you...</a>
The question I have is:<p>Why isn't there a combined MTA/DKIM/DNS server ?<p>Why am I installing and configuring and running and monitoring all of these different server software packages ?<p>I'd like to have a single config file that handles everything required to run a mailserver.<p>If I have more complex needs I can split things up make them modular ... but I don't.
DNS has been repurposed for everything from security validation to load balancing.<p>DNS over HTTPS and DNSSEC attempted to address some longstanding issues, but in the end everyone still has a host they know is going to get hammered harder.<p>Not too difficult to understand, but it is complicated given the number of sub-optimal use-cases that emerged. =)
Some years ago, I had a miserable couple of days when an internal DNS server quit working. Eventually I traced it to a corrupted cache file. Once that was deleted, life returned to normal.<p>Though unpleasant, it was an enlightening experience. I had never imagined how much of the internet ran on Amazon by then (2010?).
DNS-noob here:
For what use case is it useful to debug DNS like described in this article?
I'm a programmer since 2001 and I never ever had to investigate what is going on behind name resolution, I never encountered any kind of DNS-related issue.
Julia also created this comic about DNS, which is helpful for learning: <a href="https://wizardzines.com/zines/dns/" rel="nofollow noreferrer">https://wizardzines.com/zines/dns/</a>
Wow, her "human" output for dig is wonderful. I learned nslookup first, and to this day I still use it because dig's output is such a mess. If dig looked as she suggests, I would have switched years ago.
I find the syntax and formatting confusing. The article mentions a desired "+human" flag. How about DNS tools that output everything in JSON? I want to see the structs as key+value.
i hate to be that guy but it's not hard to learn. The tools are just from a different era where expectations where... different. However, even in my day BIND was avoided in favor of other servers like that one by the qmail guy.. can't remember his name.<p>this comment reminds me of one of my favorite Dilberts<p>old guy watching Dilbert at his computer: you kids today and your fancy graphical user interfaces. Back in my day all we had were ones and zeros ...and sometimes we didn't even have ones.<p>Dilbert: you had zeros? we had to use the letter 'o'
A fun and colorful explanation of how DNS works.
<a href="https://howdns.works/" rel="nofollow noreferrer">https://howdns.works/</a>
i had hella trouble with it, but not just it - all the related topics - reverse dns/proxy setups, 10 ways to configure it/them, cname flattening and the 15 ways to configure that, the 15 ways you can do it with cloudflare or not, dis/advantages of all those setups/combinations/permutations, etc.<p>doubt a course could teach me half of what i would need to know in some reasonable amount of time - say, two or three months.<p>but open to the idea.
DNS itself is pretty simple. Implementing a basic DNS server or client is not a big deal.<p>The management software around it is what is generally terrible.
Doing:<p>bastion named server<p>Multizone<p>Hidden Master<p>Split-Horizon<p>IXFR/AXFR firewalling<p>DNSSEC and local network<p>resolv.conf hijacking<p>Remote admin security<p>All of above would and should be hard.
behold the dns camel:<p><a href="https://datatracker.ietf.org/meeting/101/materials/slides-101-dnsop-sessa-the-dns-camel-01" rel="nofollow noreferrer">https://datatracker.ietf.org/meeting/101/materials/slides-10...</a>
TIL that `dig` does not have TLDR page <a href="https://github.com/tldr-pages/tldr">https://github.com/tldr-pages/tldr</a>
Use chatGPT to understand dig output:<p>Explain this :
dig +human +norecurse @8.8.8.8 google.com
HEADER:
opcode: QUERY
status: NOERROR
id: 11653
flags: qr ra
records: QUESTION: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1<p>QUESTION SECTION:
google.com. IN A<p>ANSWER SECTION:
google.com. 21 IN A 172.217.4.206<p>ADDITIONAL SECTION:
EDNS: version: 0, flags:; udp: 512<p>ChatGPT
This is a result from the "dig" command, a DNS (Domain Name System) tool used to query DNS servers and retrieve information about domain names. Let's break down the output:<p>dig: This is the command being executed.<p>+human: This flag is not a standard dig option. It is likely an extension that is meant to provide a more human-readable output, making it easier for people to understand the results.<p>+norecurse: This flag tells the DNS server (in this case, the server at IP address 8.8.8.8, which is a public Google DNS server) not to perform recursive lookups. Recursive lookups involve the DNS server querying other DNS servers if it doesn't have the requested information in its cache. By using +norecurse, the server will only provide information if it has it directly available.<p>@8.8.8.8: This indicates the DNS server being queried, which is the Google Public DNS server located at IP address 8.8.8.8.<p>google.com: This is the domain name being queried.<p>Now, let's look at the different sections of the response:<p>HEADER: This section provides general information about the response.
opcode: QUERY: Indicates that the query type is a standard query.
status: NOERROR: Indicates that the query was successful, and there were no errors.
id: 11653: This is a unique identifier for the query, used to match queries with their respective responses.
flags: qr ra: The flags indicate that this is a response (qr) and that recursion is available and allowed (ra).
QUESTION SECTION: This section shows the domain name and the type of record being queried.
google.com. IN A: The domain name "google.com" is being queried for its IPv4 address (A record).
ANSWER SECTION: This section provides the answer to the query.
google.com. 21 IN A 172.217.4.206: This line shows the answer. It says that the IPv4 address of "google.com" is 172.217.4.206. The "21" is the time to live (TTL) value in seconds, which indicates how long the information can be cached by other DNS servers or clients before it should be refreshed.
ADDITIONAL SECTION: This section provides additional information related to the query.
EDNS: version: 0, flags:; udp: 512: This line indicates the use of EDNS (Extension Mechanisms for DNS), which is a set of extensions to the DNS protocol. The version is 0, and the flags are empty. "udp: 512" refers to the maximum payload size that can be used for UDP communication.
In summary, the query asked for the IPv4 address (A record) of "google.com" from the Google Public DNS server at IP address 8.8.8.8. The response confirms that the IPv4 address is 172.217.4.206, and it is valid for 21 seconds.