What happens when you load a URL? (2015)

Reminds me of &quot;I, pencil&quot;:<p>&quot;A charming story which explains how something as apparently simple as a pencil is in fact the product of a very complex economic process based upon the division of labor, international trade, and comparative advantage.&quot;<p><a href="https:&#x2F;&#x2F;oll.libertyfund.org&#x2F;titles&#x2F;read-i-pencil-my-family-tree-as-told-to-leonard-e-read-dec-1958&#x2F;simple" rel="nofollow">https:&#x2F;&#x2F;oll.libertyfund.org&#x2F;titles&#x2F;read-i-pencil-my-family-t...</a>

I ask this question from time to time and look for different responses depending on the type of candidate. When I ask an engineer, I am looking to see if they can grasp the problem space and ask clarifying questions about what I&#x27;m looking for. When I&#x27;m interviewing for an engineering manager I am looking to see if they can grasp the problem space and ask clarifying questions about what I&#x27;m looking for. Since I tend to work on technical products, when I interview a product manager I tend to look for...<p>So much of the engineering mindset jumps to solutions instead of clarifying problems. I&#x27;m guilty of this constantly.

I tried writing an answer to (almost) this exact question in non-technical language[0], starting from packets and working upwards.<p>The document ended up much longer than I intended and I am still not sure my answer is that helpful.<p>[0] <a href="https:&#x2F;&#x2F;sheep.horse&#x2F;2017&#x2F;10&#x2F;how_you_are_reading_this_page.html" rel="nofollow">https:&#x2F;&#x2F;sheep.horse&#x2F;2017&#x2F;10&#x2F;how_you_are_reading_this_page.ht...</a>

Reminds me of when Richard Feynman analysed the question of “how do magnets work?”. There are so many levels to it, coupled with assumed knowledge, that the question itself is shown to be improperly bounded.

I find I run into similar discussions whenever a beginner programmer decides they want to capture input from the arrow keys.<p>They&#x27;ve worked through the bog standard Python tutorials using input() to capture from stdin, so it should be trivial to design a simple game that uses the arrow keys right? And suddenly this gulf of complexity opens up of event polling, keycodes, integrating third party modules, and they&#x27;re left wondering why moving one key over on the keyboard is so hard.

Interesting, but couldn&#x27;t you ask this of any existing technology with years of development and realize you don&#x27;t know how we arrived at a function we take for granted?<p>What happens when you press the accelerator on your car?

I was given this as an interview question once, it was fun. I was told to go into “as much breadth and depth as possible” but I did get stopped a couple times when I started talking about things like debouncing.

Another great resource similar to <i>How Browsers Work</i> [1] is Mariko&#x27;s <i>Inside look at modern browsers</i> series [2]<p>[1] <a href="https:&#x2F;&#x2F;www.html5rocks.com&#x2F;en&#x2F;tutorials&#x2F;internals&#x2F;howbrowserswork&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.html5rocks.com&#x2F;en&#x2F;tutorials&#x2F;internals&#x2F;howbrowser...</a><p>[2] <a href="https:&#x2F;&#x2F;developers.google.com&#x2F;web&#x2F;updates&#x2F;2018&#x2F;09&#x2F;inside-browser-part1" rel="nofollow">https:&#x2F;&#x2F;developers.google.com&#x2F;web&#x2F;updates&#x2F;2018&#x2F;09&#x2F;inside-bro...</a><p>Disclosure: Mariko and I are on the same team; I didn&#x27;t work on that series

Interesting that he skipped the part about parsing the URL and generating the HTTP (httpMethod, path, headers), assuming that is the protocol. It seems relatively simple but unless one is careful there are some gotchas. Perhaps that is why this particular part of the process ends up in so many &quot;web security&quot; write-ups; many people, including the author, just do not think it is significant. He also asssumes that the client is a &quot;browser&quot;.

I was once curious about this sort of thing. I hate that I didn&#x27;t know how a computer worked from the ground up, it made them feel like magic when I knew they were just clever machines. I wasn&#x27;t satisfied with learning about circuits so I went even lower and asked how transistors work. I spent a long time in a deep rabbit hole learning chemistry so I could understand why capacitors keep charge and etc. That&#x27;s when I realise to learn every layer of a computer to a level of expert proficiency is a lifetime&#x27;s work, not something you can expect one person to be able to do off the top of their head.<p>This is why we work together, we are not swiss army knives.

I appreciate author&#x27;s quest for bottom-up understanding of all the things we depend on daily. The quote &quot;We are standing on shoulders of giants&quot; really shines here, and we all do a great service to them by trying to understand.

A nice why-and-why journey. Yet as an interviewing approach this seems to lack an initial premise, the need.<p>For example.<p>* I want to know what does a URL represent. Is it a porn site? Am I gonna be hacked&#x2F;tracked?<p>* I need to check if this domain name is not&#x2F;taken.<p>* I want to know if URL can contain spaces, or emojis.<p>* I want to know if I get a reasonably fast response.<p>* My phone tells me the site access is Forbidden.<p>Basically, a reasonable answer needs to be also a &quot;Why?&quot;. This sets up a logical context for drilling down with an intention to locate <i>something</i>, not just for the drilling down sake.<p><i>&quot;Computer, what happens...?&quot;</i><p><i>&quot;Why do you need to load a URL, Dave?&quot;</i>

Your browser tries to guess if what you have typed is is a url or a search query. If you’re using chrome then there is a chance that your request will hit a root domain name server so that the browser can check if you are connected to a scammy Internet service provider or some other ad serving intermediary. <a href="https:&#x2F;&#x2F;arstechnica.com&#x2F;gadgets&#x2F;2020&#x2F;08&#x2F;a-chrome-feature-is-creating-enormous-load-on-global-root-dns-servers&#x2F;" rel="nofollow">https:&#x2F;&#x2F;arstechnica.com&#x2F;gadgets&#x2F;2020&#x2F;08&#x2F;a-chrome-feature-is-...</a>

Asked of Cliff Stoll for his PhD defense: &quot;Why is the sky blue?&quot;

&gt; How does the browser know to try to load a webpage? I guess it sees an &quot;<a href="http:&#x2F;&#x2F;&quot;" rel="nofollow">http:&#x2F;&#x2F;&quot;</a> or just assumes that anything with no prefix is a URL?<p>Question #7 seems related to that Chromium DNS lookup algorithm [1] that has been getting attention recently<p>[1] <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=24231857" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=24231857</a>

There seems to be a need for resources about browser internals. This is something we could potentially pursue on web.dev. However, it&#x27;s a big field. Can you all help me by:<p>* Providing examples of existing resources that explain browser internals which has made you a better developer<p>* Listing situations in which you needed to go deeper into how browser internals work and didn&#x27;t find good resources

This used to be a standard interview question I&#x27;d ask potential junior candidates. I&#x27;d see how far down I can get them to go.

Another interesting thought exercise would be &quot;can you accomplish X with as few layers&#x2F;infrastructure&#x2F;historical context&#x2F;complexity as possible?&quot;<p>Knowing what you need to accomplish, can you design a system from the ground up that sheds some of the traditionally expected layers and idioms and see what you can come up with.

I really like this question. What I get out of it is where the candidate focuses on, and where they gloss over. I find it useful to prefix the question with a statement about there being no &quot;right&quot; answer, and &quot;go in as much detail as you want or know&quot;.<p>Since I&#x27;m not hiring for hardware, when they get into minute details about keyboard and USB, I smile and gently interrupt to move to the browser making the request. Come to think of it, I may rephrase the question to &quot;What happens when you click on a link?&quot; since that covers browser events, CORS and all that fun; and does away with the keyboard.

A bit dated as well. It&#x27;s assuming I&#x27;m using a physical keyboard. Today I might be using my phone. Might be using voice recognition. It might be my fridge hitting a URL using IoT to order me some more beer.

I think it lacks of:<p>Parsing and js execution<p>TLS Handshake &#x2F; algo negotation<p>Cache

This is why I love this question as a phone-screener for onsite interviews and stuff.<p>Sounds so easy, but there is so much depth in so many areas that people can drill into. If someone starts talking convincingly about the OS handling keyboard interrupts or about HTTPS certificate revocation during the TLS handshake explanation etc, you know they are probably worth bringing in for a face to face.<p>Someone bluffing will usually fail this really hard - &quot;The browser connects to example.com and downloads the page.&quot; or something. Uh-huh. <i>Maybe</i> they talk about DNS ... <i>Maybe</i> they talk about a TCP connection ... <i>Maybe</i> they talk about retrieving the HTML and parsing it ... but even then, if they get that far that will be better than a lot of bluffers. If they have simply revised anticipating this question and can answer <i>everything</i> then congrats - they&#x27;ve just educated themselves in a lot of the core fundamentals :)<p>As an interviewer this question can easily take up a whole 30 min phone screener as good candidates get deeper and deeper.