The 7-bit Internet

My personal &quot;wow, it&#x27;s just this?&quot; moment came when, in high school, I figured out how to IRC from a Unix box that didn&#x27;t have an IRC client installed, just by telnetting to the IRC server. The only particularly annoying part was that you&#x27;d get disconnected if you were idling and didn&#x27;t respond to a PING with a PONG quickly enough.<p>It was also a fun way to learn programming. mIRC at some point added a sockets interface to mIRC script, and after that it was pretty easy to write toy clients for various text-based protocols, doing simple things like checking whether a URL was a 404. I can&#x27;t say I did anything particularly <i>useful</i> with that, but it was a nice way to learn something about programming.

For some reason, I feel as though the people in this thread against text-based protocols have never used telnet to inspect or debug an HTTP request&#x2F;response. Or used telnet to help develop a client for something like memcached.<p>The ease of discoverability of a protocol should not be underrated.

I don&#x27;t understand why SPDY &#x2F; HTTP 2.0 is being framed as &quot;fitting the needs of a few major players&quot;.<p>Google is not the only one who benefits from reduced latency, fast page loads, efficient use of SSL sessions, and server pushed resources.<p>If anything, the people who benefit the most are those who _can&#x27;t_ afford massive forward deployed CDN networks, large servers, and fat network pipes.

If you&#x27;re tempted to argue against this by pointing out that very simple tools can make irrelevant the change to a non-textual protocol -- then you are, I suspect, missing the point. There&#x27;s a real psychological difference between debugging something that is, at bottom, human-readable versus one that isn&#x27;t. I&#x27;m willing to concede that it isn&#x27;t really that big of a difference. For the sake of argument I&#x27;m even willing to concede that it&#x27;s largely illusory -- because that doesn&#x27;t matter. Who cares how irrational it may be? If it&#x27;s that prevalent in the human psyche, you&#x27;re just better off not fighting it.<p>Like scripting languages, a text-based protocol doesn&#x27;t just make it easier for you to get your hands dirty: it practically begs you to. The value of that sort of encouragement to the adoption of a global standard should not be underestimated.

&gt;At first, this struck me as a very odd thing; text is inefficient, and machines, not humans, are meant to interpret protocols. A binary setup would save bytes—bytes!— and be all-around more manageable by software.<p>&gt;It wasn’t long, however, before I realized the true genius behind this decision.<p>This reminded me very much of a talk by Jonathan Blow where he&#x27;s talking about how he &quot;nerd raged&quot; while reading the <i>Doom</i> source code. He found a section that wasn&#x27;t optimized, but in fact not optimizing it had so many other benefits (besides, the performance gain would have been negligible). Starts at 14 minutes: <a href="http://the-witness.net/news/2011/06/how-to-program-independent-games/" rel="nofollow">http:&#x2F;&#x2F;the-witness.net&#x2F;news&#x2F;2011&#x2F;06&#x2F;how-to-program-independe...</a>

I don&#x27;t get all the for-telnet arguments. Who uses telnet for this anyway? I&#x27;d bet that 90% of telnet-folk do:<p><pre><code> cake:~ mali$ telnet google.com 80 Trying 74.125.235.8... Connected to google.com. Escape character is &#x27;^]&#x27;. GET &#x2F; HTTP&#x2F;1.1 ....... HTTP&#x2F;1.1 200 OK Date: Wed, 10 Jul 2013 22:25:02 GMT Expires: -1 Cache-Control: private, max-age=0 Content-Type: text&#x2F;html; charset=ISO-8859-1 </code></pre> At which point they go &quot;oh cool&quot; and go do something else. For the rest of us who use curl, wget, requests, Chrome &#x2F; Firefox developer tools every day (who are we kidding, it&#x27;s everyone, you liars! :P), the binary transformation would be transparent.<p>Hell, if you&#x27;re going for pure cool-factor, how is pulling out your hex editor less cool? But in reality, you&#x27;d never do this.<p>For a non-standardized and obscure protocol where tooling would likely be lacking, I can see why human readability is a good idea. But we&#x27;re talking about the very protocol that makes up the fabric of the internet. Seriously, why?<p>Give me one good reason.

Oh sure, let&#x27;s keep buffer bugs, waste tons of bandwidth and storage, burn millions of CPU hours in parsing computer-generated text, completely ban encryption and screw it for all non-en_US-speakers so that it could be debugged via misuse of a protocol from 1983 that no one sane uses.

This text vs. binary discussion is pretty high level. HTTP&#x2F;2.0 has a mux layer (let&#x27;s temporarily put aside whether that&#x27;s a good idea) and some of the payloads have to be binary (e.g. JPEGs). There are basically three ways you can do this:<p>A. base64 everything. Obviously this has high overhead.<p>B. Escaping (aka byte stuffing). This is somewhat slow to escape and unescape, the overhead is variable (in rare cases 100%), and it&#x27;s fragile to read or write by hand.<p>C. Byte counting. This is the most efficient, but extremely inconvenient to write by hand.<p>You could create an efficient &quot;text&quot; mux protocol (basically BEEP), but the result is so non-human-readable&#x2F;writable IMO that I don&#x27;t think people would be any happier. HTTP&#x2F;2.0 is not complex because it&#x27;s binary; it&#x27;s binary because it&#x27;s necessarily already complex enough that text doesn&#x27;t save you anything.

I don&#x27;t think a <i>simple</i> binary protocol for HTTP would be bad at all. HTTP servers are still going to support text-based connections forever, so it only matters if you&#x27;re snooping on existing traffic; but the ability to use Wireshark or tcpdump to grab the text out of packets directly is already disappearing because of encryption (SSL), which is such a good thing in general that it far outweighs that drawback. Therefore more specialized tools are required, and those may as well parse such a protocol - it wouldn&#x27;t be that hard. The compression is a significant benefit, and can&#x27;t be done well below the level of HTTP.<p>What I don&#x27;t understand is why we need both SPDY and QUIC.<p>SPDY provides fast multiplexing, compression, and guaranteed SSL. QUIC provides fast multiplexing and guaranteed SSL at the transport(-ish) layer. QUIC does at least one thing that can&#x27;t be done at the application layer - faster opens - and seems nice to have as a generic base for multiple protocols, so I call it a good thing (and I&#x27;d like to see it in the kernel). But the plan seems to be to run SPDY over QUIC - I can&#x27;t actually find enough information on the Internet (and am too lazy to look through the source) to find out whether packets are going to be double encrypted for the time being, but even if&#x2F;when that is avoided, the multiplexing and optional encryption seem to be wasteful complexity. I would prefer if HTTP&#x2F;2.0 were a simple and easy-to-parse protocol providing compression only, expected to be used over QUIC.

I agree with most of the article, but would strongly argue against any use of the &#x27;robustness principle&#x27;. There have been huge security vulnerabilities because systems disagree with each other about what is a valid request and response.

Am I the only one who thinks its ironic that the title references 7-bits? This shows that you&#x27;re already binary encoded enough that your terminal has to parse it out. Why not just have a dumber terminal?<p>Plus, it&#x27;s SSL based. SSL means you&#x27;re already used to not looking at things through telnet. Are you really saying we shouldn&#x27;t use SSL because &quot;it&#x27;s not just 7-bit plaintext&quot;? That&#x27;s the point, after all...

I work in telecom, and this makes me think of two protocols I work a lot with, SS7 and SIP.<p>SIP is like HTTP, in fact I believe it was modelled after HTTP and has a lot in common with it. You can troubleshoot SIP issues very easily using a packet inspector like ngrep (using Telnet might be a bit difficult as there are some timers that expire if you don&#x27;t respond fast enough, but that&#x27;s besides the point).<p>Then there&#x27;s a protocol like SS7 which is binary based. It&#x27;s all structure binary bit fields. Even though it&#x27;s binary based, and not very human readable, we have tools that decode the bits into a human readable format, which in turn makes it just as easy as SIP to troubleshoot.<p>The question I ask myself is, which one would be easier to implement, text based or binary format? I guess I&#x27;d lean towards text based--in fact I have implemented a SIP client myself with success. But have yet to implement any kind of SS7 or binary based protocols. Perhaps experience has something to do with it, would having experience implementing binary protocols make it any bit easier? And to be fair, SIP is also much well more documented then SS7. As well it&#x27;s easier to test SIP.<p>I&#x27;m just starting to work with binary protocols, so it will be interesting how much progress I make. I realize there are also some libraries that help with working with binary&#x2F;bit field data.

This sounds similar to an argument you could make for writing all of your code in assembly instead of C (or other higher-level languages.) Yes, there&#x27;s a layer of obscurity between a program written in C and the resulting assembly: a tool (yes, a tool, spouting abstraction all over the place!) called a compiler. As a core tenet of engineering, we create tools that often leverage abstraction in order to let us tackle more complex problems without wrapping our heads around (or writing code to handle) the details.<p>It&#x27;s a matter of balance, and we can all choose our own camps along the abstraction &lt;=&gt; ease-of-understanding-every-detail continuum (I&#x27;d tend to plop myself toward the former.) Just as we can create a reliable C =&gt; assembly compiler, we can create reliable tools to let us leverage abstraction for the better and focus on the bigger picture rather than the smaller.

The original idea behind the web was a publishing platform that, like a classic car, would be easy to explore, maintain and develop.<p>The new vision is the application. Whether it&#x27;s on a phone, or running as a web site, these are not publishing so much as services.<p>Perhaps it&#x27;s time to fork protocols: one for publishing, one for interactive applications.

While I agree that it sucks to lose text based protocols, we already have an underlying protocol that is completely binary in nature and requires tools to look at it (TCP). Same can be said for IP. We&#x27;ve survived a long time with those technologies and we have some awesome tools for working with them. While it&#x27;s not as easy as ascii based protocols, it&#x27;s more space efficient, which I&#x27;m fine with.

&gt; You should follow me on Twitter.<p>This meme needs to die. It&#x27;s like ending every post with &quot;Just kidding, I don&#x27;t know what I&#x27;m talking about, I&#x27;m just cargo-culting blog attention-desperate person trying to build my Klout score to impress myself.&quot;

I don&#x27;t get the fuss. HTTP 2.0 is not a deprecation of HTTP 1.1&#x2F;1.

&quot;I’ve been in this business for two decades, and the only way I know to hand-debug a SOAP transaction is with a hammer and a straitjacket2. Heck, I’ll take an old-school EDI transaction over SOAP any day.&quot;<p>SOAP? You lucky. It&#x27;s 2013 and I&#x27;m still knee deep in RMI-IIOP sometimes. Wireshark with GIOP dissector helps, but I’ll take SOAP over CORBA any day:-).<p>Seriously: Your post summarized very well what I was thinking when reading the recent HTTP 2.0 posts. Full Ack.

Man, not having \n as a statement separator and having to pipe everything through xxd will definitely be a pain.

I think TLS&#x2F;SSL is the elephant in the room here. A large part of the web is effectively already using binary protocol (HTTPS), and frankly I&#x27;d wish the part was even bigger.<p>Think of HTTP 2.0 as a successor for HTTPS instead of HTTP if it makes you feel better.

Binary vs. text is one dimension and it shouldn&#x27;t be conflated with the overall alleged complexity of the HTTP&#x2F;2.0 proposal. You can have robust and non-robust binary protocols.<p>I don&#x27;t buy the &quot;argument from visibility&quot;, for example.<p>HTTPS and DNS are both binary protocols and you need tools to parse them in order to get telnet-like visibility. Does that mean when it comes to those protocols &quot;we are forced to rely on other tools to hide the underlying complexity and dumb things down to a point where they are manageable&quot;, as the author says?

I love these discussions about text vs. binary protocols. I especially love the part where people defend binary protocols because they are more &quot;efficient&quot; and don&#x27;t eat as much bandwith or processing resources.<p>May I remind these nice people that most text-based protocols we rely on today were designed in a time when CPUs were orders of magnitude slower and modems ran at 2400 baud? Talk about processing and bandwidth resources...<p>The resources consumed by having to interpret a text-based protocol pale in comparison with the resources consumed by all the levels of abstraction that modern frameworks&#x2F;language runtimes that we use today. But wait, these save developer time, so they are a good thing. So does having protocols that can be troubleshooted by a human with minimal tools.

It&#x27;s 2013, not 1973, so I&#x27;d replace 7-bit with UTF-8 which is its natural extension.

You may find this thing that we&#x27;re working on relevant:<p><a href="https://github.com/nudgepad/space" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;nudgepad&#x2F;space</a><p>It&#x27;s a very understandable and very powerful language that I think will at some point be extended to replace HTTP, amongst other things.

Amen, OP.<p>herge: You could also use the &quot;data&quot; program from Hobbit&#x27;s netcat.

Marco claims that HTTP 2.0 is overly complex, but never gives any reasons why. He claims that text protocols are better than binary, but never really gives any reason for that either. Nostalgia for the days of telnet and gopher is not a reason.<p>Then he goes off to talk about SOAP, which he hates, because &quot;the only way [he] know[s] to hand-debug a SOAP transaction is with a hammer and a straitjacket.&quot; But SOAP is text based, which directly contradicts his argument from earlier.<p>The other allegation here is that the new protocol &quot;satisf[ies] what are essentially the needs of a small group of very influential players.&quot; Again, Marco just states this with no rationale at all. In fact, the opposite is true. By being more efficient, HTTP 2.0 will serve the needs of everyone. Should startups have to pay for 10 servers when they could have paid for 5 with HTTP 2.0? Apparently Marco thinks they should. It&#x27;s easier for Google to pay for a few more servers than it is for Joe startup. This should be obvious to everyone, but apparently it&#x27;s not.<p>The whole binary versus text thing is a complete red herring. If you have ever used wireshark, you know that it makes binary fields easy to read. Binary protocols are easier to develop and standardize. Pretty much every good developer realizes this, and the responses here reflect that. The idea that making something LOOK like (but not actually be) &quot;human text&quot; makes anything better is an idea we should have buried along with COBOL and other mistakes from the past. But I guess not.

The site crashes my phone browser.<p><pre><code> Dolphin Mini. HTC Wildfire. Android 2.2.2 </code></pre> I guess I&#x27;ll check it out later.