Reasons to use protocol buffers instead of JSON

&gt; Schemas Are Awesome<p>No reason you can&#x27;t implement schemas over JSON. In fact, you typically implicitly do - what your code is expecting to be present in the data structures deserialized from JSON.<p>&gt; Backward Compatibility For Free<p>JSON is unversioned, so you can add and remove fields as you wish.<p>&gt; Less Boilerplate Code<p>How much boilerplate is there in parsing JSON? I know in Python, it&#x27;s:<p><pre><code> structure = json.loads(json_string) </code></pre> Now then, if you want to implement all kinds of type checking and field checking on the front end, you&#x27;re always welcome to, but allowing &quot;get attribute&quot; exceptions to bubble up and signal a bad data structure have always appealed to me more. I&#x27;m writing in Python&#x2F;Ruby&#x2F;Javascript to avoid rigid datastructures and boilerplate in the first place most times.<p>[EDIT] And for languages where type safety is in place, the JSON libraries frequently allow you to pre-define the data structure which the JSON will attempt to parse into, giving type safety &amp; a well define schema for very little additional overhead as well.<p>&gt; Validations and Extensibility<p>Same as previous comment about type checking, etc.<p>&gt; Easy Language Interoperability<p>Even easier: JSON!<p>And you don&#x27;t have to learn yet another DSL, and compile those down into lots of boilerplate!<p>I&#x27;m not trying to say that you shouldn&#x27;t use Protocol Buffers if its a good fit for your software, but this list is a bit anemic on real reasons to use them, particularly for dynamically typed languages.

Or, another alternative is Cap&#x27;n Proto [1] from the primary author of Protocol Buffers v2. It smooths some of the bumps of protocol buffers.<p>[1]: <a href="http://kentonv.github.io/capnproto/" rel="nofollow">http:&#x2F;&#x2F;kentonv.github.io&#x2F;capnproto&#x2F;</a>

Easy language interoperability as a reason to choose Protobuf over JSON ? Mainstream languages support both JSON and Protobuf equally well, and the others tend to support JSON more often than Protobuf.<p>Free backwards compatibility ? No. Numbered fields are a good thing, but they only help in the narrow situation where your &quot;breaking change&quot; consists in adding a new, optional piece of data (a situation that JSON handles as well). New required fields ? New representation of old data ? You&#x27;ll need to write code to handle these cases anyway.<p>As for the other points, they are a matter of libraries (things that the Protobuf gems support and the JSON gems don&#x27;t) instead of protocol --- the OCaml-JSON parser I use certainly has benefits #1 (schemas), #3 (less boilerplate) and #4 (validation) from the article.<p>There is, of course, the matter of bandwidth. I personally believe there are few cases where it is worth sacrificing human-readability over, especially for HTTP-based APIs, and especially for those that are accessed from a browser.<p>I would recommend gzipped msgpack as an alternative to JSON if reducing the memory footprint is what you want: encoding JSON as msgpack is trivial by design.

Reasons not to use protocol buffers (in C++ at least):<p><pre><code> 1) Doesn&#x27;t support Visual Studio 2013. 2) Doesn&#x27;t support Mac OS X Mavericks. 3) No &quot;nice&quot; support C++11 (i.e. move constructors) </code></pre> (These can be at least partly solved by running off svn head, but that doesn&#x27;t seem like a good idea for a product one wants to be stable)<p>With JSON I can be sure there will be many libraries which will work on whatever system I use.

We at Spotify use them extensively and are actually moving away from Protobufs, which we consider as &#x27;legacy&#x27;. The advantages of Protobufs don&#x27;t make up for its disadvantages over plain JSON. With JSON you have universal support, simple parsing, developer and debug friendly, much easier to mock, etc etc.

I think the main advantages are:<p><pre><code> - network bandwidth&#x2F;latency: smaller RPC consume less space, are received and responded to faster. - memory usage: less data is read and processed while encoding or decoding protobuf. - time: haven&#x27;t actually benchmarked this one, but I assume CPU time spent decoding&#x2F;encoding will be smaller since you don&#x27;t need to go from ASCII to binary. </code></pre> Which means, all performance improvements. They come, as usual, at the cost of simplicity and ease of debugging.

&gt; <i>When Is JSON A Better Fit?</i> &gt; <i>Data from the service is directly consumed by a web browser</i><p>This seems to me like a key issue, you need to really know beforehand that this won&#x27;t ever be the case, else you need to make your application polyglot afterwards. A risky bet for any business data service.<p>Maybe if it&#x27;s strictly infrastructure glue type internal service. But even then, maybe someone will come along wanting to monitor this thing on the browser.

Not so sure about &quot;backwards compatibility&quot; part.<p>From Protocol buffer python doc: <a href="https://developers.google.com/protocol-buffers/docs/pythontutorial" rel="nofollow">https:&#x2F;&#x2F;developers.google.com&#x2F;protocol-buffers&#x2F;docs&#x2F;pythontu...</a><p>&quot;Required Is Forever You should be very careful about marking fields as required. If at some point you wish to stop writing or sending a required field, it will be problematic to change the field to an optional field – old readers will consider messages without this field to be incomplete and may reject or drop them unintentionally. You should consider writing application-specific custom validation routines for your buffers instead. Some engineers at Google have come to the conclusion that using required does more harm than good; they prefer to use only optional and repeated. However, this view is not universal.&quot;<p>So basically I will be in trouble if I decide to get rid of some fields which are not necessary, but somehow were defined as &quot;required&quot; in the past.<p>This will potentially result in bloated protobuf definitions that have a bunch of legacy fields.<p>I will stick to the JSON, thanks.

This is a fine reason to use protocol buffers instead of JSON:<p><pre><code> [ 4738723874747487387273747838727347383827238734.00 ] </code></pre> Parsing that universally is a shit.

The biggest thing is that its smaller over the wire.<p>however if schemas scare you (shame on you if they do) then msgpack might be a better choice.

Definitely the right direction for performance. My company ended up going with python-gevent and zeromq to implement an asynchronous API server with persistent TCP connections. Our application servers are able make remote calls over a persistent tcp connection without any noticeable overhead. You could still use JSON, and we tried it--but since we&#x27;re all python anyway we decided to just pickle the objects which is way faster. We looked at protocol buffers, but found it to be a bit cumbersome. It&#x27;s been stable for two years and completely solved our scaling problems.

&gt; There do remain times when JSON is a better fit than something like Protocol Buffers, including situations where:<p>&gt; * You need or want data to be human readable<p>When things &quot;don&#x27;t work&quot; don&#x27;t you always want this feature? Over a long lifetime, this could really reduce your debugging costs. Perhaps protocol buffers has a &quot;human readable mode&quot;. If not, it seems like a risk to use it.

...in Ruby. For some applications.<p>With Node, I&#x27;d have to see a very good argument for why I should give up all of JSON&#x27;s great features for the vast majority of services. Unless the data itself needs to be binary, I see no reason why I shouldn&#x27;t use the easy, standard, well-supported, nice-to-work-with JSON.

Anyone tried ProtoBuf.js &quot;Protocol Buffers for JavaScript.&quot; on the client side?<p><a href="https://github.com/dcodeIO/ProtoBuf.js" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;dcodeIO&#x2F;ProtoBuf.js</a>

What a huge step backwards. We had decades of binary protocols. They sucked. Then everyone moved to non-binary protocols and it was much better. Let&#x27;s not do it all over again.

I&#x27;d rather use Avro. The binary encoding is more dense and there&#x27;s an officially supported JSON encoding (the text encoding for Protobufs is mostly intended for debugging)

When you convert an object from language X to JSON, validate it using a schema validation before deserializing, then is it not the same as JSON. Also now with JSON you have the opportunity to have human readable data which is great when debugging issues. I am not seeing the advantage of protocol buffers. It would be great if you can compare payload sizes and see if there is a significant savings from that perspective.

Personally, I prefer Thrift over protocol buffers. I&#x27;m surprised Thrift wasn&#x27;t mentioned.

&quot;Reason #3: Less Boilerplate Code&quot;<p>Anyone who has used Avro in Java knows that this is not true.

Use protocol buffers because you get versioning? How hard would it be to add a version number to your JSON struct?<p>(Answer: Trivial.)