My thoughts on MessagePack

cheald has an excellent comment regarding MessagePack, JSON, and Protocol Buffers in the post from 15 hours ago: <a href="http://news.ycombinator.com/item?id=4091051" rel="nofollow">http://news.ycombinator.com/item?id=4091051</a><p><pre><code> MessagePack is not smaller than gzip'd JSON MessagePack is not faster than JSON when a browser is involved </code></pre> Other comments from that post include:<p><pre><code> MessagePack is not human readable MessagePack has issues with UTF-8 Small packets also have significant TCP/IP overhead </code></pre> Really, anyone who hasn't read the other comments should: <a href="http://news.ycombinator.com/item?id=4090831" rel="nofollow">http://news.ycombinator.com/item?id=4090831</a>

Zerorpc (<a href="http://github.com/dotcloud/zerorpc-python" rel="nofollow">http://github.com/dotcloud/zerorpc-python</a>) uses msgpack as its primary serialization format. Among other things it is significantly more efficient for floating point data. At dotCloud we use it to move many millions of system metrics per day, so it adds up. Also worth noting: msgpack maps perfectly to json, so there's nothing keeping you from translating it to json on its way to the browser, where json is indeed a better fit. In practice this doesn't affect performance since you typically need to parse and alter the message at the browser boundary anyway, for some sort of access control.

We're using MessagePack in a Rails application to log user behaviors and analyze them.<p>Compared with other serialization libraries such as Protocol Buffers, Avro or BSON, one of the advantages of MessagePack is compatibility with JSON. (In spite of its name, BSON has special types which cause incompatibility with JSON)<p>It means we can exchange objects sent from browsers (in JSON format) between servers written in different languages without losing information.<p>I will not use MessagePack with browsers but it's still useful to use it with web applications.

As much as we would like to jump back on the MessagePack is no JSON alternative here, I would like to commend The author on taking the criticism posted earlier like a mature adult and explaining his point of view. Even admitting that some of the benchmarks might have been misleading.

I've personally been using BSON[1] as a binary alternative to JSON, and it's worked out great. I've written an Objective-C wrapper around the C lib, in case anybody's interested. Every other language has a solid implementation from 10gen (the MongoDB guys). It's a solid format with a clear spec that's extensible and fully supports UTF-8.<p>[1]: <a href="http://bsonspec.org/" rel="nofollow">http://bsonspec.org/</a>

I think it's hilarious that 2 of the 3 comments on the gist are about how he formatted his Markdown. <i>sigh</i><p>I think MessagePack is great. Good for him for trying to make something better. He openly admits it's not better all the time. Why not help?

Very interesting discussion. I work on a 2D MMORPG for Android. This is extremely relevant to me. I have a few questions though.<p>What if you take compression and deserialization out of the picture? For example, in my server I have a hash like data structure that gets turned into JSON for browsers and byte array for mobile clients.<p>For example, because the data has to be transferred at fast rates and will be going over mobile networks. The size of the packet matters because every millisecond counts.<p>Then to read the data, I simply read the stream of bytes and build the objects I need on the client. This has to happen mostly without allocations for example on Android to avoid the GC.<p>So a few questions: Does deserializing JSON cause any memory allocations? If you're not tokenizing the data and don't need to parse it, will it be a significant gain over s serialized byte protocol or JSON?<p>In any case, I'll experiment on my end and perhaps blog about my own findings.

The best way of serialization at present. I like the thought, performance and various implementations.