TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Begrudgingly Choosing CBOR over MessagePack

55 pointsby jjgreen2 months ago

10 comments

magicalhippo2 months ago
The article links to this[1] older HN comment which argues against CBOR. I must admit this passage made me laugh out loud:<p><pre><code> A decoder that comes across a simple value (Section 2.3) that it does not recognize, such as a value that was added to the IANA registry after the decoder was deployed or a value that the decoder chose not to implement, might issue a warning, might stop processing altogether, might handle the error by making the unknown value available to the application as such (as is expected of generic decoders), or take some other type of action. </code></pre> <i>I choose not to implement `int`. I decide instead to fill up your home folder. I&#x27;m a compliant CBOR implementation.</i><p>Seems this part of the specification has been rewritten[2], so now a generic decoder is to pass on the tag and value to the user or return an error.<p>[1]: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=14072598">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=14072598</a><p>[2]: <a href="https:&#x2F;&#x2F;www.rfc-editor.org&#x2F;rfc&#x2F;rfc8949.html#name-generic-encoders-and-decode" rel="nofollow">https:&#x2F;&#x2F;www.rfc-editor.org&#x2F;rfc&#x2F;rfc8949.html#name-generic-enc...</a>
评论 #43239902 未加载
maciejw2 months ago
CBOR started as a complimentary project to previous-decade IoT (Internet of Things) and WSN (Wireless Sensor Networks) initiaties. It was designed together with 6LoWPAN, CoAP, RPL and other standards. Main improvement over message pack was discriminating between byte strings and text strings - an important usecase for firmware updates etc. Reasoning is probably available in IETF mailing archive somewhere.<p>All these standards were designed as a research and seem rather slow to gain general popularity (6LoWPAN is used by Thread, but its uptake is also quite slow - e.g. Nanoleaf announced dropping support for it).<p>I would say if CBOR fits your purpose it&#x27;s a good pick, and you shouldn&#x27;t be worried by it being &quot;not cool&quot;. Design by committee is how IETF works, and I wouldn&#x27;t call it a weakness, although in DOGE times it might sound bloated and outdated.
评论 #43239222 未加载
评论 #43239349 未加载
pwdisswordfishz2 months ago
&gt; Obviously MessagePack is what cool kids would use.<p>Why is that even a consideration?<p>&gt; To measure complexity, you can often use documentation length as a proxy. MessagePack is just a markdown file. The CBOR spec has its own gravitational field.<p>That&#x27;s a proxy for underspecification, not complexity.
评论 #43239602 未加载
评论 #43240545 未加载
randomtoast2 months ago
&gt; Everything about CBOR is uncool. It was designed by a committee. It reeks of RFCs. Acronyms are lame. Saying &quot;SEE-BORE&quot; is like licking a nickel. One of the authors is &quot;Carsten Bormann&quot;, which makes the name feel masturbatory.<p>Carsten Bormann was my professor for Rechnernetze (computer networks). He is also one of the authors of GNU Screen. I mentioned to him that I use tmux, and he asked what was wrong with screen :). His wife is also in the same IT department at the university, where she was the dean. She helped me sort out a problem regarding my course selections, a very kind person. I think he is a decent teacher and knowledgeable in his field, but if you look at his work over the past decades, it&#x27;s evident that he has a tendency to author RFCs that are rarely used.
评论 #43247513 未加载
bythreads2 months ago
Just a question where you trying to optimize for speed or size?:<p>I never tried CBOR when looking for a sub 10ms solution for websocket comms, however my use case was not bound by datasize but entirely by speed (network not inet).<p>However it all came down to a suprising realisation: &quot;compression on both ends is the primary performance culprit&quot;<p>Optimizing the hell out of the protocol over websockets got me to a fairly ok response time, just using string json and 0 compression blew it out of the water.<p>So the result was that data load was faster and easier the debug going with strings of json vs any other optimization (messagesize where in the 10-50mb realm)<p>The amount of shotty ws sever implementations and gzip operations in the communications pipeline is mindblowing - would be interested in hearing how just pure json and zero compression&#x2F;binary transforms performed :)
评论 #43239200 未加载
评论 #43240675 未加载
评论 #43239206 未加载
surfingdino2 months ago
I wasn&#x27;t aware of CBOR as I have yet to come across a project that needed an alternative to MessagePack or even MassagePack (only considered using it once in the past). However, based on my experience on various projects, if you have to get approvals and buy-ins from architects, legal, and security teams, using something that has an RFC helps you win those battles regardless of the technical merits of a RFC or non-RFC backed project&#x2F;tool&#x2F;protocol.
eidorb2 months ago
Yibico&#x27;s python-fido2 library (<a href="https:&#x2F;&#x2F;github.com&#x2F;Yubico&#x2F;python-fido2">https:&#x2F;&#x2F;github.com&#x2F;Yubico&#x2F;python-fido2</a>) contains (a minimal) CBOR implementation too: <a href="https:&#x2F;&#x2F;developers.yubico.com&#x2F;python-fido2&#x2F;API_Documentation&#x2F;autoapi&#x2F;fido2&#x2F;cbor&#x2F;index.html" rel="nofollow">https:&#x2F;&#x2F;developers.yubico.com&#x2F;python-fido2&#x2F;API_Documentation...</a><p>I found it wouldn&#x27;t encode `None`s, but didn&#x27;t dig at all, just worked around it.<p>Star count would place it about midway in the list.
aeontech2 months ago
Just curious if you considered Cap&#x27;n Proto as another option, or if it wasn&#x27;t in the running?<p>[1] <a href="https:&#x2F;&#x2F;capnproto.org&#x2F;" rel="nofollow">https:&#x2F;&#x2F;capnproto.org&#x2F;</a>
评论 #43239383 未加载
IshKebab2 months ago
I think the list of stars is probably not a good representation of popularity. Serde and JSON For Modern C++ have vastly more stars than any of those libraries and they both support CBOR and MessagePack.<p>I think CBOR is pretty decent though it is fairly inexplicable that a format designed in 2013 uses big endian.
评论 #43239814 未加载
评论 #43240356 未加载
dathinab2 months ago
TL;DR: CBOR is a bit more complex, but mainly due to additional features (tags, infinite&#x2F;unknown length types) which if you need them will make using CBOR the simpler choice and which libraries can decide to support only in a &quot;naive&quot; but simple way (ignore most tags, directly collect unknown length types with some hard size limits).<p>---<p>On interesting aspect of CBOR vs. MessagePack is simplicity.<p>But it&#x27;s not really that much of a win for MessagePack, especially if we ignore &quot;tags&quot;.<p>Like sure message packs splitting their type&#x2F;length markers at clean nibble boundaries and as such if you need to read a message pack file through a hex editor its easier.<p>but why would you do so without additional tooling????<p>Like, if we are realistic most developer won&#x27;t even read (non trivial small) JSON without tooling which displays the data with nice formatting and syntax highlighting. Weather it&#x27;s in your browser, editor or by piping it through jq on the command line tooling is often just a click way.<p>And if you anyway use tooling it doesn&#x27;t matter weather type&#x2F;size hints use 4&#x2F;4-bit split are a 3&#x2F;5-bit split. Implementation wise outside of maybe some crazy SIMD tricks or similar it also doesn&#x27;t matter (too) much.<p>And the moment 4&#x2F;4-bit and 3&#x2F;5-bit are seen as in practice similar complex they are now really similar in complexity.<p>CBOR is a bit more consistent with it&#x27;s encodings then MessagePack (e.g. MessagePack has &quot;fix int&quot; special cases which doesn&#x27;t follow the 4&#x2F;4-bit split). But it&#x27;s also lacking boolean and null in it&#x27;s core data model.<p>CBORs encoding of true,false and null is ... iffy (and 2 bytes). It also has both null and undefined for whatever cursed reasons.<p>MessagePack has a extension system which associate a custom type [0;127] with a byte array.<p>CBOR has a tagging system which associates a custom type [0;2*64-1] with any kind of CBOR type.<p>In this aspect MessagePack is simpler but also so restrictive that running into collisions in large interconnected enterprise use cases is not just viable but expected if extension types are widely used (but then for most apps 0..127 is good enough).<p>On the other hand if a CBOR library wants to support all kinds of complex custom tag based extension use cases it could add quite a bit to the libraries complexity. But then ignoring tags when parsing is valid. On benefit of the large tag space is that there are some pretty useful predefined tags, e.g. for a unix timestamp or that a byte slice is itself valid CBOR or big num encoding.<p>Lastly CBOR supports &quot;infinite&quot;&#x2F;&quot;unknown when encoding starts&quot; length bytes string, utf-8 strings, lists and maps. That clearly adds complexity to any library fully implementing CBOR, but if you need it removes a lot of complexity from you as you now don&#x27;t need to come up with some JsonL-ish format for MessagePack (which is trivial but outside of the spec). Most importantly you can stream insides of nested items, not viable with MessagePack).