TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

JSON-Based Universal Messaging Format (2017)

60 pointsby cjusalmost 7 years ago

17 comments

blattimwindalmost 7 years ago
&gt; Note that there is a complex canonicalisation procedure for the JSON object, and that the sender must mutate the signed object;<p>This is a big no-no and actual source of vulnerabilities. If you sign something, the signature goes <i>around</i> what you want to sign.[1] Doing &quot;in-line&quot; signatures is excessively more complex and error-prone. The easiest and most secure scheme is actually &quot;sign a blob of bytes&quot;, i.e. signing a packed representation of a message. That way, you get zero ambiguity issues as far as signature-content interactions go [2], and you don&#x27;t actually need a canonicalized message representation any more (which is not a common feature of serialization formats outside ASN.1 encodings).<p>There might be other reasons to not use UMF, but this one is completely sufficient to avoid it.<p>(Also calling HMAC tags &quot;signatures&quot; is confusing as heck and should be avoided.)<p>(Also the actual method of how the MAC is calculated is not specified; so clearly UMF <i>is not</i> a format, it is a meta-format.)<p>[1] Even JWT got that right.<p>[2] Context ambiguity AKA The Horton Principle remains, because that&#x27;s not something a format solves.
评论 #17308511 未加载
评论 #17304933 未加载
评论 #17306667 未加载
jmullalmost 7 years ago
Maybe someone can comment, but I can&#x27;t understand the value of this.<p>The meat of this proposal is the specification of the envelope. But it consists largely of things you would have to decide how to map to your application domain. Since there&#x27;s no reason different apps would&#x2F;could map these things the same way, there&#x27;s no opportunity for interop created here.<p>I guess it ends up as an idea or example of how you might express a generic message in JSON. So that&#x27;s of some use.
评论 #17305439 未加载
评论 #17304215 未加载
kodablahalmost 7 years ago
&gt; UMF is being used in IoT (Internet of Things) applications where message sizes need to remain small. To allow UMF to be used in resource contained environments a short form of UMF is offered.<p>Then offer it in Protobuf :-) In general, I don&#x27;t understand where this would be used instead of an app&#x27;s specific envelope. The use cases should be specified better because it&#x27;s really easy for any app to just send their own preferred JSON format without following this.<p>Tangentially related, I similarly tried to make a messaging format (but with more bells and whistles and more about the transfer, storage, permissions, etc). The proto files are at [0] and the messaging platform is still under active development.<p>0 - <a href="https:&#x2F;&#x2F;github.com&#x2F;cretz&#x2F;yukup&#x2F;tree&#x2F;0cc926f98d01fba64b818383ee9763cbe61a6c64&#x2F;yukup&#x2F;pb&#x2F;proto" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;cretz&#x2F;yukup&#x2F;tree&#x2F;0cc926f98d01fba64b818383...</a>
lev99almost 7 years ago
A few issues I have with UMF.<p>It&#x27;s missing a mission statement, or use case.<p>The intro says it&#x27;s used to avoid inconsistent formatting, but there exists other message formats.<p>The body field is application specific, which introduces an area for inconsistent formatting and reduces the interoperability, or universalness, of the message format.<p>The destination doesn&#x27;t need to be in the message, most of the time the destination is implicit in where the message is sent. (If I send a UMF over email the destination would be in the email metadata and the UMF metadata)<p>The total amount of metadata that would be included in a UMF message sent over TCP&#x2F;IP would be silly.<p>There are arbitrary and undefended specification decisions. The transport protocol is left ambiguous, but both the schema and the format are specified. The metadata around body is very standardized, but anything goes inside body, which is the primary payload.
评论 #17308452 未加载
deeglesalmost 7 years ago
Don&#x27;t fields need to be sorted in order for signatures to be reproducible with JSON objects? there&#x27;s no mention of that.
评论 #17306376 未加载
valarauca1almost 7 years ago
My dude should quit his architect positions.<p>Consistent hashing of messages requires that every application which handles the message will lay the fields out in the same manner, this is a recursive problem for every layer that will put its data in the UMF. This right out the gate creates caching complexity issues.<p>Also for being so no cache control options by default.<p>Also there isn’t an error handling&#x2F;logging field so the responder will have to do something out-of-band to acknowledge errors.<p>Also there’s no mention of compression or alternative message formats, XML, proto, etc.<p>The sender field should likely be a URI encoded data. Then which mode (of several) which could get the message is an infrastructure not an encoding program. Getting this deep here is kind of bad, same with the TTL. If we are going this deep... why no cache control&#x2F;consistent hashing&#x2F;compressing?<p>If you care about strictness or correctness you should use RFC3339 instead of ISO8601. Also not including an implicit UnixTime&#x2F;UnixTime Nano seconds is a poor choice.<p>Offering alternative keys for standard keys is dumb. This feels like an accident that nobody caught until later, and made a backwards compatible monkey patch.<p>Why haven’t you considered protobuf if you have concerns about message size, and are working in power constraints? UTF&#x2F;Base64 is kind of shitte for power savings.
评论 #17308352 未加载
continuationalalmost 7 years ago
Projects like these need a motivation section.<p>It’s not clear what the usecase is.
CyanLite2almost 7 years ago
I love open source projects with no documentation or Getting Started guides.....said no one ever.
评论 #17303282 未加载
cjusalmost 7 years ago
Wow - I really ruffled some feathers with this post :-D As I scanned the feedback, I found myself agreeing with some, but not all, of the comments.<p>I actually found some of the direct attacks amusing and got a good laugh. That said, I&#x27;d like to thank everyone who took the time to comment. One of the goals of any specification should be to iterate the spec based on the valuable feedback of others. I&#x27;ll definitely take this opportunity to do that.<p>Thanks again.
zamalekalmost 7 years ago
Doesn&#x27;t JSON-LD (especially AcitivityPub+co) mostly deal with this already?
profalseidolalmost 7 years ago
<a href="https:&#x2F;&#x2F;github.com&#x2F;edn-format&#x2F;edn" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;edn-format&#x2F;edn</a><p>edn is an extensible data notation. A superset of edn is used by Clojure to represent programs, and it is used by Datomic and other applications as a data transfer format. This spec describes edn in isolation from those and other specific use cases, to help facilitate implementation of readers and writers in other languages, and for other uses.
cjusalmost 7 years ago
Reading through the comments here I&#x27;m realizing that my single biggest error may have been the use of the word &quot;Universal&quot; :-D When you consider the spec as a &quot;basis for agreement&quot; between distributed application authors and not a unified theory of messaging - then the spec becomes a lot clearer.
zevebalmost 7 years ago
I&#x27;ll just note, once again, how verbose, unattractive &amp; difficult to parse JSON is compared to S-expressions. Here are several of the examples in the spec in both formats, in order (I&#x27;ve rearranged the field so that required fields come first &amp; optional fields come later).<p>2.1:<p><pre><code> { &quot;mid&quot;: &quot;ef5a7369-f0b9-4143-a49d-2b9c7ee51117&quot;, &quot;rmid&quot;: &quot;66c61afc-037b-4229-ace4-5ec4d788903e&quot;, &quot;to&quot;: &quot;uid:123&quot;, &quot;from&quot;: &quot;uid:56&quot;, &quot;type&quot;: &quot;dm&quot;, &quot;version&quot;: &quot;UMF&#x2F;1.4.3&quot;, &quot;priority&quot;: &quot;10&quot;, &quot;timestamp&quot;: &quot;2013-09-29T10:40Z&quot;, &quot;body&quot;: { &quot;message&quot;: &quot;How is it going?&quot; } } (message ef5a7369-f0b9-4143-a49d-2b9c7ee51117 (to uid:123) (from uid:56) (version SMF&#x2F;1.4.3) (timestamp 2013-09-29T10:40Z) (rmid 66c61afc-037b-4229-ace4-5ec4d788903e) (type dm) (priority 10) (body (message &quot;How is it going?&quot;))) </code></pre> 2.2.11:<p><pre><code> { &quot;mid&quot;: &quot;ef5a7369-f0b9-4143-a49d-2b9c7ee51117&quot;, &quot;to&quot;: &quot;uid:56&quot;, &quot;from&quot;: &quot;game:store&quot;, &quot;version&quot;: &quot;UMF&#x2F;1.3&quot;, &quot;timestamp&quot;: &quot;2013-09-29T10:40Z&quot;, &quot;body&quot;: { &quot;type&quot;: &quot;store:purchase&quot;, &quot;itemID&quot;: &quot;5x:winnings:multiplier&quot;, &quot;expiration&quot;: &quot;2014-02-10T10:40Z&quot; } } (message ef5a7369-f0b9-4143-a49d-2b9c7ee51117 (to uid:56) (from game:store) (version UMF&#x2F;1.3) (timestamp 2013-09-29T10:40Z) (body (type store:purchase) (itemID &quot;5x:winnings:multiplier&quot;) (expiration 2014-02-10T10:40Z))) </code></pre> 2.2.11.2<p>Note how JSON has to rely on metadata to indicate that a Base64 sequence, whereas it&#x27;s natively supported by canonical S-expressions. Note also how the S-expression format natively supports types (&#x27;display hints&#x27;) for its values.<p><pre><code> { &quot;mid&quot;: &quot;ef5a7369-f0b9-4143-a49d-2b9c7ee51117&quot;, &quot;to&quot;: &quot;uid:134&quot;, &quot;from&quot;: &quot;uid:56&quot;, &quot;version&quot;: &quot;UMF&#x2F;1.3&quot;, &quot;timestamp&quot;: &quot;2013-09-29T10:40Z&quot;, &quot;body&quot;: { &quot;type&quot;: &quot;private:message&quot;, &quot;contentType&quot;: &quot;text&#x2F;plain&quot;, &quot;base64&quot;: &quot;SSBzZWUgeW91IHRvb2sgdGhlIHRyb3VibGUgdG8gZGVjb2RlIHRoaXMgbWVzc2FnZS4=&quot; } } (message ef5a7369-f0b9-4143-a49d-2b9c7ee51117 (to uid:134) (from uid:56) (version SMF&#x2F;1.3) (timestamp 2013-09-29T10:40Z) (body (type private:message) [text&#x2F;plain]|SSBzZWUgeW91IHRvb2sgdGhlIHRyb3VibGUgdG8gZGVjb2RlIHRoaXMgbWVzc2FnZS4=|)) </code></pre> 2.2.11.3<p>One might expect that S-expressions might shine when it comes to sending multiple items, and of course one would be correct.<p>Also note how the parallel structure of the message &amp; message&#x2F;body&#x2F;message objects raises the question of whether the message&#x2F;body&#x2F;message schema should also be UMF.<p><pre><code> { &quot;mid&quot;: &quot;ef5a7369-f0b9-4143-a49d-2b9c7ee51117&quot;, &quot;to&quot;: &quot;uid:134&quot;, &quot;from&quot;: &quot;chat:room:14&quot;, &quot;version&quot;: &quot;UMF&#x2F;1.3&quot;, &quot;timestamp&quot;: &quot;2013-09-29T10:40Z&quot;, &quot;body&quot;: { &quot;type&quot;: &quot;chat:messages&quot;, &quot;messages&quot;: [ { &quot;from&quot;: &quot;moderator&quot;, &quot;text&quot;: &quot;Susan welcome to chat Nation NYC&quot;, &quot;ts&quot;: &quot;2013-09-29T10:34Z&quot; }, { &quot;from&quot;: &quot;uid:16&quot;, &quot;text&quot;: &quot;Rex, you are one lucky SOB!&quot;, &quot;ts&quot;: &quot;2013-09-29T10:30Z&quot; }, { &quot;from&quot;: &quot;uid:133&quot;, &quot;text&quot;: &quot;Rex you&#x27;re going down this next round&quot;, &quot;ts&quot;: &quot;2013-09-29T10:31Z&quot; } ] } } (message ef5a7369-f0b9-4143-a49d-2b9c7ee51117 (to uid:134) (from chat:room:14) (version SMF&#x2F;1.3) (timestamp 2013-09-29T10:40Z) (body (type chat:messages) (messages (message (from moderator) (text &quot;Susan welcome to chat Nation NYC&quot;) (ts 2013-09-29T10:34Z)) (message (from uid:16) (text &quot;Rex, you are one lucky SOB!&quot;) (ts 2013-09-29T10:30Z)) (message (from uid:133) (text &quot;Rex you&#x27;re going down this next round&quot;) (ts 2013-09-29T10:31Z))))) </code></pre> 2.2.17<p>Note that there is a complex canonicalisation procedure for the JSON object, and that the sender must mutate the signed object; by contrast, the S-expression format is properly layered and doesn&#x27;t mutate signed objects (which implies that it&#x27;s possible to chain signatures cleanly).<p><pre><code> { &quot;mid&quot;: &quot;ef5a7369-f0b9-4143-a49d-2b9c7ee51117&quot;, &quot;to&quot;: &quot;uid:123&quot;, &quot;from&quot;: &quot;uid:56&quot;, &quot;version&quot;: &quot;UMF&#x2F;1.4.6&quot;, &quot;signature&quot;: &quot;c0fa1bc00531bd78ef38c628449c5102aeabd49b5dc3a2a516ea6ea959d6658e&quot;, &quot;body&quot;: {} } (signature (message ef5a7369-f0b9-4143-a49d-2b9c7ee51117 (to uid:123) (from uid:56) (version SMF&#x2F;1.4.6) (body)) |c0fa1bc00531bd78ef38c628449c5102aeabd49b5dc3a2a516ea6ea959d6658e|) </code></pre> It&#x27;s not to late to switch away from JSON, it really isn&#x27;t.
评论 #17304146 未加载
评论 #17305041 未加载
评论 #17304651 未加载
评论 #17305741 未加载
评论 #17304913 未加载
评论 #17308362 未加载
setqukalmost 7 years ago
WS-* meets JSON.
rektidealmost 7 years ago
This spec should be rebuilt from the start to extend <a href="http:&#x2F;&#x2F;cloudevents.io" rel="nofollow">http:&#x2F;&#x2F;cloudevents.io</a> .
lloydsparkesalmost 7 years ago
<a href="https:&#x2F;&#x2F;xkcd.com&#x2F;927&#x2F;" rel="nofollow">https:&#x2F;&#x2F;xkcd.com&#x2F;927&#x2F;</a>
pkilgorealmost 7 years ago
Direct link to the spec: <a href="https:&#x2F;&#x2F;github.com&#x2F;cjus&#x2F;umf&#x2F;blob&#x2F;master&#x2F;umf.md" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;cjus&#x2F;umf&#x2F;blob&#x2F;master&#x2F;umf.md</a><p>Relevant xkcd: <a href="https:&#x2F;&#x2F;xkcd.com&#x2F;927&#x2F;" rel="nofollow">https:&#x2F;&#x2F;xkcd.com&#x2F;927&#x2F;</a>
评论 #17303448 未加载
评论 #17303278 未加载