JSON

JSON is awesome, but it&#x27;s troublesome that it can&#x27;t represent binary data (without a separate encoding, which requires metadata, and more code on both sides of the wire).<p>I &quot;discovered&quot; a format that easily solves this, which I call QSN (quoted string notation):<p><a href="http:&#x2F;&#x2F;www.oilshell.org&#x2F;release&#x2F;latest&#x2F;doc&#x2F;qsn.html" rel="nofollow">http:&#x2F;&#x2F;www.oilshell.org&#x2F;release&#x2F;latest&#x2F;doc&#x2F;qsn.html</a><p>It&#x27;s just Rust string literals with single quotes instead of double quotes. Rust string literals have \x01 for arbitrary bytes like C or Python string literals, but without the legacy of say \v and octal, and with just \u{} rather than \u and \U.<p>I use in Oil to unambiguously parse and print filenames, display argv arrays, etc. All of these are arbitrary binary data.<p>GNU tools have about 10 different ways to do this, but QSN makes it consistent and precise.<p>I&#x27;m expanding it to QTSV, a variant of TSV, but if you like you could also make some JSON variant. Technically using single quotes rather than double would make it backward compatible with JSON.

JSON is great and all, but we really, really need to agree on a binary format that can support a few more data type including binary and dates.<p>I get that JSON being text is easy to debug, but text is just a binary format that has viewers built into everything (utf8) if we agreed on a a more structured hierarchal binary format there would be a viewer for it everywhere.<p>Taking it further a text file should just be one of these fictional hierarchal binary format files with a single string node for the text, maybe with some agreed metadata nodes as well, same with most other file formats.

JSON Numbers cannot represent all possible values of an IEEE754 Double (IE Javascript numbers) since it&#x27;s missing all of the NANs and infinities.<p>JSON Numbers cannot all be represented in an IEEE754 Double, since they&#x27;re arbitrary precision sequences of unbounded size.<p>This bit me recently, and so I&#x27;m slightly bitter about it.

I&#x27;m always impressed by this website&#x27;s design and clear information. Obvious flow charts, the basic info, and all on one simple page. I know json is very simple, but I would love it if other technologies followed this design.

JSON has been goto format for data exchange from a while. I think native browser support is biggest reason of its success.<p>Is there any competing standard? I dont think I miss any features in JSON.

I like JSON a lot, it&#x27;s my go to for ad-hoc data storage. I&#x27;ve found it extremely convenient because nearly all modern APIs can return json. Further, jq makes working with json extremely ergonomic.<p>For certain tasks (personal use) I hand roll my own json document storage database. I make a bunch of rest API calls to collect data and store it in flat files. While doing this, I use JQ to store subsets of that information (the stuff I really care about) in smaller json blobs. I then write a script to aggregate all of the smaller blobs into a larger json array.<p>That&#x27;s almost no work and you already get a nice schema-less database. Write some commands (stored-procedures) to do any kind of filtering&#x2F;modifications and you can immediately get whatever view you want of your data with one command. Write a wrapper script to identify documents by a field (primary key) and you can make data modifications in an ergonomic way. I run &quot;modify-document &lt;primary-key&gt;&quot; and it runs a tiny readline bash script prompting me for info which immediately modifies the corresponding database row.<p>A work flow could be...<p>1. Make API requests (with gnu parallel) storing the response json and a smaller json blob.<p>2. Aggregate the small blobs into an array.<p>3. Filter blobs for any that are missing information.<p>4. Manually update blobs missing information with readline script.<p>5. Filter for blobs that need processing.<p>6. Process blobs, use readline script to mark them as processed.<p>7. Continue until all blobs are processed.<p>This is the kind of thing I would use excel to do in the past. Hopefully I never make that mistake again.

Protocol buffers solve most of the issues mentioned by people here, on top of being typesafe, space efficient (no need to encode key names because everything is an ordered struct), and having great tooling.<p><a href="https:&#x2F;&#x2F;developers.google.com&#x2F;protocol-buffers" rel="nofollow">https:&#x2F;&#x2F;developers.google.com&#x2F;protocol-buffers</a>