TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Q: A faster re-implementaiton of jq written in Reason Native/OCaml

250 pointsby davesnxover 4 years ago

27 comments

aasasdover 4 years ago
For everyone pining for a Jq with a different syntax: I have a bunch of links to alternatives collected, you might want to try some of them (some may be for different things than JSON):<p><a href="https:&#x2F;&#x2F;github.com&#x2F;fiatjaf&#x2F;awesome-jq" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;fiatjaf&#x2F;awesome-jq</a><p><a href="https:&#x2F;&#x2F;github.com&#x2F;TomConlin&#x2F;json2xpath" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;TomConlin&#x2F;json2xpath</a><p><a href="https:&#x2F;&#x2F;github.com&#x2F;antonmedv&#x2F;fx" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;antonmedv&#x2F;fx</a><p><a href="https:&#x2F;&#x2F;github.com&#x2F;fiatjaf&#x2F;jiq" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;fiatjaf&#x2F;jiq</a><p><a href="https:&#x2F;&#x2F;github.com&#x2F;simeji&#x2F;jid" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;simeji&#x2F;jid</a><p><a href="https:&#x2F;&#x2F;github.com&#x2F;jmespath&#x2F;jp" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;jmespath&#x2F;jp</a><p><a href="https:&#x2F;&#x2F;github.com&#x2F;cube2222&#x2F;jql" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;cube2222&#x2F;jql</a><p><a href="https:&#x2F;&#x2F;jsonnet.org" rel="nofollow">https:&#x2F;&#x2F;jsonnet.org</a><p><a href="https:&#x2F;&#x2F;github.com&#x2F;borkdude&#x2F;jet" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;borkdude&#x2F;jet</a><p><a href="https:&#x2F;&#x2F;github.com&#x2F;jzelinskie&#x2F;faq" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;jzelinskie&#x2F;faq</a><p><a href="https:&#x2F;&#x2F;github.com&#x2F;dflemstr&#x2F;rq" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;dflemstr&#x2F;rq</a><p>Personally I think that next time I might just fire up Hy and use its functional capabilities.
评论 #24476950 未加载
评论 #24471484 未加载
评论 #24473384 未加载
评论 #24473280 未加载
评论 #24471812 未加载
评论 #24472627 未加载
评论 #24475064 未加载
评论 #24475930 未加载
评论 #24479845 未加载
jeffbeeover 4 years ago
1) refuses to operate on stdin; requires a filename argument, which is so irritating.<p>2) doesn&#x27;t accept values that jq accepts<p><pre><code> % time jq -r &#x27;[expression]&#x27; &lt; parcels | wc 365 1454 7978 jq -r &lt; parcels 1.39s user 0.00s system 99% cpu 1.390 total wc 0.00s user 0.00s system 0% cpu 1.390 total % time ~&#x2F;.yarn&#x2F;bin&#x2F;q &#x27;[expression]&#x27; parcels | wc q: internal error, uncaught exception: Yojson.Json_error(&quot;Line 56, bytes -1-32:\nJunk after end of JSON value: &#x27;{\n \&quot;OBJECTID\&quot;: 155303,\n \&quot;BOOK\&quot;&#x27;&quot;)</code></pre>
评论 #24470914 未加载
评论 #24470111 未加载
评论 #24470688 未加载
toastalover 4 years ago
Are we sure it should get a single-letter &#x27;q&#x27; binary name though? Docs seem to point that it&#x27;s short for &#x27;query-json&#x27;? Why not call it &#x27;query-json&#x27; and let the user decide that as a shell alias or whatever. Even the ubiquitous &#x27;ls&#x27; and &#x27;cd&#x27; are two characters.
评论 #24471524 未加载
评论 #24469815 未加载
评论 #24470600 未加载
评论 #24469435 未加载
评论 #24469131 未加载
评论 #24469156 未加载
评论 #24470719 未加载
andylynchover 4 years ago
This looks interesting, but could be confusing given the programming language of the same name (<a href="https:&#x2F;&#x2F;code.kx.com&#x2F;q&#x2F;" rel="nofollow">https:&#x2F;&#x2F;code.kx.com&#x2F;q&#x2F;</a>)
评论 #24469350 未加载
mkesperover 4 years ago
I&#x27;d long for such a tool with a better comprehensible query language.
评论 #24469616 未加载
评论 #24470624 未加载
评论 #24470733 未加载
评论 #24470217 未加载
评论 #24470746 未加载
评论 #24470490 未加载
评论 #24469250 未加载
as-jover 4 years ago
&gt; Aside from that, q isn&#x27;t feature parity with jq which is ok at this point, but jq contains a ton of functionality that query-json misses and some of the jq operations aren&#x27;t native, are builtin with the runtime. In order to do a proper comparision all of this above would need to take into consideration.<p>&gt; The report shows that q is between 2x and 5x faster than jq in all operations tested and same speed (~1.1x) with huge files (&gt; 100M).<p>While faster for somethings....that&#x27;s a pretty large set of caveats!
评论 #24470654 未加载
ristonover 4 years ago
Would be good if someone adds an explanation why this new approach is better, is it that the OCaml is faster, more efficient algorithms were used, etc?
评论 #24526781 未加载
jakubobozaover 4 years ago
Do we need to make jq faster ? Anyone has issues with current speed ? Is there any specific reason other than &quot;because we can&quot; ?
评论 #24469382 未加载
评论 #24469703 未加载
评论 #24469906 未加载
评论 #24469609 未加载
评论 #24471824 未加载
ksmgover 4 years ago
Hm, I thought q is synonym for querying CSV files <a href="https:&#x2F;&#x2F;harelba.github.io&#x2F;q&#x2F;" rel="nofollow">https:&#x2F;&#x2F;harelba.github.io&#x2F;q&#x2F;</a>
评论 #24469743 未加载
评论 #24472924 未加载
jamil7over 4 years ago
As an outsider I get very confused by the Reason &#x2F; Reason Native &#x2F; OCaml &#x2F; Bucklescript &#x2F; Rescript?! ecosystem. What does it mean for it to be written in Reason Native&#x2F;OCaml?
评论 #24471085 未加载
评论 #24470070 未加载
StavrosKover 4 years ago
This looks nice, but I was a bit dismayed at &quot;friends don&#x27;t let friends curl | bash, to install this run curl | bash&quot;.
评论 #24469323 未加载
评论 #24469045 未加载
评论 #24470516 未加载
jonemiover 4 years ago
I used to be a regular user of jq, but I was never parsing very large JSON. I now do what I used to do with jq in my browser&#x27;s developer tools console. Map and filter are far more familiar than jq&#x27;s syntax where I found myself referring to the documentation most of the time.<p>I&#x27;m sure other people have use cases where the browser wouldn&#x27;t meet their needs, but for me, I find jq unnecessary.
评论 #24472322 未加载
gkfasdfasdfover 4 years ago
Curious, any description as to why it&#x27;s faster? Something intrinsic to Reason Native&#x2F;OCaml? Architectural changes? Reduced feature set?
评论 #24469671 未加载
评论 #24470270 未加载
评论 #24469679 未加载
dkdk8283over 4 years ago
Is jq slow? I have only worked with datasets up to 1mb but I’ve never had a performance issue that wasn’t attributed to my error.
评论 #24469060 未加载
评论 #24469148 未加载
评论 #24469063 未加载
评论 #24469558 未加载
评论 #24469307 未加载
Ericson2314over 4 years ago
This is funny because Stephen Dolan, the original jq author, works on OCaml itself.
评论 #24479829 未加载
vasergenover 4 years ago
The speed is not concern for me. I am wondering if there something better than `jq` in terms of syntax. Whenever I want to get something more that just prettify json output in the console or simply get value by specific field name I have a problem, for me it is just difficult to remember jq syntax without looking into history. As well have in my notes links to examples like this one<p><a href="https:&#x2F;&#x2F;mosermichael.github.io&#x2F;jq-illustrated&#x2F;dir&#x2F;content.html" rel="nofollow">https:&#x2F;&#x2F;mosermichael.github.io&#x2F;jq-illustrated&#x2F;dir&#x2F;content.ht...</a>
评论 #24469784 未加载
评论 #24469768 未加载
评论 #24470269 未加载
评论 #24470751 未加载
评论 #24475976 未加载
layoutIfNeededover 4 years ago
Umm... There&#x27;s already a language called Q for array processing.
评论 #24470112 未加载
heycosmoover 4 years ago
In case anyone is interested in yet another alternative, I have this old, unpolished project: <a href="https:&#x2F;&#x2F;github.com&#x2F;bauerca&#x2F;jv" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;bauerca&#x2F;jv</a><p>It is a JSON parser in C without heap allocations. The query language is piddly, but the tool can be useful for grabbing a single value from a very large JSON file. I don&#x27;t have time for it, but someone could fork and make it a real deal.
Borkdudeover 4 years ago
If you&#x27;re into Clojure, check out <a href="https:&#x2F;&#x2F;github.com&#x2F;borkdude&#x2F;jet" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;borkdude&#x2F;jet</a>
评论 #24472556 未加载
RMPRover 4 years ago
Upcoming q-rs a rewrite of q in Rust :p
评论 #24479823 未加载
muktabhover 4 years ago
Slightly out of context here, I find the entire stack of bsb, bsb-native, ocaml and esy pretty cool. However, I just dont find enough resources, good tutorials etc on Google search. Is there a good set of beginner tutorials anyone can point to ? Thanks in advance.
评论 #24470506 未加载
skywhopperover 4 years ago
This is cool, but I’m not sure it’s fair to claim it’s “faster” yet when it doesn’t do 95% of what jq does—-particularly the command line options. If it’s still faster when you can match 80% of the functionality, then it might be a claim worth making.
评论 #24479835 未加载
YesThatTom2over 4 years ago
Great! Now improve the syntax!
评论 #24469587 未加载
评论 #24470756 未加载
评论 #24470519 未加载
brundolfover 4 years ago
I&#x27;d love to hear some speculation - from the author or otherwise - as to why a fresh OCaml implementation would so dramatically outperform a mature C implementation
评论 #24479814 未加载
nikolayover 4 years ago
JMESPath is the only viable alternative, which probably has a wider footprint than even jq as it&#x27;s part of AWS CLI.
评论 #24475269 未加载
tus88over 4 years ago
Isn&#x27;t JQ written in C? I doubt LISP is going to be faster.
评论 #24479842 未加载
yahyaheeeover 4 years ago
I’m with Q!
评论 #24479844 未加载