TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Zq: An easier and faster alternative to jq

462 点作者 mccanne大约 3 年前

49 条评论

mccanne大约 3 年前
Hi, all. Author here. Thanks for all the great feedback.<p>I&#x27;ve learned a lot from your comments and pointers.<p>The Zed project is broader than &quot;a jq alternative&quot; and my bad for trying out this initial positioning. I do know there are a lot of people out there who find jq really confusing, but it&#x27;s clear if you become an expert, my arguments don&#x27;t hold water.<p>We&#x27;ve had great feedback from many of our users who are really productive with the blend of search, analytics, and data discovery in the Zed language, and who find manipulating eclectic data in the ZNG format to be really easy.<p>Anyway, we&#x27;ll write more about these other aspects of the Zed project in the coming weeks and months, and in the meantime, if you find any of this intriguing and want to kick the tires, feel free to hop on our slack with questions&#x2F;feedback or file GitHub issues if you have ideas for improvements or find bugs.<p>Thanks a million!<p><a href="https:&#x2F;&#x2F;github.com&#x2F;brimdata&#x2F;zed" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;brimdata&#x2F;zed</a> <a href="https:&#x2F;&#x2F;www.brimdata.io&#x2F;join-slack&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.brimdata.io&#x2F;join-slack&#x2F;</a>
评论 #31172512 未加载
评论 #31172558 未加载
评论 #31171047 未加载
评论 #31244771 未加载
评论 #31171209 未加载
weinzierl大约 3 年前
jq is incredibly powerful and I&#x27;m using it more and more. Even better, there is a whole ecosystem of tools that are similar or work in conjunction with jq:<p>* jq (a great JSON-wrangling tool)<p>* jc (convert various tools’ output into JSON)<p>* jo (create JSON objects)<p>* yq (like jq, but for YAML)<p>* fq (like jq, but for binary)<p>* htmlq (like jq, but for HTML)<p>List shamelessly stolen from Julia Evans[1]. For live links see her page.<p>Just a few days ago I needed to quickly extract all JWT token expiration dates from a network capture. This is what I came up with:<p><pre><code> fq &#x27;grep(&quot;Authorization: Bearer.*&quot; ) | print&#x27; server.pcap | grep -o &#x27;ey.*$&#x27; | sort | uniq | \ jq -R &#x27;[split(&quot;.&quot;) | select(length &gt; 0) | .[0],.[1] | gsub(&quot;-&quot;;&quot;+&quot;) | gsub(&quot;_&quot;;&quot;&#x2F;&quot;) | @base64d | fromjson]&#x27; | \ jq &#x27;.[1]&#x27; | jq &#x27;.exp&#x27; | xargs -n1 -I! date &#x27;+%Y-%m-%d %H:%M:%S&#x27; -d @! </code></pre> It&#x27;s not a beauty but I find the fact that you can do it in one line, with proper parsing and no regex trickery, remarkable.<p>[1] <a href="https:&#x2F;&#x2F;jvns.ca&#x2F;blog&#x2F;2022&#x2F;04&#x2F;12&#x2F;a-list-of-new-ish--command-line-tools&#x2F;" rel="nofollow">https:&#x2F;&#x2F;jvns.ca&#x2F;blog&#x2F;2022&#x2F;04&#x2F;12&#x2F;a-list-of-new-ish--command-l...</a>
评论 #31168116 未加载
评论 #31171977 未加载
评论 #31168709 未加载
评论 #31168455 未加载
评论 #31175369 未加载
评论 #31177510 未加载
评论 #31175437 未加载
msluyter大约 3 年前
Whenever jq comes up I feel obligated to mention &#x27;gron&#x27;[1]. If all you&#x27;re doing is trying to grep some deeply nested field, it&#x27;s way easier with gron, IMHO.<p>[1] <a href="https:&#x2F;&#x2F;github.com&#x2F;tomnomnom&#x2F;gron" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;tomnomnom&#x2F;gron</a>
评论 #31170962 未加载
评论 #31169528 未加载
评论 #31169262 未加载
psacawa大约 3 年前
Since no one seems to know about it, jq is described in <i>great</i> detail on the github wiki page [0]. That flattens the learning curve a lot. It&#x27;s not as arcane as it seems.<p>The touted claim that is fundamentally stateless is not true. jq is also stateful in the sense that it has variables. If you want, you can write regular procedural code this way. Some examples [1]<p>The real problem of jq is that it is currently lacking a maintainer to assess a number of PRs that have accumulated since 2018.<p>[0] <a href="https:&#x2F;&#x2F;github.com&#x2F;stedolan&#x2F;jq&#x2F;wiki&#x2F;jq-Language-Description" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;stedolan&#x2F;jq&#x2F;wiki&#x2F;jq-Language-Description</a><p>[1] <a href="https:&#x2F;&#x2F;github.com&#x2F;fadado&#x2F;JBOL&#x2F;blob&#x2F;master&#x2F;fadado.github.io&#x2F;array&#x2F;array.jq" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;fadado&#x2F;JBOL&#x2F;blob&#x2F;master&#x2F;fadado.github.io&#x2F;...</a>
评论 #31168383 未加载
评论 #31168827 未加载
评论 #31168225 未加载
评论 #31167863 未加载
评论 #31168916 未加载
评论 #31173225 未加载
评论 #31170283 未加载
eatonphil大约 3 年前
If jq is getting too slow for you (that&#x27;s never happened for me), it really seems like it&#x27;s time to put your data in a database like sqlite or duckdb at least.<p>Incidentally there are many tools that help you do this like dsq [0] (which I develop), q [1], textql [2], etc.<p>[0] <a href="https:&#x2F;&#x2F;github.com&#x2F;multiprocessio&#x2F;dsq" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;multiprocessio&#x2F;dsq</a><p>[1] <a href="https:&#x2F;&#x2F;github.com&#x2F;harelba&#x2F;q" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;harelba&#x2F;q</a><p>[2] <a href="https:&#x2F;&#x2F;github.com&#x2F;dinedal&#x2F;textql" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;dinedal&#x2F;textql</a>
评论 #31168592 未加载
algesten大约 3 年前
I don&#x27;t get it. &quot;Instead of learning jq DSL, learn zq DSL&quot;.<p>To me they look similarly complicated and the examples stresses certain aggregation operations that are harder to do in jq (due to it being stateless).
评论 #31167418 未加载
评论 #31167843 未加载
评论 #31167549 未加载
knome大约 3 年前
These guys must really hate functional programming.<p>I can see where jq might confuse someone new to it, but their replacement is irregular, stateful, still difficult, and I don&#x27;t even see variable binding or anything.<p>jq requires you to understand that `hello|world` will run world for each hello, passing the world out values to either the next piped expression, the wrapping value-collecting list, or printing them to stdout.<p>it&#x27;s a bit unintuitive if you come in thinking of them as regular pipelines, but it&#x27;s a constant in the language that once learned always applies.<p>this zed thing has what appears to be a series of workarounds for its own awkwardness, where they kept tacking on new forms to try to bandaid those that came before.<p>additionally, since they made attribute selectors barewords where jq would require a preceding reference to a variable or the current value (.), I&#x27;m not sure where they&#x27;ll go for variables should they add them.
评论 #31168201 未加载
评论 #31171324 未加载
评论 #31168724 未加载
评论 #31173653 未加载
评论 #31171214 未加载
评论 #31170293 未加载
micimize大约 3 年前
Their syntax comparison under &quot;So you like chocolate or vanilla?&quot; is disingenuous. You can do variable assignment and array expansion in jq:<p><pre><code> expand_vals_into_independent_records=&#x27; .name as $name | .vals[] | { name: $name, val: . } &#x27; echo &#x27;{&quot;name&quot;:&quot;foo&quot;,&quot;vals&quot;:[1,2,3]} {&quot;name&quot;:&quot;bar&quot;,&quot;vals&quot;:[4,5]}&#x27; | jq &quot;$expand_vals_into_independent_records&quot; </code></pre> Also, generally, not a fan of the tone of this article.
评论 #31170187 未加载
diehunde大约 3 年前
Pardon my ignorance, but would I spend time learning something like jq or zq when it only takes me a couple of minutes to develop a script using some high-level language? I&#x27;ve had to process complex JSON files in the past, and a simple Python script gets the job done, and the syntax is much more familiar and easier to memorize. Is there a use case I&#x27;m missing?
评论 #31167948 未加载
评论 #31167997 未加载
评论 #31167976 未加载
评论 #31168076 未加载
评论 #31168223 未加载
评论 #31171953 未加载
评论 #31171223 未加载
评论 #31171632 未加载
评论 #31167986 未加载
评论 #31168022 未加载
评论 #31168291 未加载
brushfoot大约 3 年前
The name of its corporate progenitor may leave a bad taste in some mouths, but I highly recommend PowerShell for this sort of thing. It&#x27;s cross platform, MIT licensed, and comes with excellent JSON parsing and querying capabilities. Reading, parsing, and querying JSON to return all red cars:<p><pre><code> Get-Content cars.json | ConvertFrom-Json | ? { $_.color -eq &#x27;red&#x27; } </code></pre> The beauty of this is that the query syntax applies not just to JSON but to every type of collection, so you don&#x27;t have to learn a specific syntax for JSON and another for another data type. You can use Get-Process on Linux to get running processes and filter them in the same way. The same for files, HTML tags, etc. I think nushell is doing something similar, though I haven&#x27;t tried it yet.<p>I prefer this approach to another domain-specific language, as interesting as jq&#x27;s and zq&#x27;s are.
评论 #31167677 未加载
评论 #31167605 未加载
评论 #31168795 未加载
评论 #31168000 未加载
评论 #31168186 未加载
评论 #31170582 未加载
评论 #31169018 未加载
评论 #31170676 未加载
评论 #31168680 未加载
评论 #31170322 未加载
AcerbicZero大约 3 年前
I&#x27;m pretty new to jq (maybe 2 years of exposure) but from my perspective - on some level, jq does to json what powershell does to everything windows, except powershell gives me the get-member cmdlet, so when I don&#x27;t know what is even in my object, I can explore.<p>Sometimes jq -r &#x27;.[]&#x27; works, but its all just trial and error. I use plenty of jq in my scripts, but I can never seem to visualize how jq looks at the data. I just have to toss variations of &#x27;.[whateveriwant].whatever[.want.]&#x27; until something works....I suppose the root of my complaint is that jq does not do a good job of teaching you to use jq. It either works, or gives you nothing, and while I&#x27;ve learned to work around that, I&#x27;ll try anything that claims to be even 1% better than jq.
评论 #31175183 未加载
abledon大约 3 年前
There is also &quot;JP&quot; <a href="https:&#x2F;&#x2F;github.com&#x2F;jmespath&#x2F;jp" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;jmespath&#x2F;jp</a><p>which follows the jmespath standard
评论 #31169053 未加载
hbbio大约 3 年前
jq is awesome, last time I used it is... today :)<p>Or rather the pure Go rewrite <a href="https:&#x2F;&#x2F;github.com&#x2F;itchyny&#x2F;gojq" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;itchyny&#x2F;gojq</a> which is a better faster implementation, with bugs fixed
评论 #31168227 未加载
politelemon大约 3 年前
&gt; HomeBrew for Mac or Linux<p>Please do not recommend HomeBrew for Linux. A binary download is safer compared to how HomeBrew clobbers a Linux machine. If you do not wish to use a Linux package manager, simply point at the binary download. It is much safer and less intrusive.
评论 #31172062 未加载
sfink大约 3 年前
The thing that I find myself wanting, which is lacking in both jq and zq afaik, is interactive exploration. I want to move around in a large JSON file, narrow my context to the portion I&#x27;m interested in, and do specialized queries and transformations on just the data I care about.<p>I wrote a tool to do this -- <a href="https:&#x2F;&#x2F;github.com&#x2F;hotsphink&#x2F;sfink-tools&#x2F;blob&#x2F;master&#x2F;bin&#x2F;json" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;hotsphink&#x2F;sfink-tools&#x2F;blob&#x2F;master&#x2F;bin&#x2F;jso...</a> -- but I do not recommend it to anyone other than as perhaps a source of inspiration. It&#x27;s slow and buggy, the syntax is cryptic and just matches whatever I came up with when I had a new need, etc. It probably wouldn&#x27;t exist if I had heard of jq sooner.<p>But for what it does, it&#x27;s <i>awesome</i>. I can do things like:<p><pre><code> % json somefile.json &gt; ls 0&#x2F; 1&#x2F; 2&#x2F; &gt; cd 0 &gt; ls info&#x2F; files&#x2F; timings&#x2F; version &gt; cat version 1.2b &gt; cat timings&#x2F;*&#x2F;mean timings&#x2F;firstPaint&#x2F;mean = 51 timings&#x2F;loadEventEnd&#x2F;mean = 103 timings&#x2F;timeToContentfulPaint&#x2F;mean = 68 timings&#x2F;timeToDomContentFlushed&#x2F;mean = 67 timings&#x2F;timeToFirstInteractive&#x2F;mean = 658 timings&#x2F;ttfb&#x2F;mean = 6 </code></pre> There are commands for searching, modifying data, aggregating, etc., but those would be better done in a more principled, full-featured syntax like jq&#x27;s.<p>I see ijq, and it looks really nice. But it doesn&#x27;t have the context and restriction of focus that I&#x27;m looking for.
评论 #31174063 未加载
评论 #31173702 未加载
评论 #31173588 未加载
评论 #31173701 未加载
评论 #31174198 未加载
评论 #31173784 未加载
评论 #31174135 未加载
lichtenberger大约 3 年前
I&#x27;m working on a JSONiq based implementation to jointly process JSON data and XML. The compiler uses set-oriented processing (and thus uses hash joins for instance wherever applicable) and is meant to provide a base for JSON based database systems with shared common optimizations (but can also be used as a standalone in-memory query processor):<p><a href="http:&#x2F;&#x2F;brackit.io" rel="nofollow">http:&#x2F;&#x2F;brackit.io</a><p>The language itself borrows a lot of concepts from functional languages as higher order functions, closures... you can also develop modules with functions for easy reuse...<p>A simple join for instance looks like this:<p><pre><code> let $stores := [ { &quot;store number&quot; : 1, &quot;state&quot; : &quot;MA&quot; }, { &quot;store number&quot; : 2, &quot;state&quot; : &quot;MA&quot; }, { &quot;store number&quot; : 3, &quot;state&quot; : &quot;CA&quot; }, { &quot;store number&quot; : 4, &quot;state&quot; : &quot;CA&quot; } ] let $sales := [ { &quot;product&quot; : &quot;broiler&quot;, &quot;store number&quot; : 1, &quot;quantity&quot; : 20 }, { &quot;product&quot; : &quot;toaster&quot;, &quot;store number&quot; : 2, &quot;quantity&quot; : 100 }, { &quot;product&quot; : &quot;toaster&quot;, &quot;store number&quot; : 2, &quot;quantity&quot; : 50 }, { &quot;product&quot; : &quot;toaster&quot;, &quot;store number&quot; : 3, &quot;quantity&quot; : 50 }, { &quot;product&quot; : &quot;blender&quot;, &quot;store number&quot; : 3, &quot;quantity&quot; : 100 }, { &quot;product&quot; : &quot;blender&quot;, &quot;store number&quot; : 3, &quot;quantity&quot; : 150 }, { &quot;product&quot; : &quot;socks&quot;, &quot;store number&quot; : 1, &quot;quantity&quot; : 500 }, { &quot;product&quot; : &quot;socks&quot;, &quot;store number&quot; : 2, &quot;quantity&quot; : 10 }, { &quot;product&quot; : &quot;shirt&quot;, &quot;store number&quot; : 3, &quot;quantity&quot; : 10 } ] let $join := for $store in $stores, $sale in $sales where $store=&gt;&quot;store number&quot; = $sale=&gt;&quot;store number&quot; return { &quot;nb&quot; : $store=&gt;&quot;store number&quot;, &quot;state&quot; : $store=&gt;state, &quot;sold&quot; : $sale=&gt;product } return [$join] </code></pre> Of course you can also group by, count, order by, nest FLWOR clauses...
评论 #31172940 未加载
arwineap大约 3 年前
I&#x27;ve never found jq to be particularly hard, or slow
评论 #31175207 未加载
cosmiccatnap大约 3 年前
I would love to see what jq looks like on something like a 1mil line Json vs this. In my experience jq syntax is fine and I&#x27;ve not ran into a performance issue on any one file but I seem to see a jq clone every few months on here so someone seems to need that, or maybe it&#x27;s just the new volume slider problem who knows.
评论 #31168971 未加载
xg15大约 3 年前
A bit OT:<p>The post links to the tutorial &quot;An Introduction to JQ&quot; at [1].<p>Somewhere inside the tutorial, array operators are introduced like this:<p>&gt; <i>jq lets you select the whole array [], a specific element [3], or ranges [2:5] and combine these with the object index if needed.</i><p>This is not supposed to be criticism on this particular tutorial (I&#x27;ve seen this kind of description quite often), but I could imagine this to be a typical &quot;eyes glaze over&quot; moment, where people subtly lose track of what is happening.<p>It appears to make sense on first glance, but leaves open the question what &quot;selecting the whole array&quot; actually means - especially, since you can write both &quot;.myarray&quot; and &quot;.myarray[]&quot; and both will select the whole array in a sense.<p>I think this is the point where one would really need to learn about sequences and about jq&#x27;s processing model to not get frustrated later.<p>[1] <a href="https:&#x2F;&#x2F;earthly.dev&#x2F;blog&#x2F;jq-select&#x2F;" rel="nofollow">https:&#x2F;&#x2F;earthly.dev&#x2F;blog&#x2F;jq-select&#x2F;</a>
评论 #31173504 未加载
knowsuchagency大约 3 年前
jq is a great tool, but my favorite alternative, by far, is jello and the libraries the author has created around it <a href="https:&#x2F;&#x2F;blog.kellybrazil.com&#x2F;2020&#x2F;03&#x2F;25&#x2F;jello-the-jq-alternative-for-pythonistas&#x2F;" rel="nofollow">https:&#x2F;&#x2F;blog.kellybrazil.com&#x2F;2020&#x2F;03&#x2F;25&#x2F;jello-the-jq-alterna...</a>
qmacro大约 3 年前
There&#x27;s a lot of references here to jq being &#x27;arcane&#x27;. For me, one of the challenges in improving my jq fu has been to find examples of larger programs, from which to learn.<p>One thing that seems to be perhaps a misconception amongst some is that jq invocations are short and only &#x27;one-liners&#x27;, and that a &#x27;real script&#x27; (in a &#x27;real language&#x27;) would be better in many cases. I think this lack of larger program examples probably helps to perpetuate this misunderstanding too.<p>Anyway, I was inspired enough by the article in question to write up some of my own thoughts on jq and statelessness: <a href="https:&#x2F;&#x2F;qmacro.org&#x2F;blog&#x2F;posts&#x2F;2022&#x2F;05&#x2F;02&#x2F;some-thoughts-on-jq-and-statelessness&#x2F;" rel="nofollow">https:&#x2F;&#x2F;qmacro.org&#x2F;blog&#x2F;posts&#x2F;2022&#x2F;05&#x2F;02&#x2F;some-thoughts-on-jq...</a>
29athrowaway大约 3 年前
&quot;Easier&quot; is subjective. For simple use-cases, zq is harder to understand than jq.<p>I also have never seen jq as a performance bottleneck.<p>jq is stable, I have never encountered a bug with it and I have never seen it getting stuck after years of usage. It is dependable and practical.<p>jq has helped me put out countless fires throughout my career. I should donate to it one day.
pm90大约 3 年前
It took me a while to grok jq, but now that I do I kinda like it? I don&#x27;t think I want to learn yet another thing.<p>I do like tools that complement&#x2F;supplement jq though, like jid: <a href="https:&#x2F;&#x2F;github.com&#x2F;simeji&#x2F;jid" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;simeji&#x2F;jid</a>
ilyash大约 3 年前
While we are at it, I have a list of JSON tools for command line here - <a href="https:&#x2F;&#x2F;ilya-sher.org&#x2F;2018&#x2F;04&#x2F;10&#x2F;list-of-json-tools-for-command-line&#x2F;" rel="nofollow">https:&#x2F;&#x2F;ilya-sher.org&#x2F;2018&#x2F;04&#x2F;10&#x2F;list-of-json-tools-for-comm...</a>
eru大约 3 年前
Jq being secretly a sort-of functional programming language is part of what makes it great.<p>Why would you change that?
gcmeplz大约 3 年前
I like using `jq` to create line-delimited JSON and then using a language I know well (Node) to process it after that point. I find `jq &#x27;.[] | select(.location==&quot;Stockholm&quot;)&#x27;` less readable than something like `nq --filter &#x27;({location}) =&gt; location === &quot;Stockholm&quot;&#x27;` because I&#x27;m much more used to Node syntax.<p>- <a href="https:&#x2F;&#x2F;github.com&#x2F;thisredone&#x2F;rb" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;thisredone&#x2F;rb</a> is a widely used ruby version of this idea<p>- <a href="https:&#x2F;&#x2F;github.com&#x2F;KelWill&#x2F;nq#readme" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;KelWill&#x2F;nq#readme</a> is something similar that I wrote for my own use
评论 #31168342 未加载
kaliszad大约 3 年前
For me, transforming JSON on the command line was a pain, another DSL to learn. Now, I can just use Babashka&#x2F; Clojure + one or two functions from Cheshire <a href="https:&#x2F;&#x2F;github.com&#x2F;dakrone&#x2F;cheshire" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;dakrone&#x2F;cheshire</a> where I need to. If I needed a standalone tool, I would perhaps reach for <a href="https:&#x2F;&#x2F;github.com&#x2F;borkdude&#x2F;jet" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;borkdude&#x2F;jet</a> by the same author, Michiel Borkent, as Babashka or use jq that everybody else would find more familiar.
ducaale大约 3 年前
In the theme of jq alternatives, there is fx[1] which has an interactive view and supports querying JSON in Javascript, Python and Ruby. It used to be a node CLI but was recently rewritten in golang[2]<p>[1] <a href="https:&#x2F;&#x2F;github.com&#x2F;antonmedv&#x2F;fx" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;antonmedv&#x2F;fx</a><p>[2] <a href="https:&#x2F;&#x2F;twitter.com&#x2F;antonmedv&#x2F;status&#x2F;1515429017582809090" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;antonmedv&#x2F;status&#x2F;1515429017582809090</a>
phibz大约 3 年前
I think of<p>echo &#x27;1 2 3&#x27; | jq ....<p>as creating three separate json documents, each with a single number as their top level &quot;document&quot; , body, or content.<p>So of course you can&#x27;t sum them. They are fed as separate documents to the jq pipeline as if you processed three separate jq commands.<p>Perhaps by stateless you mean no mutuable global state? But it certainly maintains state from the location in the input document to the output of each selector&#x2F;functor.<p>IMO it helps if you have a background in some of the concepts of functional programming.
ilyash大约 3 年前
In Next Generation Shell (author here), it is not as ergonomic (yet?) but on the other hand it&#x27;s a fully fledged no-nonsense programming language... and I claim quite a readable.<p>good_data = fetch(&quot;openlibrary.json&quot;).docs.filter({&quot;author_name&quot;: Arr, &quot;publish_year&quot;:Arr})<p>good_data.map({{&quot;title&quot;: A.title, &quot;author_name&quot;: A.author_name[0], &quot;publish_year&quot;: A.publish_year[0]}}).group(&quot;author_name&quot;).mapv(len).sortv((&gt;=)).limit(3)
taude大约 3 年前
I&#x27;m surprised no one mentioned rq [1] yet. It&#x27;s come up before in older HN threads [2] whenever the discussion on jq comes up...<p>[1] <a href="https:&#x2F;&#x2F;github.com&#x2F;dflemstr&#x2F;rq" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;dflemstr&#x2F;rq</a> [2] <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=13090604" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=13090604</a>
gzapp大约 3 年前
I&#x27;m sure I&#x27;m not the only person that got fed up with occasionally needing to do something more advanced and just finding the JQ incantations inscrutable.<p>Also prob not the first to create a project for personal use that just wraps evals in another language haha: <a href="https:&#x2F;&#x2F;www.npmjs.com&#x2F;package&#x2F;jsling" rel="nofollow">https:&#x2F;&#x2F;www.npmjs.com&#x2F;package&#x2F;jsling</a>
bradwood大约 3 年前
Nothing beats gron in my view.<p>That plus good old fashioned sed&#x2F;grep&#x2F;awk give me everything I need to do on the cli.<p>If I want more, it&#x27;s python or node.
quotemstr大约 3 年前
As an aside --- isn&#x27;t the traditional flat namespace of unix command names getting a <i>bit</i> crowded nowadays?
评论 #31175569 未加载
henrydark大约 3 年前
I have recently started to use jq massively, and I love it.<p>Zq looks cool, but the fact that this piece doesn&#x27;t contain a single instance of the word &quot;map&quot; tells me the authors still haven&#x27;t gotten jq. Especially with the running strawman example of adding numbers.
stblack大约 3 年前
Why all the hate HN?<p>I feel the author makes his case clearly, then presents an alternative. Underneath all this is a ton of work, for which I applaud OP.<p>It may not scratch your particular itch, but come on!<p>Being an ass on HN is a choice. It happens far too often, and I wish everyone would just dial it back.
评论 #31168639 未加载
评论 #31169692 未加载
dimensionc132大约 3 年前
Simple json tasks .... read from, write to, read a value and save it as a variable in BASH .... where are those examples?<p>The question for is this; can I do with json files what i can do with Python using Zq?
jrm4大约 3 年前
Okay, so I&#x27;m a big scripter and not much of a programmer and I definitely have found jq to be mostly worthless to me; but it also looks like zq doesn&#x27;t much help?<p>Seems to me that if you&#x27;re in a shell, then you should be &quot;shell-like.&quot; There should not be much of a learning curve at all, and when in doubt, try to behave like other shell tools, in a Unix way way. Make pipe behavior generally predictable, especially for those who aren&#x27;t deep into json et al.<p>And if you&#x27;re not going to do that, say so on &quot;the box?&quot;<p>(Disclaimer, it could be that I&#x27;m an idiot when it comes to all of this and I&#x27;m missing something big. Kind of feels that way, and I welcome correction)
评论 #31169494 未加载
caymanjim大约 3 年前
&lt;really long rant about jq being hard to use&gt; &lt;description of equally hard to use replacement&gt;<p>I almost gave up before I got to the first mention of zq, and then wished I had.
pygar大约 3 年前
I really wish that a jq type program was included in coreutils (or similar). I have wanted to to use it a few times but could never be sure it was going to be installed.
tus666大约 3 年前
The worst thing about JQ is printing out several values from an object at once. The syntax is so bad I have to look it up on SO every time.
spiralx大约 3 年前
Looks interesting, and I like the analysis of JQ&#x27;s model in order to compare and contrast it with ZQ. It reminds me somewhat of the difference between XPath 1 to XPath 2 - the former worked great if you were selecting stuff and using the built-in functions to do some simple transformations but if you tried to go further it was a hot mess of incompatible types each with their own constraints and quirks. In XPath 2 they completely revised the data model and type system, making everything an atomic value, node or a sequence of values and&#x2F;or nodes, allowing types to be queried and operated on, and added a ton of useful built-in functions and operators, making impossible queries in XPath 1 possible.<p><a href="https:&#x2F;&#x2F;docs.microsoft.com&#x2F;en-us&#x2F;archive&#x2F;msdn-magazine&#x2F;2003&#x2F;january&#x2F;the-xml-files-introducing-xpath-2-0" rel="nofollow">https:&#x2F;&#x2F;docs.microsoft.com&#x2F;en-us&#x2F;archive&#x2F;msdn-magazine&#x2F;2003&#x2F;...</a><p>Anyway, I&#x27;ve installed ZQ and will look to use it, even my simple usage of JQ had already led to thoughts of writing my own, better version :)<p>Quick bug report: On the Aggregate Functions page the link to _countdistinct_ goes to the page for _count_, and there actually isn&#x27;t a page at <a href="https:&#x2F;&#x2F;zed.brimdata.io&#x2F;docs&#x2F;language&#x2F;aggregates&#x2F;countdistinct&#x2F;" rel="nofollow">https:&#x2F;&#x2F;zed.brimdata.io&#x2F;docs&#x2F;language&#x2F;aggregates&#x2F;countdistin...</a>.
kryptozinc大约 3 年前
Is there a universal json normalizer (to csv for example) that doesn&#x27;t require learning a terse language?
评论 #31171191 未加载
harbor11012大约 3 年前
btw, in case you don&#x27;t know, you can actually run jq using a curl command:<p><a href="https:&#x2F;&#x2F;xbin.io&#x2F;w&#x2F;tool&#x2F;jq" rel="nofollow">https:&#x2F;&#x2F;xbin.io&#x2F;w&#x2F;tool&#x2F;jq</a>
tzury大约 3 年前
yq uses jq like syntax but works with YAML, JSON and XML.<p><a href="https:&#x2F;&#x2F;github.com&#x2F;mikefarah&#x2F;yq" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;mikefarah&#x2F;yq</a>
Aeolun大约 3 年前
Both zq and jq seem like black magic to me.
xg15大约 3 年前
So, admitted jq fanboy here, but I found a lot of the criticism from the articale really sensible.<p>I think jq has a pretty elegant data model, but the syntax is often very clunky to work with.<p>So here is a half thought-out idea how you might improve the syntax for the &quot;stateful operations&quot; usecase the OP outlined:<p>I think it&#x27;s not quite true that different elements of a sequence can never interact. The OP mentioned reduce&#x2F;foreach, but it&#x27;s also what any function that takes argument does:<p>If you have an expression &#x27;foo | bar&#x27;, then bar is called once for every element foo emits. However, foo could also a function that takes arguments. Then you can specify bar as an argument of foo like this: &#x27;foo(bar)&#x27;. In this situation, execution of bar is completely controlled by foo. In particular, foo gets to see <i>all</i> elements that foo emits, not just one each. I believe this is how e.g. [x] can collect all elements of x into an array.<p>In the same way, you could write a function &#x27;add_all(x)&#x27; which calls x and adds up all emitted elements to a sum.<p>However, this wouldn&#x27;t help you with collecting all input lines, as there is nothing for you function to &quot;wrap around&quot;. Or at least, there <i>used</i> to be nothing, but I think in one of the recent build, an &quot;inputs&quot; function was added, which emits all remaining inputs. So now, you can write e.g. &#x27;[., inputs]&#x27; to reimplement slurp. In the same way, you could sum up all input lines by writing &#x27;add_all(., inputs)&#x27;.<p>However, this is still ugly and unintuitive to write, so I think introducting some syntactic sugar for this would be useful. E.g., you could imagine a &quot;collect operator&quot;, e.g. &#x27;&gt;&gt;&#x27; which treats everything left of it as the first argument to the function to the right of it.<p>e.g., writing &#x27;a &gt;&gt; b&#x27; would desugar to &#x27;b(a)&#x27;.<p>Writing &#x27;a | b &gt;&gt; c&#x27; would desugar to &#x27;c(a | b)&#x27;.<p>Any steps further to the right are not affected:<p>&#x27;a | b &gt;&gt; c | d&#x27; would desugar to &#x27;c(a | b) | d&#x27;.<p>Scope to the left could be controlled with parantheses:<p>&#x27;a | (b &gt;&gt; c)&#x27; would desugar to &#x27;a | c(b)&#x27;.<p>To make this more useful for aggregating on input lines, you could add a special rule that, if the operator is used with no parantheses, it will implicitly prepend &#x27;(., inputs)&#x27; as the first step.<p>So if the entire top-level expression is &#x27;a | b &gt;&gt; c&#x27;, it would desugar to &#x27;c((., inputs) | a | b)&#x27;.<p>This would make many usecases that require keeping state much more straight-forward. E.g. collecting all the &quot;baz&quot; fields into an array could be written as &#x27;.baz &gt;&gt; []&#x27; which would desugar to &#x27;[(., inputs) | .baz]&#x27;<p>Summing up all the bazzes could be written as &#x27;.baz &gt;&gt; add_all&#x27; which would desugar to &#x27;add_all((., inputs) | .baz)&#x27;<p>...and so on.<p>On the other hand, this could also lead to new confusion, as you could also write stuff like &#x27;... | (.baz &gt;&gt; map) | ...&#x27; which would really mean &#x27;map(.baz)&#x27; or &#x27;foo &gt;&gt; bar &gt;&gt; baz&#x27; which would desugar to the extremely cryptic expression &#x27;baz((., inputs) | bar((., inputs) | foo))&#x27;. So I&#x27;m not quite sure.<p>Any thoughts about the idea?
评论 #31175197 未加载
marmada大约 3 年前
I see a lot of JQ experts on this thread, so I&#x27;ll bite the bullet here as a novice.<p>The purpose of life is not to know JQ. I just want to process the JSON so I can move on and do whatever is actually important. Ideally, I&#x27;d just be able to tell GPT-codex to do what I want to do to the JSON in English.<p>We&#x27;re not there yet, but in the meantime if there&#x27;s another tool that allows me to know less in exchange for doing more, I&#x27;ll gladly use it.
评论 #31173520 未加载
评论 #31172753 未加载
评论 #31173481 未加载
评论 #31173632 未加载
评论 #31173650 未加载
评论 #31173292 未加载
评论 #31174150 未加载
ctur大约 3 年前
It takes a while to get to the point so I’ll save others some time and tldr this very lengthy and agenda-driven blog post:<p>jq had a tough learning curve so you should switch to zq which is a (closed source?) wrapper around an obscure language you’ve never heard of that we promise is easier because reasons. Also coincidentally it’s the language of an ecosystem we were funded to build.<p>Edit: mea culpa, turns out you can download the source (revealed half way through the article).
评论 #31167398 未加载
评论 #31167473 未加载