TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

JMESPath – A query language for JSON

219 pointsby selrondover 7 years ago

28 comments

justin_oaksover 7 years ago
I like JMESPath, but it has some serious limitations which prevent it from being as general purpose as jq.<p>JMESPath limitations:<p>- No simple if&#x2F;else, but it is possible using a hack, documented below.<p>- The to_number function doesn&#x27;t support boolean values, but it is possible using a hack, documented below.<p>- can&#x27;t reference parents when doing iteration. Why? All options for iteration, [* ] and map, all use the iterated item as the context for any expression. There&#x27;s no opportunity to get any other values in. May be possible for a fixed set of lengths. Something akin to the following (except there is no syntax for switching or if statements):<p><pre><code> switch (length): case 1: [expression[0]] case 2: [expression[1], expression[1]] case 3: [expression[0], expression[1], expression[2]] ... </code></pre> - Key name can&#x27;t come from an expression. Why? The ABNF for constructing key-value pairs is given as: keyval-expr = identifier &quot;:&quot; expression. The key is an identifier, which gives no possibility for making it an expression. No functions modify keys in such a way as to allow using an expression as a key.)<p>- No basic math operations, add, multiply, divide, mod, etc. Why? Nobody added those operators&#x2F;functions.<p>- There&#x27;s a join, but no split.<p>- No array indexing based on expression. Why? Indexing is done based on a number or a slice expression, which also doesn&#x27;t support expressions. Here&#x27;s the ABNF:<p><pre><code> bracket-specifier = &quot;[&quot; (number &#x2F; &quot;* &quot; &#x2F; slice-expression) &quot;]&quot; &#x2F; &quot;[]&quot; </code></pre> - No ability to group_by an expression.<p>- No ability to get the index of an element in a list<p>Hacks:<p>Convert true&#x2F;false to number:<p><pre><code> boolean_expression &amp;&amp; `1` || `0` </code></pre> If&#x2F;else:<p>Option 1)<p><pre><code> [{q:CONDITIONAL_EXPRESSION, v:IF_RESULT_EXPRESSION},{q:!COND_EXPRESSION,v:ELSE_RESULT_EXPRESSION}][?q]|[0].v </code></pre> Option 2)<p><pre><code> {&quot;if&quot;:CONDITIONAL_EXPRESSION, &quot;ctx&quot;:@} | [{q:if ,v:ctx.IF_EXPRESSION},{q:!if,v:ctx.ELSE_EXPRESSION}][?q]|[0].v</code></pre>
gingerlimeover 7 years ago
I love JMESPath. I first discovered it when using the AWS CLI `--query` option[0].<p>I then realized that using it in my code would make things much more declarative and easy to grok than a bunch of maps, filters etc. Here&#x27;s a real example which I think illustrates it[1]. It has libraries for lots of languages with a clear specification&#x2F;compliance test[2].<p>The cherry on the top is the interactive query on the website. You can tweak any of the examples (both queries and data) and get results instantly. Extremely useful for playing around, building queries to work with JSON data (webhooks, API responses etc)<p>[0] <a href="https:&#x2F;&#x2F;docs.aws.amazon.com&#x2F;cli&#x2F;latest&#x2F;userguide&#x2F;controlling-output.html" rel="nofollow">https:&#x2F;&#x2F;docs.aws.amazon.com&#x2F;cli&#x2F;latest&#x2F;userguide&#x2F;controlling...</a><p>[1] <a href="https:&#x2F;&#x2F;gist.github.com&#x2F;gingerlime&#x2F;757c7b4778c1ab68605dfce66ceb8378" rel="nofollow">https:&#x2F;&#x2F;gist.github.com&#x2F;gingerlime&#x2F;757c7b4778c1ab68605dfce66...</a><p>[2] <a href="http:&#x2F;&#x2F;jmespath.org&#x2F;libraries.html" rel="nofollow">http:&#x2F;&#x2F;jmespath.org&#x2F;libraries.html</a>
johnhenryover 7 years ago
Besides&#x27;s this projects cli, jp (<a href="https:&#x2F;&#x2F;github.com&#x2F;jmespath&#x2F;jp" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;jmespath&#x2F;jp</a>), I see jl (<a href="https:&#x2F;&#x2F;github.com&#x2F;chrisdone&#x2F;jl" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;chrisdone&#x2F;jl</a>) and jq (<a href="https:&#x2F;&#x2F;github.com&#x2F;stedolan&#x2F;jq&#x2F;" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;stedolan&#x2F;jq&#x2F;</a>) in the comments. I wonder if anyone has had experience with all three (or even just one) and can comment on their experiences?
评论 #16401181 未加载
评论 #16401456 未加载
评论 #16402344 未加载
评论 #16401140 未加载
评论 #16401824 未加载
评论 #16401193 未加载
评论 #16401618 未加载
评论 #16401336 未加载
xchaoticover 7 years ago
The reinvention of XML in JSON is almost complete - JMESPath vs XPath, JSON Schema vs XML Schema etc. If you need semi structured data to that level, consider using XML instead - you can validate it, there&#x27;s plenty of tools, it&#x27;s very stable and mature etc
评论 #16400646 未加载
评论 #16400561 未加载
评论 #16400992 未加载
评论 #16400886 未加载
评论 #16400914 未加载
评论 #16400607 未加载
评论 #16402676 未加载
评论 #16400864 未加载
评论 #16402410 未加载
评论 #16400793 未加载
评论 #16401851 未加载
评论 #16400709 未加载
评论 #16401051 未加载
评论 #16400573 未加载
评论 #16400678 未加载
crooked-vover 7 years ago
The example on the front page seems equivalent to (and only marginally less verbose than):<p><pre><code> locations .filter(l =&gt; l.state === &#x27;WA&#x27;) .map(l =&gt; l.name) .sort() .join(&#x27;, &#x27;)</code></pre>
评论 #16400571 未加载
评论 #16400509 未加载
评论 #16400510 未加载
zero_intpover 7 years ago
Thanks for taking the time to write this tool. Can you explain how it is distinguished from jq?
maltalexover 7 years ago
I want to add one more into the mix - Couchbase has something called &quot;N1QL&quot; (&#x27;nickel&#x27;), which is actual SQL adapted for JSON:<p><a href="https:&#x2F;&#x2F;www.couchbase.com&#x2F;products&#x2F;n1ql" rel="nofollow">https:&#x2F;&#x2F;www.couchbase.com&#x2F;products&#x2F;n1ql</a><p>It&#x27;s not standalone though, you need Couchbase to use it.
danbrucover 7 years ago
Does this have any mathematical foundation like the relational algebra for SQL? Or more generally, does a mathematical framework exist to treat this or similar constructs and that goes beyond what relational algebra provides and that, for example, also handles aggregate functions?<p>The reason I am asking is that I am currently trying to build a tool to analyze a kind of time series data, think log file entries, in order to look for anomalies and visualize them. I could of course just build all the transformations I am interested in in an ad hoc fashion but it would be nice to have a mathematical framework in order to start out with a small set of basic operations and then compose those while having some guarantees about the expressiveness of that the basic operations and ideally also a rigorous foundation for transforming them, for example for performance optimizations.<p>But so far I was unable to find something that seems fitting, everything I am aware of is either to limited like relational algebra or way to general like general functions. It feels like what I am looking for should exist but I am unable to find it.
评论 #16400954 未加载
评论 #16400908 未加载
jjuhlover 7 years ago
See also <a href="https:&#x2F;&#x2F;stedolan.github.io&#x2F;jq&#x2F;" rel="nofollow">https:&#x2F;&#x2F;stedolan.github.io&#x2F;jq&#x2F;</a>
woobyover 7 years ago
A worthwhile alternative to this approach (a JSON-specific query language) is a language for converting JSON structures to newline-delimited records. Then, standard shell tools can be used to query and join: <a href="https:&#x2F;&#x2F;github.com&#x2F;micha&#x2F;json-table" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;micha&#x2F;json-table</a>
rozzieover 7 years ago
For those interested in arbitrarily transforming JSON objects (for example, in a communications pipeline) I’d recommend JSONata. It’s quite useful and we’re well along in a Golang port with $function extensibility. <a href="http:&#x2F;&#x2F;jsonata.org" rel="nofollow">http:&#x2F;&#x2F;jsonata.org</a>
评论 #16402955 未加载
dmorenoover 7 years ago
There is also JSON Pointer (<a href="https:&#x2F;&#x2F;tools.ietf.org&#x2F;html&#x2F;rfc6901" rel="nofollow">https:&#x2F;&#x2F;tools.ietf.org&#x2F;html&#x2F;rfc6901</a>) from IETF.<p>Very simple standard and easy to implement, but not as powerful as jmespath nor jq.
评论 #16400715 未加载
nimishover 7 years ago
jq exists, is fast, and works well. Is this compatible?
评论 #16401271 未加载
评论 #16401194 未加载
maxsavinover 7 years ago
This looks interesting - but doesn&#x27;t MongoDB basically achieve the same effect? I kind of prefer MongoDB because you query JSON with JSON - but I&#x27;m open to changing my mind :)
评论 #16400498 未加载
avoidworkover 7 years ago
why choose this over jsonpath (like xmlpath) or jq?
评论 #16401196 未加载
nurettinover 7 years ago
I already have a query language for json. I insert json into mssql 2017 community edition and query it there. <a href="https:&#x2F;&#x2F;docs.microsoft.com&#x2F;en-us&#x2F;sql&#x2F;relational-databases&#x2F;json&#x2F;json-data-sql-server" rel="nofollow">https:&#x2F;&#x2F;docs.microsoft.com&#x2F;en-us&#x2F;sql&#x2F;relational-databases&#x2F;js...</a>
syatsover 7 years ago
One more alternative to many listed here is SPARQL-Generate. A single query language that works for XML and JSON, and has syntax borrowed from SPARQL.<p><a href="https:&#x2F;&#x2F;ci.mines-stetienne.fr&#x2F;sparql-generate&#x2F;playground.html" rel="nofollow">https:&#x2F;&#x2F;ci.mines-stetienne.fr&#x2F;sparql-generate&#x2F;playground.htm...</a>
xonixover 7 years ago
My tiny lib with very similar functionality: [1]. The query syntax is slightly different though. Also I decided to re-use JS for evaluation of sub-expressions instead of implementing own full-fledged parser.<p>[1] <a href="https:&#x2F;&#x2F;github.com&#x2F;xonixx&#x2F;jsqry" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;xonixx&#x2F;jsqry</a>
评论 #16401515 未加载
octrefover 7 years ago
I built this VS Code plugin to convert JSON interactively using JMESPath: <a href="https:&#x2F;&#x2F;marketplace.visualstudio.com&#x2F;items?itemName=octref.vscode-json-transform" rel="nofollow">https:&#x2F;&#x2F;marketplace.visualstudio.com&#x2F;items?itemName=octref.v...</a><p>Might be useful if you are testing API or playing with JSON data.
评论 #16401455 未加载
danvkover 7 years ago
I&#x27;m sad that JSONSelect (<a href="https:&#x2F;&#x2F;github.com&#x2F;lloyd&#x2F;JSONSelect" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;lloyd&#x2F;JSONSelect</a>) never caught on. It uses CSS selectors to query JSON, which has the nice side effect that learning to use it improves your CSS as well!
jnordwickover 7 years ago
I remember when xml started down this road too. &quot;We aren&#x27;t going to be sgml, but just a lightweight markup. Xpath, xslt, etc. And now we have xml today, the modern day sgml.<p>But this time it&#x27;s different?
hoppelhaseover 7 years ago
There is also JsonPath.<p><a href="http:&#x2F;&#x2F;goessner.net&#x2F;articles&#x2F;JsonPath&#x2F;" rel="nofollow">http:&#x2F;&#x2F;goessner.net&#x2F;articles&#x2F;JsonPath&#x2F;</a>
ex3ndrover 7 years ago
Does someone knows nice human-friendly search language like the one in Jira, Slack, etc?
thrownaway954over 7 years ago
the second I looked at the example on the homepage and saw this:<p>sort(@)<p>I&#x27;m like nope! What is this &quot;@&quot; symbol? Why can&#x27;t that be &quot;name&quot;? I&#x27;m already passing judgment that this library will be a nightmare to use which isn&#x27;t good.<p>Now I know I can read the docs and eventually what I can pass the sort expression and what it all means, however, this is an issue I come across more and more with new libraries in programming... show simple examples, not &quot;smart&quot; or complicated ones. I shouldn&#x27;t have to read through docs to try to decipher an introductory example. There is a reason every programming language starts with &quot;Hello World&quot;.
评论 #16400894 未加载
评论 #16400910 未加载
blattimwindover 7 years ago
I&#x27;m rather disappointed its not called JPath.
评论 #16400570 未加载
medleybronover 7 years ago
this, combined with GraphQL would be awesome
adamretterover 7 years ago
So no mention of JSONiq which came before JMESPath?
eliover 7 years ago
May also be interested in the `jq` CLI, which on first glance appears to use a similar but not identical query language. <a href="https:&#x2F;&#x2F;stedolan.github.io&#x2F;jq&#x2F;" rel="nofollow">https:&#x2F;&#x2F;stedolan.github.io&#x2F;jq&#x2F;</a>
评论 #16400700 未加载
评论 #16400551 未加载
评论 #16400621 未加载
评论 #16400735 未加载