TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Show HN: LogsQL – opinionated query language for logs

61 点作者 valyala大约 1 年前
I don&#x27;t like the existing query languages for Elasticsearch and Grafana Loki, because they are too awkward to use for typical logs&#x27; investigation cases. So I designed new query language - LogsQL - and wrote reference implementation for it as a part of VictoriaLogs - an open source database for logs. LogsQL is based on the following principles:<p>- Simplicity. It is easy to write typical queries over logs in it. For example, a single word `error` is a valid LogsQL query, which returns all the logs with the `error` word. Another example is `_time:5m error`, which returns all the logs with the &#x27;error&#x27; word over the last 5 minutes.<p>- Composable building blocks similar to Unix pipes, which allow powerful filtering, transforming and calculating stats over the selected logs. For example, `_time:5m error | stats count() as rows` returns the number of logs with the `error` word over the last 5 minutes.<p>- Readability. Typical LogsQL queries must be easy to read and understand even for persons unfamiliar with it.<p>Take a look at LogsQL docs [1] and try using VictoriaLogs [2] in production. If you like Unix way and KISS design principle, then you&#x27;ll enjoy LogsQL :)<p>[1] <a href="https:&#x2F;&#x2F;docs.victoriametrics.com&#x2F;victorialogs&#x2F;logsql&#x2F;" rel="nofollow">https:&#x2F;&#x2F;docs.victoriametrics.com&#x2F;victorialogs&#x2F;logsql&#x2F;</a><p>[2] <a href="https:&#x2F;&#x2F;docs.victoriametrics.com&#x2F;victorialogs&#x2F;" rel="nofollow">https:&#x2F;&#x2F;docs.victoriametrics.com&#x2F;victorialogs&#x2F;</a>

6 条评论

rixrax12 个月前
Am I the only one that feels that EVERYTHING is wrong in this ELK, Splunk, etc. Grafana world? The user interfaces that these monstrosities present us with are barely useable, everyone has their own query language, they force us to install their own agents own our hosts and servers, when I upload logs, many can&#x27;t even take random JSON logs and input them in a structured way without defining pipeline rules or what now. And did I say that the Logstashes and Promtails and Vectors and what not pipeline tools with their Grok etc. filters feel like somebody wanted to really make busywork cool.<p>I am happy that in my day to day work I can dump my mostly Linux logs to rsyslog, and eventually forward them to S3 glacier for a few years.<p>So I am guessing the question I am asking is that what in the world are you doing with these observability or SIEM platforms and is anyone actually deriving some REAL value from using them?
评论 #40478508 未加载
评论 #40477713 未加载
评论 #40477249 未加载
评论 #40480502 未加载
评论 #40477725 未加载
评论 #40478113 未加载
BillFranklin12 个月前
Interesting to see a new approach!<p>You wrote that you don&#x27;t like Loki&#x27;s LogQL, but it looks quite similar (Victoria&#x27;s LogQL first):<p><pre><code> log.level:error _stream:{app!~&quot;buggy_app|foobar&quot;} {app!~&quot;buggy_app|foobar&quot;} | &quot;log.level:error&quot; </code></pre> The pipes are arguably a bit noisy in Loki queries (compared to spaces in Victoria&#x27;s), but I find they do make the queries a bit more readable, and it&#x27;s easier to understand under the hood how the queriers will handle the query, and coming from PromQL I found Loki&#x27;s approach quite intuitive: <a href="https:&#x2F;&#x2F;grafana.com&#x2F;docs&#x2F;loki&#x2F;latest&#x2F;query&#x2F;" rel="nofollow">https:&#x2F;&#x2F;grafana.com&#x2F;docs&#x2F;loki&#x2F;latest&#x2F;query&#x2F;</a>.<p>Maybe I missed something fundamental though, interested to hear more about the differences, since I only read the couple links you shared!
评论 #40476580 未加载
评论 #40486803 未加载
评论 #40478123 未加载
bdcravens12 个月前
You have to feed your logs into the VictoriaLogs database in order to use LogsQL, right?<p>&quot;LogsQL is a simple yet powerful query language for VictoriaLogs.&quot;
评论 #40486975 未加载
ople12 个月前
Recently there seems to be an bunch of SPL (Splunk) -like query languages popping up: PQL, PRQL, Grafana Explore Logs syntax, Kusto.. Probably others as well. Does yet another similar but slightly different language make sense? Why not leverage an existing one?
评论 #40486958 未加载
piterrro12 个月前
I&#x27;m a <a href="https:&#x2F;&#x2F;logdy.dev" rel="nofollow">https:&#x2F;&#x2F;logdy.dev</a> (logs to UI interface) author and been recently thinking about how to enable users use a query language to search throught logs beyond usual filter. I was looking at LogsQL but then I felt that is just another QL a user will need to learn. My next though was on SQL, but it was not designed for this purpose. Any ideas? I would appreciate any recommendation (peter at logd.dev)<p>[1] <a href="https:&#x2F;&#x2F;github.com&#x2F;logdyhq&#x2F;logdy-core">https:&#x2F;&#x2F;github.com&#x2F;logdyhq&#x2F;logdy-core</a>
评论 #40487000 未加载
dangoodmanUT12 个月前
IMO SQL is great for logs, ClickHouse and DuckDB SQL in particular are awesome
评论 #40477559 未加载