TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Grep one-liners as CI tasks

111 点作者 fphilipe超过 3 年前

15 条评论

gravypod超过 3 年前
I think this is a great example of where build systems, that understand the need for testing, can help out. In a build system I use quite often (Bazel) you can express this as an `sh_test()` [0] which provides documentation about what you&#x27;re attempting to do <i>and</i> provides you with a way to reproduce the failure locally. You don&#x27;t have to push the code and wait for CI to fail to find out, or debug, this error.<p>Extra fun thing to do: print a message describing why the decision was made and how to resolve the error in the failure message of your test!<p>[0] - <a href="https:&#x2F;&#x2F;docs.bazel.build&#x2F;versions&#x2F;main&#x2F;be&#x2F;shell.html#sh_test" rel="nofollow">https:&#x2F;&#x2F;docs.bazel.build&#x2F;versions&#x2F;main&#x2F;be&#x2F;shell.html#sh_test</a>
评论 #29942960 未加载
评论 #29943915 未加载
cfors超过 3 年前
In today&#x27;s world of excellent CLI tools I don&#x27;t think grep is a good choice, especially for checking irregular languages like XML. [0]<p>I use tools like `jq` [1] or `yq` [2] all the time for CI checks. One useful check, is we have a configuration file stored as several hundred lines of YAML. Its a nice thing to maintain a sorted order for that, so we have a git pre-commit hook that runs the following:<p>&gt; yq eval --inplace &#x27;.my_key|= sort&#x27; my_file.yaml<p>Of course, a pre-commit hook or CI both work. There&#x27;s pros and cons of both. For our team, the pre-commit hook is a low enough level of effort, and doesn&#x27;t require a CI check for something that executes in milliseconds.<p>[0] <a href="https:&#x2F;&#x2F;stackoverflow.com&#x2F;a&#x2F;1732454" rel="nofollow">https:&#x2F;&#x2F;stackoverflow.com&#x2F;a&#x2F;1732454</a><p>[1] <a href="https:&#x2F;&#x2F;github.com&#x2F;stedolan&#x2F;jq" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;stedolan&#x2F;jq</a><p>[2] <a href="https:&#x2F;&#x2F;github.com&#x2F;mikefarah&#x2F;yq" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;mikefarah&#x2F;yq</a>
zX41ZdbW超过 3 年前
Here is my favorite: <a href="https:&#x2F;&#x2F;github.com&#x2F;ClickHouse&#x2F;ClickHouse&#x2F;blob&#x2F;master&#x2F;utils&#x2F;check-style&#x2F;check-style#L251" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;ClickHouse&#x2F;ClickHouse&#x2F;blob&#x2F;master&#x2F;utils&#x2F;c...</a><p>&quot;Too many exclamation marks&quot;
评论 #29941871 未加载
评论 #29942404 未加载
excircul超过 3 年前
&gt; I personally prefer ripgrep as it is much faster than grep, but usually that is not available on CI machines.<p>I recommend git grep, which is comparable in speed to ripgrep, since it ignores non-tracked files and searches the object storage directly. It is also able to run in parallel.
eminence32超过 3 年前
The semgrep[1] tool seems like the logical next step, for when you&#x27;ve outgrown plain ol&#x27; grep.<p>[1] <a href="https:&#x2F;&#x2F;semgrep.dev&#x2F;" rel="nofollow">https:&#x2F;&#x2F;semgrep.dev&#x2F;</a>
david_allison超过 3 年前
Android&#x27;s default linting already contains a &quot;missing translation&quot; lint rule which you can activate: &quot;MissingTranslation&quot;[0]<p>For Android specifically: Gradle and Android Studio both support a powerful linting framework (to the level that it can provide auto-fixes to the IDE). It&#x27;s better to provide an in-editor to guide your contributors before it hits CI, then have CI nag if they didn&#x27;t fix the in-editor warnings&#x2F;errors:<p>Some examples of custom lint rules[1] and the default rules which Android Studio runs[2]:<p>[0] <a href="https:&#x2F;&#x2F;android.googlesource.com&#x2F;platform&#x2F;tools&#x2F;base&#x2F;+&#x2F;32923407563b7e2c943cf372e3cd9d4398d09789&#x2F;lint&#x2F;libs&#x2F;lint-checks&#x2F;src&#x2F;main&#x2F;java&#x2F;com&#x2F;android&#x2F;tools&#x2F;lint&#x2F;checks&#x2F;TranslationDetector.java#79" rel="nofollow">https:&#x2F;&#x2F;android.googlesource.com&#x2F;platform&#x2F;tools&#x2F;base&#x2F;+&#x2F;32923...</a><p>[1] <a href="https:&#x2F;&#x2F;github.com&#x2F;ankidroid&#x2F;Anki-Android&#x2F;tree&#x2F;master&#x2F;lint-rules&#x2F;src&#x2F;main&#x2F;java&#x2F;com&#x2F;ichi2&#x2F;anki&#x2F;lint&#x2F;rules" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;ankidroid&#x2F;Anki-Android&#x2F;tree&#x2F;master&#x2F;lint-r...</a><p>[2] <a href="https:&#x2F;&#x2F;github.com&#x2F;ankidroid&#x2F;Anki-Android&#x2F;blob&#x2F;master&#x2F;lint-release.xml" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;ankidroid&#x2F;Anki-Android&#x2F;blob&#x2F;master&#x2F;lint-r...</a>
jrockway超过 3 年前
Don&#x27;t you worry about what happens when the translation software produces a semantically equivalent empty string, or something? Like `&lt;string&gt;&lt;![CDATA[]]&gt;&lt;&#x2F;string&gt;`. An XML parser will find that to be empty, grep will be like &quot;ooh lots of text in there&quot;.
chubot超过 3 年前
If you think of a CI configuration as a shell script, then this is normal and not surprising.<p>A CI config is just a big job that invokes a bunch of tools like &quot;go build&quot; or &quot;npm test&quot;, which is exactly what a shell script is too. I would get rid of the YAML and use shell for the whole thing :)<p>Shell does have some problems, like needing to be more parallel, incremental, and reproducible, but the YAML-based CI systems all have those problems too.<p>Related: <a href="http:&#x2F;&#x2F;www.oilshell.org&#x2F;blog&#x2F;2021&#x2F;04&#x2F;build-ci-comments.html#ci-services-as-a-bridge-between-shell-and-distributed-systems" rel="nofollow">http:&#x2F;&#x2F;www.oilshell.org&#x2F;blog&#x2F;2021&#x2F;04&#x2F;build-ci-comments.html#...</a> (and the entire post)
xorcist超过 3 年前
Wouldn&#x27;t quick sanity checks like these make more sense in a git push hook?<p>No reason to wait for the CI. There&#x27;s also the risk of broken intermediary commits unless the CI check each and every commit in isolation.
评论 #29942027 未加载
评论 #29942089 未加载
joatmon-snoo超过 3 年前
Shameless plug: if you find yourself wanting more advamced custom painting, you can try <a href="http:&#x2F;&#x2F;trunk.io" rel="nofollow">http:&#x2F;&#x2F;trunk.io</a>, which provides you with three different ways to write your own linters (pass&#x2F;fail based on your script&#x27;s exit code, spit out a patch that should be applied to your code, or LSP diagnostics) that can get propagated to VSCode and CI (as well as just as a CLI tool, if that&#x27;s your preference).<p>Disclaimer: I work on trunk :)
ilyash超过 3 年前
Interesting to see how nobody cares about exit code 2. Modify your grep command to have syntax error and you will always think you have all the translations.<p>Edit: ah.. forgot the whole point.. In Next Generation Shell (author here) this doesn&#x27;t happen: ngs -e &#x27;$(grep ...).not()&#x27; does right thing. grep exit code 0 becomes 1, 1 becomes 0, 2 becomes 240 (exception)
rtuin超过 3 年前
Oh the power of Linux.<p>I’m also using grep in CI as a simple but effectieve check on dependency violations in some codebases.
jeffbee超过 3 年前
grep and other shell tool one-liners also make excellent kubernetes liveness checks. I have some kubernetes daemonsets that don&#x27;t do anything other than assert that things on the node are as they should, via grep exit status.
评论 #29941940 未加载
matmann2001超过 3 年前
Wouldn&#x27;t this example do the incorrect thing in the event of a grep error (exit code 2)?
评论 #29941793 未加载
评论 #29941719 未加载
indigodaddy超过 3 年前
Learn something new everyday.. Never knew about —exclude&#x2F;include, very cool.