I love awk, and I find myself reaching for it a fair bit. One of the main things I use it for is “sed with state,” so for things like matching on a line, but only if it was preceded by some other line. I find this to be really useful for creating one-off linters, for example I made one recently to check all our migration files for CREATE INDEX without CONCURRENTLY on a particular set of very large tables where it would cause issues. Since sql statements can be spread over multiple lines, it was difficult to write a straightforward match, but awk can track state like “I’m in a create statement,” “I’m creating an index,” etc. across multiple lines, which allowed me to cobble together something that has worked well for about a year now.
I suspect that anyone reading this thread is likely to be equally interested in "Ask HN: Share a shell script you like" from a fortnight ago (though at 78 comments, it didn't get as much traction / comments as I hoped it would when I saw it)<p><a href="https://news.ycombinator.com/item?id=37112991">https://news.ycombinator.com/item?id=37112991</a>
I maintain a minor side interest in Awk, along side Lisp and other things.<p>I developed cppawk in 2022: <a href="https://www.kylheku.com/cgit/cppawk/about/" rel="nofollow noreferrer">https://www.kylheku.com/cgit/cppawk/about/</a><p>cppawk extends Awk with preprocessing.<p>There is a loop macro that supports a vocabularly of clauses. Clauses can be combined for parallel and cross-product iteration. And they are user-extensible. By writing five simple macros, you can define a new clause.<p>Something potentially useful if you use Awk.<p>Cppawk is documented with multiple man pages, and covered by unit tests which run with gawk and mawk.
Perhaps my old sysadmin hat is showing through, but I don’t quite see what the advantage of awk is over just writing the same thing in perl. I’ve seen my fair share of horrendous shell scripts from junior sysadmins, and every time I think to myself “the text processing portion would be so much cleaner in Perl”.
One somewhat not-well-known thing with gawk is that it typically ships with some useful extensions that give you access to things like readdir(), ord(), chr(), gettimeofday(), sleep(), etc.<p><a href="https://www.gnu.org/software/gawk/manual/html_node/Extension-Samples.html" rel="nofollow noreferrer">https://www.gnu.org/software/gawk/manual/html_node/Extension...</a>
awk one-liners are a slam dunk. The tough question whether to invest in more complex awk programming. Invariably some processing task requires more complex logic and awk provides that, but in the terse and arcane ways of early computing. Yet reaching for a modern alternative is also an overhead, may not be particularly intuitive either (hello pandas) and may even have performance issues...
Hello! Author here.<p>I am pleased to announce a new version of my "CLI text processing with GNU awk" ebook.<p>Learn the `GNU awk` command step-by-step from beginner to advanced levels with hundreds of examples and exercises. This book will dive deep into field processing, show examples for filtering features, multiple file processing, how to construct solutions that depend on multiple records, how to compare records and fields between two or more files, how to identify duplicates while maintaining input order and so on. Regular Expressions will also be discussed in detail.<p>Links:<p>* PDF/EPUB versions: <a href="https://learnbyexample.gumroad.com/l/gnu_awk" rel="nofollow noreferrer">https://learnbyexample.gumroad.com/l/gnu_awk</a> (free till 31-August-2023)<p>* Web version: <a href="https://learnbyexample.github.io/learn_gnuawk/" rel="nofollow noreferrer">https://learnbyexample.github.io/learn_gnuawk/</a><p>* Markdown source, example files, etc: <a href="https://github.com/learnbyexample/learn_gnuawk">https://github.com/learnbyexample/learn_gnuawk</a><p>* Interactive TUI app for exercises: <a href="https://github.com/learnbyexample/TUI-apps/blob/main/AwkExercises">https://github.com/learnbyexample/TUI-apps/blob/main/AwkExer...</a><p>Bundle offers:<p>* Magical one-liners (<a href="https://learnbyexample.gumroad.com/l/oneliners/new_awk_release" rel="nofollow noreferrer">https://learnbyexample.gumroad.com/l/oneliners/new_awk_relea...</a>) is $5 (normal price $15) — grep, sed, awk, perl and ruby one-liners bundle<p>* All Books Bundle (<a href="https://learnbyexample.gumroad.com/l/all-books/new_awk_release" rel="nofollow noreferrer">https://learnbyexample.gumroad.com/l/all-books/new_awk_relea...</a>) is $12 (normal price $32) — all my 13 programming ebooks<p>I would highly appreciate it if you'd let me know how you felt about this book. It could be anything from a simple thank you, pointing out a typo, mistakes in code snippets, which aspects of the book worked for you (or didn't!) and so on. Reader feedback is essential and especially so for self-published authors. Happy learning :)<p>---<p>Previous discussions:<p>* Learn to use Awk with hundreds of examples (<a href="https://news.ycombinator.com/item?id=15549318">https://news.ycombinator.com/item?id=15549318</a>) — <i>478 points, Oct 2017, 116 comments</i><p>* Show HN: An eBook with hundreds of GNU Awk one-liners (<a href="https://news.ycombinator.com/item?id=22758217">https://news.ycombinator.com/item?id=22758217</a>) — <i>539 points, April 2020, 48 comments</i>
I did this golfing a while back: Drawing a heart with AWK - <a href="https://gist.github.com/auselen/906a53b47a7d616b080dbef85eb8f776" rel="nofollow noreferrer">https://gist.github.com/auselen/906a53b47a7d616b080dbef85eb8...</a>
99.9% of my awk use case is to split a line (a la "cut - d\ - f) while discarding successive spaces.<p>e.g.:<p><pre><code> $ echo "key: value" | awk '{print $1}'
value
</code></pre>
Open to a simpler replacement :-)
confession: plug<p>I once wrote a diff2html script ported from bash and it was much, much faster (for obvious reasons). And awk makes it much more readable than bash script. And I could learn the language, debug, understand bugs and fix them in a night.<p>Not sure, if it is idiomatic way to awk, but have to say it is a really nice language.<p><a href="https://github.com/berry-thawson/diff2html/blob/master/diff2html.sh">https://github.com/berry-thawson/diff2html/blob/master/diff2...</a>
A few years back I decided to just get as capable as I could with jq, which is fast and functional enough to cover 99% of awk/sed use cases, plus cases you'd never want to touch with awk/sed.<p>No regrets!
Of possible interest - instead of making a whole new programming language like awk, you can also just systematize generating code for an existing one with a command-line harness.<p>This can even stay terse & keep a fairly fast edit-test turnaround in a fully statically typed language like Nim: <a href="https://github.com/c-blake/bu/blob/main/doc/rp.md">https://github.com/c-blake/bu/blob/main/doc/rp.md</a>
Never learned awk or committed esoteric cli incantations to memory. Don’t get me wrong, I can get around on the cli, but sed, awk, etc just didn’t seem like a good cost/benefit investment. I’m also not a sysadmin.<p>Thankfully I waited long enough and LLMs can write them for me better than I ever could.
I have been using ChatGPT for generating these kind of small CLI like this. My prompts look like this:<p><pre><code> - use jq to count a nested array "a.b.c.d"
- find and delete empty folders using `find`
- find and replace text using sed/awk
</code></pre>
I found that using ChatGPT for these purposes boosted my productivity tremendously.
Awk is fine and dandy but, like wity Sed, I think that it's almost always replaceable with Perl which is way nicer to use, and ubiquitous. Every OS (except Windows) I laid my hands on in the last 15 years has had Perl installed in either its default install or pulled in as a dependency almost immediately (a LOT of stuff depends on Perl in any Unix system).<p>This is, unless you are running on an embedded environment, but in that case you are stuck with something like busybox's Awk which is way more limited than gawk...
I love awk. Enough to shill for this:<p><a href="https://www.oreilly.com/library/view/effective-awk-programming/9781491904930/" rel="nofollow noreferrer">https://www.oreilly.com/library/view/effective-awk-programmi...</a><p>If TFA is an excerpt for a book forthcoming on dead-tree media, then I'll be buying that one as well.