<p><pre><code> ls | grep '.csv$' | xargs cat | grep 'cake' | cut -d, -f2,3 > cakes.csv
</code></pre>
That's quite a few antipatterns in one go. Unless you have a bajillion files the `xargs` is unnecessary, the `cat` and `ls` are unnecessary (and `ls` in shell scripts is a whole class of antipatterns by itself). You might want to use something like this instead:<p><pre><code> grep cake *.csv | cut -d, -f2,3 > cakes.csv</code></pre>
For those interested in this topic I would suggest these incredible lectures by MIT [0], especially the data wrangling one.<p>Lectures are hosted on YouTube, they are extremely valuable and easy to follow and they give a pretty good insight on a lot of Unix topics.<p>[0]: <a href="https://missing.csail.mit.edu/2020/" rel="nofollow">https://missing.csail.mit.edu/2020/</a>
The 'comm' command should be in there. With no options 'comm' takes two files, F1 and F2, which should be lexically sorted, and produces 3 columns of output.<p>The first column consists of lines that are only in F1, the second column consist of lines that are only in F2, and the third column consists of lines that are common to both files.<p>The option -1 tells it to not print column 1, -2 tells it not to print column 2, and -3 does the same for column 3. These can be combined, so -12 would only print column 3 (the lines that are in both files) and -13 would only print column 2 (the lines that are in F2 but not F1).
"After some digging, it was easy to find the HTTP request that pulled this information from the server. And it even had all the birthdates in the JSON!"<p>HR needs to know this, but it shouldn't be available to random employees.
> regex is so ubiquitous and valuable that if you don’t know it yet, you should learn it)<p>Regex is one of those things I have to learn every single time I need to use it. I just can't seem to force myself to remember.
For doing work with JSON data, I'd add:<p><a href="https://stedolan.github.io/jq/" rel="nofollow">https://stedolan.github.io/jq/</a>
jq ... | sed -E 's/([0-9][0-9]).([0-9][0-9]).[0-9]*$/\2_\1/'<p>this fails for me since the jq output lines are surrounded by quotes. had to remove $. did i do something different or are we running different jq versions?
> Was it worth it?<p>> 1 minute to do this<p>> 1 minute to do that<p>and 1 minute to introduce RCE vulns into company #589179283672's pipeline due to the "you don't understand the security implications of using fragile UN*X tools" problem which applies to anyone actually learning something from this article DAY OF THE SEAL SOON,
Most of the justifications for using collections of command-line Unix tools are no longer valid today. Instead you should be using a proper programming language.<p>Note that people who still do use complex solutions built from cat, head, cut, etc, and who know what they're doing, will typically either write a shell script (which won't be structured particularly differently from the equivalent Python or whatever) or will rely heavily on awk (itself a full-featured programming language, no easier to learn than any other scripting language), or both.<p>One-liners which pipe text between four or five different commands are the equivalent of hand-soldered boards or bitwise arithmetic. Interesting to learn about for historical reasons but of no practical utility.<p>The use of things like xargs and jq in this solution, difficult to invoke Unix utilities for doing things that are trivial in any reasonable language, makes this even more clear.