It seems many things can be done with a one-liner or short shell script, but I'm not fluent enough to recognize those patterns yet. I do know a few basic commands, but much of *nix seems like a bag of undocumented tricks; and man pages are hieroglyphic IMO. Is there a cohesive path to learning all the tricks? All of them!
Read the whole POSIX specification for the "Shell Command Language", and in particular how input is processed by the shell through all the multiple expansion passes.<p><a href="https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18" rel="nofollow">https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...</a><p>From there, you have a better background for reading the documentation of shell implementations, like Bash.<p>Also study Awk:<p><a href="https://pubs.opengroup.org/onlinepubs/9699919799/utilities/awk.html" rel="nofollow">https://pubs.opengroup.org/onlinepubs/9699919799/utilities/a...</a><p>Those who don't know Awk do comical things with cockamamie combinations of cut, sed, grep, echo ... and clumsy loops written in shell.<p>Other than shell syntax and semantics, it's a matter of knowing a large number of utilities.
If you encounter a problem or task, then do it using shell commands. This will push into asking the question "what command can do X?" and "how do I connect command X to command Y?" and "how do loop commands over output?". Then just keep doing that over and over again.<p>Learn the shell chainsaw: `xargs`. `xargs -L` vs `xargs` and `xargs -I` (examples will say use `{}` but I'm partial to `xargs -I @` because that works with Fish and Bash, plus it's a single character.
My way is:<p>Try to write small loops and pipes on the command line.
I do things like this all the time:<p><pre><code> for i in foo*.txt; do mv $i OLD-$i; done
</code></pre>
You'll find this breaks sometimes, so use control-r to recall the command and fix it<p><pre><code> for i in foo*.txt; do mv "$i" "OLD-$i"; done
</code></pre>
if it gets too long, copy/paste it into a file like ~/bin/renamer and multiple lines and indenting and make it a more permanent part of your collection.<p>Also, shell isn't the end. I move to python when shell scripts get cumbersome. python can easily parse arguments (argparse), handle filenames with weird characters in them (os.path), walk trees of files (os.walk) and more.
Best is to identify the shell as a solution within your daily work. The maybe obvious step 1 is to switch to an unixoid OS such as Linux or Mac OS. Step 2 is to make the shell easily reachable, for example with buttons to open a shell in your current directory or with a drop-down terminal like <a href="http://guake-project.org/" rel="nofollow">http://guake-project.org/</a><p>Then, whenever you have some everyday problem (frequently some "batch" or "bulk" job operating on many files), consider using the shell instead some 3rd party tools.<p>If you really are into it, consider working with suitable files. For instance, drop office files (MSO or LibreOffice/OpenDocument) and instead switch to plain text files (Markdown or Latex). They are much easier to process in the command line. Also consider using git instead of dropbox and friends, because it is CLI-first. There is a universe waiting to be explored: There are CLI clients for almost everything (chatting, mail, web, etc.).
I’d recommend the Advanced Bash Scripting Guide <a href="https://tldp.org/LDP/abs/html/" rel="nofollow">https://tldp.org/LDP/abs/html/</a> and make peace with the fact that you’ll never know it all and enjoy the journey. Years later you’ll still be stumbling across new stuff.
Unix evolved in an age of batch processing.<p>Files were the default datastore, and text streams the default inter-process interface.<p>Although the world is changing, a surprising amount of it still operates through those tools.<p>A good way to understand the tooling is to do one of the Linux from scratch or similar projects, perhaps look at swapping out some tools for self-written alternatives, and scratching your own itch building projects that solve problems you have yourself.<p>Most people who reach a degree of proficiency in shell also have an innate understanding of the kernel's general organization, the nature of filesystems, the boot process, database types, are capable of programming in multiple languages, comoprehend network programming and the challenges of locking and distributed state, and so on. You can't learn all of it in one go, so just leverage that interest and enthusiasm and devote the time.
Use it a lot and find ways to learn how it works:<p>- read the manual (see other posts)
- customize it (and persevere)
- make it fun<p>The second is really helpful. A lot of my shell commands/utilities [0] are simply better-for-me ergonomics on commonly available commands. These include things like `fzf` wrappers and more sophisticated aliases.<p>I specifically found the making of `fzf` based utilities to be a source of major learnings. It requires you to know quite a bit of shell mechanics to do things nicely. It's very similar to building a UI with APIs. You invariably learn the surrounding concepts (HTTP, auth for APIs, content types, caching, etc.)
Shameless plug:<p>I created a tutorial on Linux Terminal Tools a while ago where I covered many topics but not in too much detail. Perhaps going down the rabbit holes on these slides might help: <a href="https://ketancmaheshwari.github.io/pdfs/LPT_LISA.pdf" rel="nofollow">https://ketancmaheshwari.github.io/pdfs/LPT_LISA.pdf</a>
By reading 1 dedicated shell book e.g. "Linux Command Line and Shell Scripting" you'll be at the top 10% of shell connoisseurs.<p>That said, I challenge the usefulness of deep shell knowledge.<p>If you're planning on writing 300 lines shell scripts, you're most likely doing it wrong, you should be using a full featured, modern programming language.
Unix Power Tools was a great help when I was learning shell scripting:<p><a href="https://www.oreilly.com/library/view/unix-power-tools/0596003307/" rel="nofollow">https://www.oreilly.com/library/view/unix-power-tools/059600...</a>
I love the fact that shell gods exist :)<p>Honestly? If you can think of anything that needs to be automated, then write a shell script to automate it!
Two oreilly books to give you an overview, sorta know the lie of the land.<p>Classic Shell Scripting by Arnold Robbins<p>Unix Power Tools 3rd edition<p>Two additional worthwhile books:<p>The Unix Programming Environment by Kernighan and Pike<p>Mastering Regular Expressions by Friedl.<p>All 4 are classics.<p>Also try tldr.sh website in addition to man pages. They are easier to grok.
The bash manual is an okay place to start:<p><a href="https://www.gnu.org/software/bash/manual/" rel="nofollow">https://www.gnu.org/software/bash/manual/</a>