Make sure to have ShellCheck either integrated in your editor or run it before executing.<p>It really tells many of the rules you're supposed to abide by and helps you write cleaner shell script.<p><a href="https://github.com/koalaman/shellcheck" rel="nofollow">https://github.com/koalaman/shellcheck</a>
This is the kind of ignorance (just use python for anything it's better + and discover subprocess(): it's great) that burns me up. Not so many years ago actively insisting that python/tcl/perl were to be used instead of a 50 line shell script would have gotten you kicked out of a serious discussion. You used the power and reflection of these languages when you needed something that didn't integrate cleanly with the rest of the unix toolkit and/or required sophisticated data structures, controls and abstraction. These days I won't even write python or tcl for trivial projects (including http API work just use curl/jq and bash/(g)awk). The number of shell one liners that just get things done in 30 seconds (rather than loading 5 libraries in python to write the same code) are innumerable. This is google thought policing the historically ignorant yet again. They are terribly opposed to the unix toolkit and C pragma that prevailed for 20 years...because they believe that they know the best way to make the world safe and productive. Self serving, smug rejections that serve the bottom line and coincidentally gain mind share for their products and approaches.
I really wonder whether there is really any point in writing shell scripts anymore. Practically every Unix/Linux box in existence has at least some version of Python 2 that can be used as a complete and total replacement. I can't think of a single situation where I would need a shell script and a Python script wouldn't be much cleaner, simpler, and more maintainable.
Shell scripts have their well-deserved place in Unix systems. A general recommendation to use Python or other high-level scripting languages without a discussion about the reasons why and when to use shell may lead to wrong conclusions.
I'm not sure I'm comfortable with python for a typical shell script. But then again, what alternatives exist?<p>I've been thinking that we are kind of stuck with it. Much like javascript.<p>So, we could go the route of typescript and have a translation layer and outputting shell scripts. I actually think that could work quite well from a technical standpoint.<p>A big part of shellscripts though is having the source available. So maybe even have the translated, original code, and the generated shell output in the same file.<p>The source code on top (out of view from a shell interpreter) and the generated code below. This would allow it to be readable by anyone and executable by anyone, but modifying it would require the transpiler (which by itself would probably be pretty lightweight as well - certainly compared to python).<p>Certainly not perfect, but preferable to rediscovering the nuances and gotchas of shell-scripts every other day. Or maybe such a solution would only prolong the suffering and we should start all over.
Some time ago I started to doubt if using 'set -e' is a good idea.<p>I mean, if it would work as you expect it to, it certainly is a good idea to exit a script as soon as something fails. But sadly not all implementations behave similarly [1] and if you call a function from within a condition, 'set -e' gets deactivated/doesn't work.<p>For illustration, take a look at the following example:<p><pre><code> #!/bin/bash
foo() {
set -e
false
echo "Sucks"
}
printf 'Normal: %s\n' "$(foo)"
printf 'Condition: %s\n' "$(foo || true)"
</code></pre>
Output:<p><pre><code> Normal:
Condition: Sucks
</code></pre>
Probably not what you would have expected. Not using 'set -e' is no good solution either, so, for the time being, I still use it, but I am still looking for a better solution.<p>[1] <a href="https://www.in-ulm.de/~mascheck/various/set-e/" rel="nofollow">https://www.in-ulm.de/~mascheck/various/set-e/</a>
Can mktemp fail? I recently wrote a script that ensures that the directory created by mktemp actually exists and starts with /tmp so that when the script wipes out its temporary data at the end, it will not ever run something like "rm -rf /". Using fully-qualified paths for everything is also (not) fun.<p>In another script I was leery about running rm -f -- /path/foo* so I instead used<p><pre><code> find /path -mindepth 1 -maxdepth 1 -name "foo*" -delete
</code></pre>
because then the glob can't affect find's behaviour in weird ways.<p>Most likely set -e is enough protection against mktemp failing and the rm -f glob <i>probably</i> would've been just fine, but shell scripting has enough footguns that sometimes I can't help but go overboard with paranoia.
I’m constantly surprised that we got a safe language that compiles to Javascript (TypeScript) but never got a safe language that compiles to shell script. Why can’t I write in (a subset of) some other scripting language $lang, but with restricted-to-pure-/bin/sh semantics, and then cross-compile it to actual portable shell script for distribution?<p>Hopefully not by generating a megabyte of polyfill runtime code; more just by the compiler refusing to compile code in $lang unless it has a direct equivalent in /bin/sh. Any $lang source file the compiler accepted, would just look like “a shell script translated into $lang” already. But the compiler would inject $lang’s safe semantics (exceptions, type-checking, etc.) during the compilation, so you’d at least get that benefit.<p>Alternately, I’d <i>also</i> be satisfied with “emscripten for shell”: a compiler that generated shell scripts containing a small WASM-like emulator and an embedded bytecode stream to feed it. As long as it was lightweight enough. (A large point of these environments’ use of /bin/sh is that they’re small and embedded and can’t load too much into memory at once.)<p>My only guess for why this hasn’t happened, is that the people who write things like shell scripts that execute in initramfs, or shell scripts that are intended to install stuff even on lesser-known UNIXes, are all old-hand ops people who know shell-scripting cold, and certainly <i>aren’t</i> developers with any experience in compiler theory.
Checking $PATH or explicitly calling outside resources is probably also advisible.<p>In general, this article seems very light. No mention of LD_LIBRARY_PATH, for example. I imagine there's probably a better guide to writing secure scripts somewhere else.
For more concrete recommendations in a similar spirit, check out <a href="https://github.com/pesterhazy/blissful-bash" rel="nofollow">https://github.com/pesterhazy/blissful-bash</a>
Instead of<p><pre><code> set -euf -o pipefail
</code></pre>
please consider<p><pre><code> set -o errexit
set -o nounset
set -o noglob
set -o pipefail
</code></pre>
which is easier to lookup/search for if the reader is less familiar with shell scripting, and makes for cleaner diffs when a flag is removed/added.<p>Caveat: some of the longer forms might be Bashisms (e.g. not present in ksh, dash, etc.)
I find exceptionally difficult to write anything but trivial shell scripts without bugs. This one took years, and I suspect #bash could still find a bug: <a href="https://github.com/jakeogh/commandlock" rel="nofollow">https://github.com/jakeogh/commandlock</a><p>On the other hand, this is amazing: <a href="https://github.com/speed47/spectre-meltdown-checker" rel="nofollow">https://github.com/speed47/spectre-meltdown-checker</a>
Hi. Author of Next Generation Shell here.<p>Here is my take on bash vs Python and friends. I am frustrated by both.<p>bash - domain specific language and huge library (CLI utilities) and lets you get the job done but not a language I would like to use (syntax, error handling, very limited structured data support).<p>Python - okayish as a language but doing domain specific things (working with files and processes) is so much more verbose than bash.<p>My solution in NGS - have a <i>high level, modern</i> language with the typical goodies like ... exceptions (wow, completely new concept!) and the language is still domain specific. Working with processes for example has it's own syntax and is much more concise and straightforward (stole bits from bash).<p>Right now the project has the language, which is useful enough to write scripts (which we do at work). Regarding contribution to the project, my idea is that as much as possible should be in NGS language (as opposed to the lower level C). The UI will be implemented completely in NGS, allowing contributing to the project using the <i>same language</i> you are writing your scripts in.<p><a href="https://github.com/ngs-lang/ngs" rel="nofollow">https://github.com/ngs-lang/ngs</a><p>As a side note, while the UI is not there yet, I do plans for it which include interaction with objects on the screen as opposed to current "here is your dump of text" situation... and in general be more considerate of the user.<p><a href="https://github.com/ngs-lang/ngs/wiki/UI-Design" rel="nofollow">https://github.com/ngs-lang/ngs/wiki/UI-Design</a>
Here's my boilerplate bash header with comments describing each option, which I find practically helpful for starting new scripts<p><a href="https://gist.github.com/crizCraig/f42bc250754bed764ada5f95d101dbea/" rel="nofollow">https://gist.github.com/crizCraig/f42bc250754bed764ada5f95d1...</a><p>raw: <a href="https://gist.githubusercontent.com/crizCraig/f42bc250754bed764ada5f95d101dbea/raw/e3814fdb85c42afcf5766abf039f51eed67d8068/bash_boilerplate.sh" rel="nofollow">https://gist.githubusercontent.com/crizCraig/f42bc250754bed7...</a>
I personally think "don't" is right. For those who are interested in trying to use Python instead, I found this to be a helpful resource: <a href="https://github.com/ninjaaron/replacing-bash-scripting-with-python" rel="nofollow">https://github.com/ninjaaron/replacing-bash-scripting-with-p...</a>
I've done a bit of porting bash scripts to posix recently and I've found the following boilerplate pretty useful:<p><pre><code> case "$(readlink /proc/$$/exe)" in */bash) set -euo pipefail ;; *) set -eu ;; esac
</code></pre>
That takes care of setting -eu for shells that don't support -o pipefail and pipefail for bash.
More useful tips in the same spirit<p><a href="http://robertmuth.blogspot.com/2012/08/better-bash-scripting-in-15-minutes.html" rel="nofollow">http://robertmuth.blogspot.com/2012/08/better-bash-scripting...</a>
Since it's bash-specific, I think people should use zsh instead. No need to quote variables as by default, splitting is not done. And you get access to arrays and associative arrays.
On set -e:<p>This is like adding a setting to your Java or Python program to immediately exit if any method call throws any kind of exception, with no possibility of catching and handling that exception.<p>This does not seem to be a smart thing to do as compared to checking return codes, etc.<p>Every script I've seen with set -e has been buggier than without it.<p>The main problem with scripts is that they often do not go through a normal software review and testing process.
The first piece of advice, "Don't", is the best. "shellcheck myscript.sh" should return a fatal error if the shell script file you're checking exists, and "shellcheck -f myscript.sh" should fix your shell script by removing it.
At the beginning of each file:<p><pre><code> #!/bin/bash
set -euf -o pipefail
cd $(dirname $0)
</code></pre>
Then add "" everywhere ;) And they say, write Python instead, except I’m dubious because python programs can have a lot of dependencies which can be tricky to install.
Not to be that guy but this is mostly garbage. I suggest simply paying attention and testing. Switches can be a useful part of the tool but they can break things. It doesn't make things better rather it alters behaviors of the shell in sometimes unintended ways. Try some loops with some -eo. If you have a pipe that fails your script you're just ripping the heart and potential out of your script. I think you might just be looking for one-liners. It can be good to catch your failures and try other things vs "woop, damn, nope." Maybe you need things to fail first.<p>"Quote liberally" - also ripe. Mind your aPostrophes and Quotes because they do things.<p>I like writing for me and what I need done, not what or how others might have me write or how others might think it should be done.<p>The shell is a base orchestration and interface tool for your os. If you think it's just that bad then just stick to your language of choice but please refrain from suggesting lazy habits make a better shell, they don't.<p>Fly your own kite. It's much more fun.