<i>“One thing well” misses the point: it should be “One thing well AND COMPOSES WELL”</i><p>If the implementation isn't respecting <i>The Rule of Composition</i> it's actually not adhering to the Unix philosophy in the first place. The tweet is referring to one of Doug McIlroy's (one of the Unix founders, inventor of the Unix pipe) famous quotes:<p><i>"This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface."</i><p>Pure beauty, but it's almost too concise a definition if you haven't <i>experienced the culture</i> of Unix (many years of usage / reading code / writing code / communication with other followers).
ESR's exhaustive list of Unix rules in plain English might be a better start for the uninitiated (among which one will find the aforementioned <i>Rule of Composition</i>).<p>For all those seeking enlightenment, go forth and read the <i>The Art of Unix Programming</i>:<p><a href="https://en.wikipedia.org/wiki/The_Art_of_Unix_Programming" rel="nofollow">https://en.wikipedia.org/wiki/The_Art_of_Unix_Programming</a><p>17 Unix Rules:<p><a href="https://en.wikipedia.org/wiki/Unix_philosophy#Eric_Raymond.E2.80.99s_17_Unix_Rules" rel="nofollow">https://en.wikipedia.org/wiki/Unix_philosophy#Eric_Raymond.E...</a>
Here's one more tip: did you ever notice that "ls" displays multiple columns, but "ls | cat" prints only one filename per line? Or how "ps -f" truncates long lines instead of wrapping, while "ps -f | cat" lets the long lines live?<p>You can do it too, and if you're serious about writing Unix-style filter programs, you will someday need to. How do you know which format to write? Call "isatty(STDOUT_FILENO)" in C or C++, "sys.stdout.isatty()" in Python, etc. This returns true if stdout is a terminal, in which case you can provide pretty output for humans and machine-readable output for programs, automatically.
I'm not sure I agree with the "no JSON, please" remark. If I'm parsing normal *nix output I'm going to have to use sed, grep, awk, cut or whatever and the invocation is probably going to be different for each tool.<p>If it's JSON and I know what object I want, I just have to pipe to something like jq [1].<p>PowerShell takes this further and uses the concept of passing objects around - so I can do things like ls | $_.Name and extract a list of file names (or paths, or extensions etc)<p>[1]: <a href="http://stedolan.github.io/jq/" rel="nofollow">http://stedolan.github.io/jq/</a>
A nitpicky tip: --help is normal execution, not an error, so the usage information should be printed to stdout, not stderr (and it should exit with a successful status). Nothing is more annoying than trying to use a convoluted program with a million flags (which should have a man page in the first place) and piping --help into less with no success.
1978 called. It wants its pipes back.<p>That approach dates from the days when you got multi-column directory listings with<p><pre><code> ls | mc
</code></pre>
Putting multi-column output code in "ls" wasn't consistent with the UNIX philosophy.<p>There's a property of UNIX program interconnection that almost nobody thinks about. You can feed named environment variables into a program, but you can't get them back out when the program exits. This is a lack. "exit()" should have taken an optional list of name/value pairs as an argument, and the calling program (probably a shell) should have been able to use them. With that, calling programs would be more like calling subroutines.<p>PowerShell does something like that.
Additional tip: if writing a tool that prints a list of file names, provide a -0 option that prints them separated by '\x0' rather than white space. Then the output can be piped through xargs -0 and it won't go wrong if there are files with spaces in their paths.<p>I suggest -0 for symmetry with xargs. find calls it -print0, I think.<p>(In my view, this is poor design on xargs's part; it should be reading a newline-separated list of unescaped file names, as produced by many versions of ls (when stdout isn't a tty) and find -print, and doing the escaping itself (or making up its own argv for the child process, or whatever it does). But it's too late to fix now I suppose.)
Great article. The other thing I've always wished for command-line tools is some kind of consistency for flags and arguments. Kind of like a HIG for the command line. I know some distros have something like this, and that it's not practical to do as many common commands evolved decades ago and changing the interface would break pretty much everything. But things like `grep -E,--extended-regexp` vs `sed -r,--regexp-extended` and `dd if=/a/b/c` (no dashes) drive me nuts.<p>In a magical dream world I'd start a distro where every command has its interface rewritten to conform to a command line HIG. Single-letter flags would always mean only one thing, common long flags would be consistent, and no new tools would be added to the distro until they conformed. But at this point everyone's used to (and more importantly, the entire system relies on) the weird mismatches and historical leftovers from older commands. Too bad!
Lots of great points here, but as always, these can be taken too far. Header lines are really useful for human-readable output, and can be easily skipped with an optional flag. (-H is common for this).<p>The "portable output" thing is especially subjective. I buy that it probably makes sense for compilers to print full paths. But it's nice that tools like ls(1) and find(1) use paths in the same form you gave them on the command-line (i.e., absolute pathnames in output if given absolute paths, but relative pathnames if given relative paths). For one, it means that when you provide instructions to someone (e.g., a command to run on a cloned git repo), and you want to include sample output, the output matches exactly what they'd see. Similarly, it makes it easier to write test suites that check for expected stdout contents. And if you want absolute paths in the output, you can specify the input that way.
Not every program will be able to take input in stdin and output to stdout. If you have a --file (or -f) option, you'd do well to support a "-" file argument, which means either stdin or stdout, depending if you're reading or writing to -f. But you won't support "-" if the -f option requires seeking backwards in a file. Neither will you be using stdin or stdout if binary is involved (because tty drivers).<p>'One thing well' is often intended to make people's lives easier on the console. Sometimes this means assuming sane defaults, and sometimes just a simpler program that does/assumes less. Take these two examples and tell me which you'd prefer to type:<p><pre><code> user@host~$ ls *.wav | xargs processAudio -e mu-law --endian swap -c 2 -r 16000
user@host~$ find . -type f -maxdepth 1 -name '*.wav' -exec processAudio -e mu-law --endian swap -c 2 -r 16000 {} \;
</code></pre>
Write concise technical documentation. Imagine it's your first day on a new job and you need to learn how all your new team's tools work; do you want to read every line of code they've written just to find out how it works, or do you want to read a couple pages of technical docs to understand in general how it works? (That's a rhetorical question)<p>Definitely provide a verbose mode. When your program doesn't work as expected, the user should be able to figure it out without spending hours debugging it.
I think it's insane to restrict programs to just STDOUT & STDERR. Why 2? Why not use another file descriptor, maybe STDFMT, to capture all the formatting markup? This would avoid -0 options (newlines are markup sent to stdfmt, all strings on stdout are 0-terminated), it would avoid -H options (headers go straight to STDFMT), it would allow for less -R to still work, etc.<p>It's possible other descriptors would be useful, like stdlog for insecure local logs, stddebug for sending gobs of information to a debugger. It's certainly not in POSIX, so too bad, but honestly stdout is hard to keep readable and pipe-able. Adding just one more file descriptor separates the model from the view.
I agree with what is exposed on the article and I've actually added more details in how to apply this "principles" to shell scripting:<p><a href="http://javier.io/blog/en/2014/10/21/hints-in-writing-unix-tools-with-shell-scripting.html" rel="nofollow">http://javier.io/blog/en/2014/10/21/hints-in-writing-unix-to...</a>
I would add to this list:<p>If you are intercepting UNIX signals (starting with SIGINT), go back to the drawing board and think again. Don't do it. There is almost never a good reason for doing it, and you will likely get it wrong and frustrate users.