While the idea that there can be a lot of small and composable tools sounds good in theory, in practice the implementation is very poor. To use tools in a script, they should have standartized machine-friendly input and output formats, but instead they have arbitrary and difficult to parse formats. It means that they are not intended for use in scripts.<p>- spaces or newlines in file names are often not handled properly. Some linux software based on shell scripts requires that a user install it to a path that has no spaces.<p>- the output format is often arbitrary and not machine-friendly. For example, `ls -l` output is difficult to parse properly. Some programs might output a file size with different suffixes, like 10k or 10M, that you have to parse.<p>- output format often depends on locale settings and scripts can break if used with different locale<p>So the programs that rely on line-oriented inputs/outputs like `grep`, `wc`, `ls` (without -l), `sort` work relatively good, but other programs that produce more complex output, are unusable for scripting.<p>Scripts often use ugly hacks like parsing output with `sed` or, even worse, `awk`, a 40 years old cryptic legacy tool. Instead they could use something like JSONpath, for example `parse $.*.size -- ls -l` to get sizes of all files (although I don't think that using JSON is a good idea; it's hard to read by human. I think something indentation-based like YAML would be better; however, YAML itself is overengineered and I recommend against it).<p>Also the shells are an awful choice for scripting. For example, there is no syntax to write a glob pattern to select all files including dot-files, but excluding pseudo-files like . and ..<p>To conclude, "Unix philosophy" of composing utilities using shell scripts is often advertised as a superior and innovative approach but in practice it is a bunch of inconvenient to use programs glued together with unrealiable buggy scripts written in outdated language. It is easier to use Python and ignore all this "philosophy".<p>By the way, `/proc` filesystem also is not scripting-friendly and not standartized. So it is usable only as a debugging tool for humans.
I’m currently reading <i>The Great Mental Models</i>. The part I’m reading now is called <i>The Map Is Not The Territory</i>, and it is about formulating abstractions.<p>In this chapter, the author reminds us that we run into problems when our knowledge is that of the map we’re reading, rather than that of the territory it describes.<p>I don’t think a network of micro-services is analogous to a chain of small UNIX programs piped together. Those aren’t pipes between micro-services; they’re web servers and network requests. It’s a whole different ball game.<p>The abstraction is an attractive one, and perhaps it’s why micro-services are all we’ve been able to talk about for the past decade. But it’s still not a good abstraction. It’s an oversimplification.
The implication goes way beyond code. Software writing is miniature organization design. Making clear interface is easier said than done. Zooming out a bit, a software team should be modular and composable. Zooming even more out, the whole company should be modular and composable. The whole cliche of "communicate more, collaborate more" is just a euphemism of bad interface design of organization. If you can get things done without communicating back and forth, who wants to spend 20 hours a day in meeting/brainstorm/catch-up?<p>Can we imagine having to email the author of `ls` to clarify the usage of flag `-l` in the name of breaking the silo? In reality it does happen but most of us just happily accept it or consult the documentation if needed. Now, can we imagine having to email the author of an internal library in your own company to clarify usage? I bet it happens all the time. Interface design is hard. Code-wise or org-wise.
Tiny tweaks and improvements (almost daily) to my Unix-related workflows makes a huge difference for me.<p>This is a good reminder for me to read The Art of Unix Programming.
TIL<p>manularity
[prob. fr. techspeak manual + granularity] A notional measure of the manual labor required for some task, particularly one of the sort that automation is supposed to eliminate.
Regarding the article, I think that "small" tools are better for scripting (but they should be made scripting-friendly; many Linux CLI tools are not), and complicated tools are better for usage by humans.<p>For example, take a word processor. You probably don't want to use one tool to type the text, then copy it into another tool for spell checking, into third tool to add equations, fourth tool to find and download clipart images, and so on. You want to have everything integrated.