Unix is not an acceptable Unix

Interesting examples, especially in light of the recent debates about Unix shells and PowerShell.<p>In PowerShell, the `ls` command <i>is</i> a simple command that produces output without flags to control output format, ordering or filtering. Like with higher order functional programming, those tasks are delegated to cmdlets like Format-List, Format-Table, Select-Object, Where-Object.<p>Some PowerShell examples:<p>To list the files and directories (children) of a directory:<p><pre><code> ls </code></pre> To get just the names or the full path:<p><pre><code> ls | select Name ls | select FullName </code></pre> To discover <i>which</i> properties are available from `ls`:<p><pre><code> ls | gm </code></pre> To select only files larger than 200MB:<p><pre><code> ls | ? Length -gt 200MB </code></pre> (ls is alias for Get-ChildItem, gm is alias for Get-Member, ? is alias for Where-Object)<p>It is somewhat ironic that PowerShell is more true to the principle of &quot;do one thing and do it well&quot; than the Unix shells.

As I see it ls does one job (list directory contents) and does it well (by supporting 37 flags).<p>If ls was a human, we would call him an expert.<p>Since unix tools expect input as plain text, the flags that ls supports are absolutely useful and I believe this is obvious to anyone who has written more than 50 lines of bash. When your input doesn&#x27;t support a standard complex structure (e.g json with some standard set fields), claiming that “sort by creation time” is a simple filter is funny.<p>First you would need a ls flag so that ls will output creation times in an easy to process time format (seconds since epoch), then you would need to ask sort to work on a certain (1 flag) numerical (1 more flag) field. Last you would have to pass sort&#x27;s output to a stream processor to get rid of unwanted fields.<p>Current “list files by creation time”:<p><pre><code> ls --sort=time --time=ctime </code></pre> Replace sort flag with filters:<p><pre><code> ls -l --time=ctime --time-style=&quot;+%s&quot; | sort -n -k 6 -r | awk &#x27;{ $1=$2=$3=$4=$5=$6=&quot;&quot;; print substr($0,7) }&#x27;</code></pre>

Yes, there is this strange idea in new generations that GNU&#x2F;Linux is UNIX, without having tried DG&#x2F;UX, Tru64, Solaris, HP-UX, Aix, Irix and many others.<p>Just drop a GNU&#x2F;Linux user into a default HP-UX installation and watch them getting around the system. Hint, those nice GNU flags and utilities aren&#x27;t there.<p>As for what UNIX turned out to be, I think Rob Pike as one of its creators is a good quote:<p>&lt;quote&gt;<p>I didn&#x27;t use Unix at all, really, from about 1990 until 2002, when I joined Google. (I worked entirely on Plan 9, which I still believe does a pretty good job of solving those fundamental problems.) I was surprised when I came back to Unix how many of even the little things that were annoying in 1990 continue to annoy today. In 1975, when the argument vector had to live in a 512-byte-block, the 6th Edition system would often complain, &#x27;arg list too long&#x27;. But today, when machines have gigabytes of memory, I still see that silly message far too often. The argument list is now limited somewhere north of 100K on the Linux machines I use at work, but come on people, dynamic memory allocation is a done deal!<p>I started keeping a list of these annoyances but it got too long and depressing so I just learned to live with them again. We really are using a 1970s era operating system well past its sell-by date. We get a lot done, and we have fun, but let&#x27;s face it, the fundamental design of Unix is older than many of the readers of Slashdot, while lots of different, great ideas about computing and networks have been developed in the last 30 years. Using Unix is the computing equivalent of listening only to music by David Cassidy.<p>&lt;&#x2F;quote&gt;<p>Taken from <a href="http:&#x2F;&#x2F;interviews.slashdot.org&#x2F;story&#x2F;04&#x2F;10&#x2F;18&#x2F;1153211&#x2F;rob-pike-responds" rel="nofollow">http:&#x2F;&#x2F;interviews.slashdot.org&#x2F;story&#x2F;04&#x2F;10&#x2F;18&#x2F;1153211&#x2F;rob-pi...</a>

I found this a weird post. The shell does know what files are (how else could you do &quot;echo * &quot;?) and programs do take arguments and have return values.<p>In any case, I think the author is kind of tilting at windmills; as far as I know, that Unix and Linux don&#x27;t really follow the Unix philosophy is reasonably accepted today, even if it wasn&#x27;t back in &#x27;83, when Rob Pike and Brian Kernighan made the presentation called <i>&quot;UNIX Style, or cat -v Considered Harmful&quot;</i>, which pointed out exactly those problems, which were already in development:<p><pre><code> Bell Laboratories Murray Hill, NJ (dec!ucb)wav!research!rob It seems that UNIX has become the victim of cancerous growth at the hands of organizations such as UCB. 4.2BSD is an order of magnitude larger than Version 5, but, Pike claims, not ten times better. The talk reviews reasons for UNIX&#x27;s popularity and shows, using UCB cat as a primary example, how UNIX has grown fat. cat isn&#x27;t for printing files with line numbers, it isn&#x27;t for compressing multiple blank lines, it&#x27;s not for looking at non-printing ASCII characters, it&#x27;s for concatenating files. We are reminded that ls isn&#x27;t the place for code to break a single column into multiple ones, and that mailnews shouldn&#x27;t have its own more processing or joke encryption code. Rob carried the standard well for the &quot;spirit of UNIX,&quot; and you can look forward to a deeper look at the philosophy of UNIX in his forthcoming book. </code></pre> Yet, Rob Pike himself uses Linux nowadays - the problem today isn&#x27;t so much acceptance, but backward compatibility.

I think the author misses the point. As far as I see it, Unix tools are optimized for one thing: the human user. If `ls` is bloated, it is only bloated from the system point of view. As a user, I only see a small command that is quick and consistent (to a reasonable extent) with other tools. I don&#x27;t need to know about 39 flags--that complexity is revealed with time, as I need to accomplish more complex tasks. For now, I only use and &quot;see&quot; two or three. And if more is needed, the program is small enough to be reasonably well documented. `man ls` perhaps with `grep` is all most people need.<p>This brings me to the second point. &quot;Streams of text.&quot; Just like `ls`, streams of text are an optimal format&#x2F;convention for humans. Many other things are better at being more compact or more efficient etc. But as formats and conventions proliferate, streams of plain text remain: readable and universal. Humans will ALWAYS be able to work with text. It is something that all humans kind of agreed upon--which cannot be said for any other formats or standards, which can offer various technical benefits at the expense of longevity, universality, and readability.<p>These two features (`ls` being relatively light and text streamy) leads to the &quot;bootstrapping&quot; effects sought after by the first generation of Unix developers. Learning about pipes and filters in one part of the system is applicable to all others. These tools scale with your level of expertise. They grow with you because despite the small quirks, there&#x27;s a remarkable consistency of interface: text! Consequently `ls` (along with many other core tools) is implemented in the same way across a staggering variety of platforms. It has survived decades of alternatives touted as &quot;better,&quot; &quot;faster,&quot; &quot;more usable,&quot; etc. etc. etc. That is the remarkable achievement of *nix &#x2F; GNU etc. approach to creating human-centric software. As we architect ever more complex systems, we would do well to understand why and how Unix has endured, warts and all.

This article gets it all wrong: The ls command does one thing well (LiSting files) and it isn&#x27;t optimized for terminal size, it&#x27;s optimized for PROGRAMMER productivity. The ls command gives you a default view that is exactly what you want most of the time and is easily copy&#x2F;pastable for other uses (it&#x27;s a single column) or more importantly, piped to some other program for further processing. And then there are easy-access options for common tasks (extra details, hidden files, sorting by attributes, etc...)<p>Powershell largely does exactly what the author is talking about. And yet it&#x27;s completely unusable for interactive use (though it&#x27;s great for writing longer-lived automations).

I think there&#x27;s a bit of a disconnect between how people talk about&#x2F;defend the unix shell and the reason it&#x27;s actually still there. People often defend the simplicity or philosophy - despite, as the article points out very well, much of it being clearly poorly designed.<p>But really the reason most of it exists unchanged is simply path-dependancy - we used it then so we use it now. It works well enough that there&#x27;s not enough incentive to drive a large change. There are dozens of different shell-like environments and scripting languages, but they never really took over out of plain intertia. That, and the conservative attitudes of people defending their Unix traditions (related in no small part to how much effort they put into learning the thing in the first place). For new people arriving on the scene, its still less effort to learn things, even when they get truely arcane, than to use something which relatively few other people use or even roll your own alternative...

I have never thought that UNIX approach is the &quot;best&quot; in a universal sense.<p>But when compared to how large, slow, complicated and opaque the &quot;alternatives&quot; are, UNIX is the clear choice for me.<p>I can modify and recompile UNIX to meet my own ends. That is all but impossible if I chose an alternative such as Windows.<p>For example, if I do not want ls to have 30+ options, I can trim it down to just a few options and recompile.<p>There are other utilities besides ls for viewing file information, e.g., BSD stat(1) or mtree -cp. The later displays mode information in octal which is something ls, despite its 30+ options, does not do.<p>Or I can write my own simple utility. I am given the full UNIX source code. Where is the source code for Windows?<p>Personally I keep my filenames short and never use spaces, so I sometimes use the shell&#x27;s echo builtin and tr(1) to get a quick list of files.<p><pre><code> echo * |tr &#x27;\040&#x27; &#x27;\012&#x27; </code></pre> If there were non-UNIX alteratives that were small, simple and transparent, perhaps I might not be using UNIX.<p>Because I have become very comfortable with UNIX, any alternatives that others suggest have to be comparable with UNIX on size and simplicity before I will take them seriously.<p>Currently, I use a kernel source tree that compresses to under 40MB; I can compile kernels with about 200MB of RAM and fully loaded kernels are about 17MB. Userland utilities are usually around 5MB as I prefer to put them in the kernel&#x27;s embedded filesystem. I do not like to rely on disks. My &quot;IDE&quot; is the same system I am compiling. There is no GUI overhead, everything can be done in textmode. The importance of the preceding two sentences cannot be understated.<p>I am always willing to consider non-UNIX alternatives that can offer the same or better flexibility, size constraints and simplicity.<p>But after decades of being open to alternatives, I am still not aware of any.

The essay &quot;Free Your Technical Aesthetic from the 1970s&quot; [1] by James Hague is appropriate here. See also The UNIX-HATERS Handbook; much of it is outdated, but I think some of it is still relevant, particularly the part on the shortcomings of pipelines.<p>I think it&#x27;s unfortunate that even if a good open-source PowerShell clone is developed (or PowerShell itself is open-sourced), it will probably never gain widespread acceptance outside of Microsoft shops. Tribalism so often trumps actual merit. But if someone did a PowerShell-like thing on top of Node.js, I could see that taking off, particularly if it were integrated with the Atom editor and its package manager.<p>[1]: <a href="http:&#x2F;&#x2F;prog21.dadgum.com&#x2F;74.html" rel="nofollow">http:&#x2F;&#x2F;prog21.dadgum.com&#x2F;74.html</a>

I can&#x27;t follow the comparison.<p>Linux is just a Kernel, most distros use the coreutils package which is developed&#x2F;maintained by GNU [1].<p>So any comparison is not between Linux and Unix, rather the GNU part in GNU&#x2F;Linux<p>[1]: <a href="https:&#x2F;&#x2F;www.gnu.org&#x2F;software&#x2F;coreutils&#x2F;coreutils.html" rel="nofollow">https:&#x2F;&#x2F;www.gnu.org&#x2F;software&#x2F;coreutils&#x2F;coreutils.html</a>

Linux is not Unix, that&#x27;s just fine, I&#x27;m happy it is not. I&#x27;ve tried “re-evaluated” shell (Powershell), no, thanks, it&#x27;s not intended for using by humans. Currently shell is still simple and powerful. Powershell is powerful and NOT simple.

Someone needs to port PowerShell to Unix :-)<p>I&#x27;m confused about the response to his tweet though - ls has a lot of switches, but I can&#x27;t really see any that don&#x27;t do something with listing files.<p>And then the response is &quot;Unfortunately, the lineage is _not_ Unix -&gt; Linux. It is Unix -&gt; Plan 9. Linux doesn&#x27;t follow Unix philosophy :-(&quot;... Except Unix&#x27;s ls eventually had 19 options, and the ls used by Linux is a port Gnu&#x27;s coreutils.<p>Nevertheless, I tend to agree. ls does a lot of formatting that could be moved to ancillary utilities. I guess what I like about PowerShell is the way it allows for redirecting into objects - that really is an innovation that Unix shells don&#x27;t have (at least as far as I&#x27;m aware...).

&quot;A system composed of a zillion tiny modules is itself a pile of mud&quot;<p>Having so many args to `ls&#x27; might be an overkill, but I certainly don&#x27;t want to combine four programs to get full coloured sorted listing of current directory.

&gt; You can’t write a function that returns a list of files because the shell doesn’t know what a “list” is, doesn’t know what “files” are, and couldn’t tell you the difference between a “function” and a “program” if its life depended on it.<p>Could it be that what this article is really groping towards is Perl? I mean Perl not as it has become, but as it was -- Perl 4 perhaps: basically the capabilities of the shell, augmented with a couple of data structures. So there are strings, lists, hashes, and functions, and that&#x27;s pretty much it.

The `ls` example is especially interesting: it made me think about perfomance of pipelines. Imagine a folder with 10^6 files. Listing them would take some time; if you only need 10^3 of files selected by the simplest filter (LIMIT 1000, for example), piping the whole list, in whatever format, to the filter program would be very far from optimal.<p>What would be a perfect solution is to use iterators instead of lists (the best example that I know of is Linq, but it seems that this approach is common enough).<p>However, now we don&#x27;t even exchange data between different programs; now we couple them, running at the same time, by a complicated interface! It seems that if we take one more logical step in this direction, we&#x27;ll get generics.<p>Don&#x27;t you start to feel that this kind of ideal shell, piping different functions together in the most correct way possible already exists, and it&#x27;s already installed on your computer? I&#x27;m talking about interpreted languages like Python, Perl, Javascript, Ruby or whatever else you fancy.<p>Of course, I don&#x27;t seriously think that we should abandon shells for language interpreters. It&#x27;s just that balance between complexity and universality is a very delicate thing, and convention, however bad, is good just because it&#x27;s a standard.<p>In the meanwhile, I&#x27;m quite happy with the Fish shell.

Seems like the article, and several comments here, equivocate Linux with GNU. While they are most commonly used in conjunction with one another, I still don&#x27;t believe that &quot;this GNU program does too many things; therefore Linux doesn&#x27;t adhere to the Unix philosophy&quot; is a valid argument.

I think it&#x27;s easy to misinterpret any of the &quot;Unix philosophies&quot;. It&#x27;s Ye Olde &quot;MIT vs New Jersey&quot; again, and Unix is relentlessly straight out of NJ. Simplicity in the &quot;NJ&quot; sense isn&#x27;t what you&#x27;d expect if you&#x27;re of the &quot;MIT&quot; mindset.

&gt; Granted, we’ve developed graphical user interfaces that keep “ordinary users” away from the command line, but we still expect “serious developers” to drop down into a demonstrably inhumane environment to get anything meaningful done.<p>Maybe I am too familiar with it, but I have yet to encounter a graphical environment that matches the effectiveness of the terminal and command line programs. I saw systems, that tried to replace shell one-liners with a hundred thousand line GUI application, that no one cared to use.

&quot;Many of the usability issues raised by Don Norman in his 1981 criticism of Unix have gone largely unaddressed&quot;<p>That&#x27;s because Don&#x27;s complaints were not about intrinsic issues, but complaints from someone who had a frustrating time developing a mental model of the components of a system and who tried to use the just-so story of &quot;cognitive design&quot; to justify his inability. The non-point about prompting y&#x2F;n to delete is particularly misguided.<p>Similar to this article.<p>&quot;ls&quot; is called &quot;ls&quot;, not because of slow 80-char terminals, but because <i>humans</i> are, on average, not fast typists. One wants to list the contents of a directory <i>a lot</i>, and &quot;ls&quot; is quicker than &quot;catalog&quot;. Similarly, &quot;ps&quot; vs &quot;tasklist&quot; and &quot;cc&quot; vs &quot;compile-a-c-program&quot; :-). Notably, in powershell, most people have aliases that do this for powershell&#x27;s necessarily verbose query functions.<p>It&#x27;s primarily a command shell for controlling the machine, not a programming language. And the UI choices made over the last 30 years reflect that.<p>That fact that ls takes arguments is a UI improvement that keeps you from having to write tiny programs all the time and pipe it&#x27;s output to sort and awk all day long. It is also why the &quot;one job well&quot; crew is only half right, and why Powershell kind of feels awkward even though it seems to get more things &quot;correct&quot; in that regard.<p>This is just yet another just-so story by someone who wants ideological purity in UI, which is completely misguided when UI, by definition is oriented towards a universe which is not ideologically pure.

Do one thing and do one thing well doesn&#x27;t equal do one simple thing. Furthermore, ls can actually do One Thing Only if this is enough for you and that is plain ls. What would be the better alternative? Have multiple ls, ls-color, ls-long-color, etc?<p>The discussion about Unix philosophy, standards, etc is a different story but I did not see that from the articles examples.

UNIX emerged in an era of more structured systems, though. Many OSes of the time did not have files in a naive sense -- they had record-based formats. &quot;Everything is a file&quot; was a revolutionary idea, and made UNIX simple and lightweight and portable.<p>It is interesting that the author doesn&#x27;t highlight &quot;everything is a file&quot; which I have always understood to be the fundamental UNIX philosophy. (&quot;Do one thing well&quot; applies to so much more than UNIX.)<p>Our understanding of how to use types and how to structure data on disk and for streaming has greatly matured; but that doesn&#x27;t mean we can ever afford to pay more to get nothing. Say for a moment that `ls` provided data in tab-separated value format (a format which is, unlike CSV, line filterable). Wouldn&#x27;t that go a long way toward allowing for the HOFs the author mentions? And it wouldn&#x27;t involve a shell type system (or &quot;Shell Object Model&quot;).

I dunno, the shell combined with sed and awk is pretty powerful and so far has taken care of everything I need to do

I think you have it backwards: there are two legitimate counterarguments.<p>One is that &#x27;ls&#x27; is the common program for querying the directory data structure (which in original Unix was just a kind of file). So it makes sense for it to output the data in different kinds of ways.<p>The second is that its a user mode command. It&#x27;s easier than having a lot of little commands (ls for all files, basically ls -1; lsF for showing file types, etc). In fact I&#x27;ll often to &#x27;ls&#x27; then realise I want more info so will type &#x27;ls -l&#x27; (say) which is really my brain saying &quot;oh yea, just like what I just asked for but more info&quot;.<p>Better to pick something more deranged like tar or cpio!

One of the reasons for ls having additional functionality, is that it is used in restricted shells - such as those used on rssh (SCP,SFTP, rsync and the like). There are no filtering utilities available in these shells:<p><pre><code> -rbash-3.2$ ls $PATH chmod cp groups ls mv rm scp</code></pre>

these arguments about unix philosophy, to me, seem to miss the point about what Unix actually is...which, again...to me, is a bunch of tools to get things done.<p>being able to get things done with Unix means that you just put up with the weird bits and move on. there doesn&#x27;t need to be an overriding philosophy. there are places where there seems to be consistent concepts, but their implementation isn&#x27;t and doesn&#x27;t really need to be consistent.<p>the way that new ideas get so intensely opposed needs to be really looked at though. there&#x27;s no need to be so insulting if something isn&#x27;t a good idea...it just won&#x27;t get used.

Incidentally, there is plenty of ls alternatives like lsp: <a href="https:&#x2F;&#x2F;github.com&#x2F;dborzov&#x2F;lsp" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;dborzov&#x2F;lsp</a>

In other words: The shell should be a lisp.

Ed is still one of favorite editors out of UNIX-like terseness.

ls&#x27;s plethora of flags doesn&#x27;t stop &quot;echo *&quot; from working.

Meta: I think someone made a comment that forced the whole page to be wider than it normally is? like when you post an image on a forum that forces the whole page to widen to fit it horizontally.