Where the Unix philosophy breaks down

I'm loathe to hand credit to Microsoft, but I think they're onto something with Powershell. Rather than being limited to passing text streams around, Powershell commands pass full-featured objects back and forth (with callbacks and everything!).<p>This flexibility really seems to break down the transaction costs that the original article talks about, when compared to traditional Unix pipes.<p>I'm still trying to come up with a Linux-y way to do this. Microsoft can mandate that all Powershell commands run under the CLR (and therefore can talk to arbitrary implementations of some CLR class) but any FOSS equivalent is going to have to deal with people wanting to interface programs written for a dozen different runtimes (and probably even some C code running on the bare metal).

Of course you're not going to rewrite Word by piping data around, for opening and saving and doing spell check.<p>Scripting languages are the next extension of unix - you take all those little programs that do something well, put them together with some control structure and you can do big things, often in an automated manner, which is difficult to do on a GUI.<p>That said, showing the GUI interface to someone and having them make selections is much more suited to other programming methods (MVC,etc.), often which can just run scripts on the back end.<p>The point of unix isn't having a bunch of programs that do different things - it's combining them like legos to do complex tasks via scripting.

In the free software world, the functional boundaries between programs are, in fact, usually respected, even with interactive programs, where his "transaction costs" would apply. Emacs edits, awk does its thing, etc.<p>I think the reason programs creep in functionality is NOT these technical sounding "transaction costs", but in order to sustain vendor lock in. In a world where nobody worries about lock in, there is no reason to bundle an editor with a calculator with an email program.<p>In the Adobe world, the boundaries of the programs are pretty tight, with a little bit of bleed over. Counter example? Maybe.<p>That was not a great article, if you ask me -- I think the analysis was trite, and MS Office software probably isn't a good example of anything.

Honest question: does the Unix philosophy apply to all kinds of software?<p>I am curious to know if it's fair to compare typical Unix software (CLI-based, text-based --- usually system software is what comes to mind) with application software (Word processing, enterprise software, etc).

I couldn't help but read this article as a demonstration that the Unix philosophy still holds strong.<p>Even the MS Office example goes that way: people who want a nice graph and a computing sheet in a Word document will embed an Excel component, which is virtually Excel running and piping its output into the Word document. The same goes for reaching Access datasets and queries to produce graphs in Excel: Access pipes its data to Excel, which in turn produces a graph.<p>"programs gain[ing] overlapping features over time" is a result of feature creep and certainly a lack of architecture foresight. The author refers to spreadsheet features in Word and database features in Excel. I can only take a guess at what he refers to but, assuming he's not talking about object embedding (which I took care of above), being able to <i>draw</i> tables graphically in Word is not equal to having a sheet able to compute formulas and render graphs, and being able to sort and filter columns in Excel is worlds apart to queries on a structured, relational table.<p>The core of the article seems to boil down to the fact that software inevitably brings duplicate functionality, and that this is good because it alleviates the effort of bridging applications. Yet duplicating functionality puts a burden on the developers that might be better spent elsewhere (including providing easy access to the "real deal"). What's more it suddenly becomes begrudgingly effortful for the user to achieve a task escaping the subset of duplicated functionality. In short, it might save you a little work, but it doesn't scale. That's why properly designed technology allows for trivial connection of software (Unix pipes) and description of contracts (sane text output) or abstracts the contract part entirely (e.g including an Excel spreadsheet in a Word document).

Lately, I've been thinking a lot about interfaces, operating systems and procedure composition and this are some of my "ruminations":<p>- None of the modern operating systems is really orthogonal: neither linux nor the BSDs embrace the real "Unix philosophy"<p>- CLIs are monodimensional, and suffer from the same problems that dataflow programming has: it's easy to split the flow of data into two pipes, but is really hard (semantically) to rejoin them.<p>- GUIs are imperative in nature, and that hurts composability.<p>- Shells and REPLs have too many overlapping features (like OSs and ProgrammingLanguages+Libraries): the de facto standard for shells, bash, has a syntax that, for a programmer, feels really unclear. Every non trivial bash script that I wrote has turned really fast into escaping hell.

On the contrary I think whenever you get into a situation where there is an overlap in functionality, that means it needs to be further broken down into more smaller independent units. I see tremendous value in adhering to this philosophy, and usefulness of Unix is a testament to it. That does not means there are no shortcoming in the overall Unix approach but as far as this particular philosophy is concerned, I think it is very smart strategy to adhere to. Look at nature, our different body parts are specialized for specific functions(for example eyes, ears etc) and they do those functions remarkably well and when you combine these, we can recognizing the physical world.

For quite a few years now it seems like a lot of Unix-style software has gone beyond text streams, to include a back channel connection to external data and processes (databases, http services, etc).<p>I generally enjoy sharing context as a data structure with a file-system-like interface (e.g. REST) between processes, and only more so when the command/control interface is a simple text stream. Of course, this reminds me of Plan 9, but there's also 1060 Research, and other examples I'm sure.<p>There is quite a lot of head room left in the Plan 9 paradigm, and you do yourself no disservice by taking some time to figure it out. It really is more Unix than Unix.

Uhhmmm... am I the only one that noticed that the sole example in the article of "programs gain overlapping features over time" is of Microsoft software ... which, like, has nothing to do with Unix? My unix/linux box doesn't have overlapping software. Every piece of software is a specialist and does what it does well. It is easily replaceable and does not interfere with other software installed on the box. The unix/linux philosophy works extremely well. That's why it's popular.

&#62; For example, think of the Microsoft Office suite.<p>Beneath the covers, don't the Office applications share a lot of common code?

Employee transaction costs are not lower than contractors. Employers typically pay vacation, sick pay, workers comp, health care, 1/2 social security plus other payroll taxes, equipment, office space, and incur certain liabilities for the employee.

It can break down, but it's a damn good place start, especially if you don't have a good philosophy to replace it.<p>Does it need stating how useful and powerful text based output can be?

&#62; Piping the output of a simple shell command to another shell command is easy. But as tasks become more complex, more and more work goes into preparing the output of one program to be the input of the next program. Users want to be able to do more in each program to avoid having to switch to another program to get their work done.<p>I see no logic here. Can anybody explain that?

We can handle objects in CSV/TSV format with commands sort, cut, join, paste, awk, perl, python, etc.<p>We can handle XML objects with commands xml_grep, xmlpath, xmlpatterns, perl, python, js (JavaScript has builtin support for XML/XPath), etc.<p>JSON format in new, but we can handle it with js (native format), perl, python. CouchDB already uses JSON over pipe to communicate with tools.