The Powershell idea to push structured data through the pipe, which this project fundamentally replicates with a relational structure, has its merits; but it will always displease someone, and carries the burden of having to rewrite millions of utilities in order to unlock the real potential.<p>What if, instead, we pushed the approach “upstream”, asking systems to have an additional stream that all shells can access? We have stdout and stderr, we could add “stddata” - a pipe that expects and produces structured data. Then it would become trivial to add support around the userland, without displeasing anyone - because it’s just an additional interface, not a replacement. The pipe should support two or three formats to start with (csv, json, and maybe xml, so you can handle both tabular and child-parent relations with a varying degree of precision) and shells could have a special character for piping stddata (I like ¥) so everything else about them would stay the same.
The compelling idea here is that they convert the output of common shell commands into tabular data that can be manipulated using common operators, so that you don't have the remember sorting/filtering/grouping flags that may be different for every different shell command. So, imagine being able to use the same sorting/filtering logic on the output of `ls` as you might on the output of `ps`, and without relying on hacky solutions like pipelines to `grep`, `cut`, and `sort`.<p>It also means shell command output can be easily transposed into JSON and CSV. Actually pretty clever!
This is almost exactly what I have dreamed with a shell. I'm building a relational language (<a href="http://tablam.org" rel="nofollow">http://tablam.org</a>) with the plan of something alike.<p>Some extra ideas:<p>- Tables are amazing. But I think we need at least support for this "shapes" of data: List, Tables, Trees and maybe graphs<p>- I think will be nice to have an equivalent of Request/responses alike http. And then a way to introspect what a command do, like auto-swagger.<p>- Having widgets like table-browser, graphs, etc AS COMPONENTS.<p>In FoxPro you could say "browse" and it display a DataTable grid with full editing capabilities on the current table (or it ask for a table to open). In this case:<p><pre><code> ls | ui_browse
</code></pre>
The key here is that the widgets work stand-alone, so is not necesary to fully build a terminal app just to see nicely data. Look like <a href="https://goaccess.io" rel="nofollow">https://goaccess.io</a> and more like this idea: <a href="https://github.com/sqshq/sampler" rel="nofollow">https://github.com/sqshq/sampler</a><p>This mean we could get a jupyter-like experience here!<p>- Some way to do curryng and maybe environment alias: You could setup an environment for the system, the user, this directory. In the environment is a .toml with vars and stuff.<p>The other thing where I think shells fail... is that are still old-style terminal. Having text output is cool, but having more rich widget system will unlock better the potential
There's a lot of comments here already so I don't know if this has been addressed already, but I don't get this shell. It seems to be fixing a problem that doesn't exist, and a lot of comments are echoing it. Current shells are not deficient, as Doug McIlroy originally stated, the universal interface is text and tools should take input and generate output based on this. You obviously can then pipe things together with these tools not necessarily knowing in advance how they will be used in a pipeline. You can get very far with sed, awk, etc, you just have to spend a little bit of time learning them, but they are not complicated. You can of course then introduce a command like jq for specialised purposes. But hte point is the shell isn't providing this, they are separate tools that you orchestrate using the shell. There is not a flaw with how it works at the moment, I truly believe and this isn't a dig at this project or anyone in particular, that people do not spend enough time learning these tools that have been there for decades. The re-inventions that appear on GitHub are not a product of the deficiency of the existing tools, I don't know what they are but it's not that. Unless I am missing something.
A brand new shell, especially in Rust is a great thing to see and also how it tackles the problems that exist in other shells.<p>However, its interesting to see so many dependencies that are required to build it (73 crates last time I checked) and as shells have always been portable across many operating systems, I wonder how portable nushell would be since it wishes to replace my current shell.
This looks great, but what exactly are the benefits over PowerShell? The article mentions PowerShell, then says:<p>> What if we could take the ideas of a structured shell and make it more functional? What if it worked on Windows, Linux, and macOS? What if it had great error messages?<p>- Any shell with piping is already very functional, and PowerShell has script blocks (higher-order functions). Or does this mean "functional" in the literal sense? What functionality is missing?
- PowerShell works on macOS, Linux and Windows
- The examples of error messages further down look almost exactly like PowerShells error messages (underlining the command in the pipeline that failed)<p>It is not clear to me what exactly the authors sought out to do different
Pretty cool idea and project. Hope you take it further and it turns into something truly spectacular!<p>My only comment is on the article. This line:<p>> Nu would not have been possible without Rust.<p>That's just patently false.<p>Perhaps you wouldn't have chosen to code it in another language. Or perhaps Rust had one or two benefits that assisted in the development of Nu. But you don't word it that way or go into any details as to what made Rust special/better than X, Y or Z language (not that you need to, but if you're going to make a wild statement, you should be prepared to back it up). The only 3 points made following that were about features that have widely existed in other languages well before Rust came along: async/await, async streams, and serde.<p>Rust is cool and is still exploring some really great ideas in making programs safer and more maintainable. But, it's not the greatest language ever (at least not yet), nor does it make the impossible possible. It can merely make some things easier at the cost of something else.<p>I'm glad you were able to use it effectively to create a tool you - and others - find useful.
Interesting! I am also looking forward to setting aside time to play around with <a href="https://elv.sh/" rel="nofollow">https://elv.sh/</a>, which takes a completely different approach with a similar philosophy.
I pretty much like the idea, but my biggest problem with current day shells is error handling.<p>Errexit (aka `set -e`) is a great idea badly executed. There are many things which can be improved (e.g. printing a stack trace by default), but at its core, there are two things which break it.<p>1. Inconsistent behavior: <a href="https://www.in-ulm.de/~mascheck/various/set-e/" rel="nofollow">https://www.in-ulm.de/~mascheck/various/set-e/</a><p>2. You can't rely on it.<p>Inconsistent behavior should be no problem if a new cross-platform implementation is coming, but not being able to rely on it sucks. To give a quick demo:<p><pre><code> #!/bin/bash
foo() {
set -e
false
echo "Sucks"
}
printf 'Normal: %s\n' "$(foo)"
printf 'Condition: %s\n' "$(foo || true)"
</code></pre>
Output:<p><pre><code> Normal:
Condition: Sucks
</code></pre>
So if someone is about to bring a new shell, please fix the error handling along the way. But hey, Yehuda Katz is part of the project. So I am looking forward to what is coming next :-)
I read this, and read all about it, and the whole time I thought the name was “nutshell,” until I saw they abbreviated it “nu.”<p>Now I’m really disappointed that it’s not called “nutshell.”<p>But this is still very cool.
While I fully appreciate & support the structured shell approach (I believe it’s the future), I wish the efforts in making a mew shell should be more directed to a small selection of projects.<p>elvish[0], uxy[1], ngs[2] and basically all shell projects that allow other languages(e.g. python: tako[3], racket: rash[4] janet: janetsh[5]) are all similar attempts; and there are numerous more alternative (non-structured) shells like fish[6], and a whole lot more.<p>As a daily user of fish and a person hyped by elvish (but not using it as a daily driver :-(), I hope some structured shells get at least some traction, but there are too much approaches.<p>Well, I didn’t start as a rant but it became one anyway.<p>[0]: <a href="https://elv.sh/" rel="nofollow">https://elv.sh/</a><p>[1]: <a href="https://github.com/sustrik/uxy" rel="nofollow">https://github.com/sustrik/uxy</a><p>[2]: <a href="https://github.com/ngs-lang/ngs" rel="nofollow">https://github.com/ngs-lang/ngs</a><p>[3]: <a href="https://takoshell.org/" rel="nofollow">https://takoshell.org/</a><p>[4]: <a href="https://rash-lang.org/" rel="nofollow">https://rash-lang.org/</a><p>[5]: <a href="https://github.com/andrewchambers/janetsh" rel="nofollow">https://github.com/andrewchambers/janetsh</a><p>[6]: <a href="https://fishshell.com/" rel="nofollow">https://fishshell.com/</a>
Interesting PoC, but does it solve a real world problem?<p>If I had to reinvent the shell by breaking backward compatibility, I would do a statistical study using all the shell scripts publicly available on GitHub, a survey about the critics of the current shells in the literature, and then I would try to optimize a metric such as : words typed to complete a task, reproductivity, time-to-market, communication, mathematical soundness, etc. The result of the study might be that the current shell is enough and the cost of switching to another is too big (I don't know, it's an hypothesis)<p>People shared many thoughts and opinions on this thread, so that's already a win by itself
Looks cool, but when I see things like this I figure there's no point in trying to learn something that won't exist on 99% of the systems I need to work on. Bash sucks, but it's almost always there. Embedded systems, desktops, big iron... Like many of the established Unix tools, bash has market penetration and staying power.
I like the concept a lot, especially the focus on outputting data in an organised way.<p>What I feel could be improved is the presentation of the tabular data. Some ideas below:<p>- The screenshots show two horizontal lines consisting of dashes, one at the start of the table, one at the end. To me they seem redundant and could be replaced with spaces (whitespace). I did a quick check and PostgreSQL's psql tables also don't have lines like this.<p>- How about using Unicode lines instead of dashes and plus sign in the header? This way there would be fewer gaps there. I'm not sure though how this would affect compatibility.
I love the idea of treating output as tabular data.<p>I see you already have a where operator. What would fit this very well would be in-built sql-like syntax. Since tables are first class, you should have an in-built way of handling them.<p>A languiage similar to q/kdb - it has a great SQL dsl, but also works as a regular programming language. It's also very concise and expressive, you can write oneliners that do mind bogglingly complex things.
Whoa. It has been a while since I've seen something that truly feels like it can revolutionise my workflow. Amazing job to everyone involved in this project!<p>Really excited to try this, surprised that there is no pre-built binaries. Are these planned?
i feel like structured data in a shell will phase out. The posix pipes allow streaming <i>any</i> data. So if programs want to stream structured data, just build programs who know how to communicate with each other using structured data. The older shells withstand the test of time with generic interfaces.<p>For instance, because of the influx of JSON the past few years I have learned to use jq very well. It does everything I need and I didn't need to upgrade my shell to use it. What if everything becomes protobuf down the road?
I like this a lot but why does this have to be a completely new shell? Couldn't this also just be a set of tools that I can use in an existing shell and get the best of both worlds?
Looks amazing! Unfortunately a `cargo install` on stable fails due to a dependency "futures-async-stream v0.1.0-alpha.4" with the "rawkey,clipboard" features.<p>The source is pretty interesting. Here's the PS command: <a href="https://github.com/nushell/nushell/blob/master/src/commands/ps.rs#L27" rel="nofollow">https://github.com/nushell/nushell/blob/master/src/commands/...</a><p>And the sort-by: <a href="https://github.com/nushell/nushell/blob/master/src/commands/sort_by.rs#L29" rel="nofollow">https://github.com/nushell/nushell/blob/master/src/commands/...</a>
How do I do this in nushell? It's an alias I created recently to ripgrep through java and go files and tell me which have both of two words:<p><pre><code> function both {
comm -1 -2 <(rg -i -l -t java -t go $1 | sort) <(rg -i -l -t java -t go $2 | sort)
}
</code></pre>
Example usage: `both sharded jobqueue`
An object pipeline is certainly interesting, and I do use PowerShell from time to time. However, there is a reason extremely simple concepts such as "everything is a file" or "input/output/error are all text" survived for this long. I will quote an answer provided in a vaguely related question [0]:<p>> There is no spoon^H^H^Hstructured data. There is only text.<p>[0]: <a href="https://unix.stackexchange.com/a/400" rel="nofollow">https://unix.stackexchange.com/a/400</a>
This nicely ties into the recent HN thread about a dream OS that would provide databases at the OS level instead of forcing the re-implementation of databases in many programs.<p><a href="https://news.ycombinator.com/item?id=20754592" rel="nofollow">https://news.ycombinator.com/item?id=20754592</a>
<p><pre><code> ^ls -la | lines | skip 1 | split-column " " perms files group user size month day time name
</code></pre>
This is neat, but does nushell have the ability (should be trivial to add) to save such a pipeline and reuse it (similar to an "alias")?<p>e.g.<p><pre><code> myls = ^ls -la | lines | skip 1 | split-column " " perms files group user size month day time name
</code></pre>
and then:<p><pre><code> myls | where size > 1000
</code></pre>
Also perhaps the built-in commands could be namespaced? E.g. nu.ls or nu-ls vs ls. Then you wouldn't need escaping, and the names wouldn't shadow traditional commands (but could allude to them).
I really enjoy seeing new ideas around interfaces. But I do wonder, rather than creating a new shell, couldn't we have a 'json/structured' mode for cli programs - and maintain compatibility with existing shells? For example if it was toggled with a SHELL_FORMAT environment variable?<p>What issues could be faced with this approach?
This project has goals similar to GNU eshell [1]. Great ideas, thank you for sharing.<p>[1] <a href="https://www.gnu.org/software/emacs/manual/html_mono/eshell.html#Introduction" rel="nofollow">https://www.gnu.org/software/emacs/manual/html_mono/eshell.h...</a>
This is great, we have a tendency to be content with existing tools we're familiar with but be blind to their limitations. I'm looking forward to how this project evolve!<p>However an important point for practical usage is the capability to run existing scripts, for example as oilshell [1] does. I skimmed through the README and doc but couldn't find the current status nor future plans.<p>[1] <a href="http://www.oilshell.org/blog/2019/06/17.html" rel="nofollow">http://www.oilshell.org/blog/2019/06/17.html</a>
Any time the first line of a project description reads "This is NewTool, written in <language>" it's an immediate red flag that makes it hard for me to continue reading.<p>Obviously it doesn't necessarily mean the project has no merits, but language choice is one of the less relevant details and it signals that the author wanted to make a toy in a language new to them. Good for them, but most of the time the rest of the world doesn't care.<p>If the project has merits, and this one might, it should talk about them in the intro.
This is really cool. I've always liked the ideas behind Powershell's approach, and so I'm glad to see someone is working on Linux-friendly shell that has the same sort of features.<p>I have to say that I don't care for the snark about "having to remember" arguments to various pipeline-munging commands. I get that this is a big challenge for new folks working with typical unix shells, but I'm not sure I see how this actually solves it. In order to avoid all those flags, a system like this requires more verbs overall, and those verbs have their own argument structure, eg `split-column " | " firstname lastname job`, so now instead of being able to label the arguments and order them how I want, I now have to remember the order they come in, and what they mean, and that `split-column " "` apparently eats extra whitespace (does `split-column " | "`?). Yes, the standard unix tools have these issues, as well. That's my point.<p>So yes, I love this idea, please keep pursuing it, and I look forward to trying it out. But focus on the benefits of working with structured data rather than pushing lines that sound good but aren't actually true.
The idea is inspiring!<p>I'm finding it hard to get over them using pipes and dashes for table borders. Is this the 70's? It's genuinely hard to read these tables. See: <a href="https://www.interaction-design.org/literature/topics/gestalt-principles" rel="nofollow">https://www.interaction-design.org/literature/topics/gestalt...</a>
Hmm I like this but there's no way it's replacing fish for me, I'd rather use it as standalone command that provides the same functionality.<p>e.g. `ls | nu where size > 10` or `nu open Cargo.toml`
As a long time Unix person who also discovered pwsh, like Yehuda did in the article:<p>Can I make nushell cmdlets (or equivalent) in something other than rust? pwsh limits me to .net languages, but I wanna use node.
This looks really cool but isn’t Powershell cross platform now? It looks to shadow much of the features of Powershell which opens scripting to the whole of the .Net library
Not sure if they're aware of Shcaml, which seems even more powerful since it supports anonymous records in lines:<p><a href="https://www.ccs.neu.edu/home/tov/code/shcaml/doc/" rel="nofollow">https://www.ccs.neu.edu/home/tov/code/shcaml/doc/</a><p>Edit: features look similar in fact, since you can name columns and refer to them by name. Neat!
I find it good in general but don't like the idea of reusing the names like `ls` and so on which is confusing with the option to invoke the real `ls`. Since they have single-character names already, a simple `l` would be one way to solve this.<p>I'm interested to see a solution to error handling (within multiple pipes!) and subprocess management as well but couldn't find much in the docs.
Mario has a similar feel to this but in Python, and lets you use arbitrary code and construct your own functions in pure Python.<p><a href="https://github.com/python-mario/mario" rel="nofollow">https://github.com/python-mario/mario</a><p><pre><code> $ mario read-csv-dicts map 'x["name"]' < hackers.csv
Alice
Bob
Carol</code></pre>
Love it! Thanks. I've always thought powershell was a great concept, badly executed (those long names with capital letters, confusing errors, and insanely long time to return when a command is missing).<p>Is anyone interested in a shell with a GUI? As in, data still passing as text, but having a shell which can render textual data into widgets when displayed in the shell?
Very neat, love it. Few things--
1) Recommend this install line (for those of us on mainline/standard who've never used a different toolchain) cargo +nightly install nu
2) Things like ls | where readonly don't work as expected-- readonly should be a bool, shouldn't it?
3) How do you paginate? ls | less doesn't work
Interesting, there's definitely a niche for this. But I'm not sure it could replace a normal shell in my opinion. I'm confused at what other commands would do in this context? Will it tabularize; echo "Hello World"?<p>There's an elegant simplicity of treating all piped output as a string rather than objects...
Feature suggestion: detect timing differences between program outputs to determine where an individual output ended and the next one began, so people can automatically scroll to the top of a stack trace for instance by double clicking it or something because the shell will recognize all those lines as a single message.
Is the processed data typed? Can you offer auto completion of data fields based on these types?<p>I also thought quite a couple of times about implementing a data centric, typed shell language. My primary goal was to have a shell where one could discover every feature by oneself through the magic of types.
People are really excited about this. Here's a question:<p>Would you pay for this?<p>My prior is that no, people would not pay for a new shell - they'd prefer sticking to bash because A. it's "standard" B. it's free.<p>I have to wonder how developing new tolls like nushell can be sustained over the long run.
I feel open parsing files is misguided. Downright wrong if it profilerates into scripts, because suddenly simple scripts can't work on JSON, because opening JSON returns something different from a stream of lines.
This looks very interesting.<p>Unfortunately, I'm having trouble building it on Mac OS X 10.14.5 and with the latest rust update installed.<p><pre><code> cargo:warning=libgit2/src/streams/stransport.c:12:10: fatal
error: 'CoreFoundation/CoreFoundation.h' file not found
cargo:warning=#include <CoreFoundation/CoreFoundation.h>
</code></pre>
Can anyone clue me in? An Xcode update perhaps? (I'm doing it, but it is taking forever). I have a CoreFoundation.h on my system under MacOSX10.14.sdk, but it is not under a CoreFoundation folder.
Bash is ubiquitous - the best selling point for knowing it and sticking to it.<p>I desire a fancy new shell but don’t want it to become a crutch, limiting my usefulness and familiarity when I’m on a system lacking it.
The idea of programs that work on structured data is quite old. Anyone ever read "structural regular expressions" by Rob Pike? Highly recommended: <a href="http://doc.cat-v.org/bell_labs/structural_regexps/" rel="nofollow">http://doc.cat-v.org/bell_labs/structural_regexps/</a><p>Also, using structural regexps for efficient interactive text editing: <a href="https://github.com/martanne/vis" rel="nofollow">https://github.com/martanne/vis</a>
This is very cool! The giant drawback of Poweshell is that it's so focused on Windows - this projects seems to provide some of that power cross platform. Very neat idea!
I'we always felt like there should be no file system and no shell but a relational database with a particularly suitable and extensible SQL dialect REPL.
In case you are interested in more sophisticated ls or find, try fselect ( <a href="https://github.com/jhspetersson/fselect" rel="nofollow">https://github.com/jhspetersson/fselect</a> ). That tool supports JSON as output as well.<p>Fselect has few really cool features, esp. when you got to deal with complex queries. Although not unix way at all.
> Everything is data<p>We use a similar approach in our data prep tool [1] for non-technical users. Lists of files, lists of emails, contents of text files, spreadsheets or database tables - piping it all together allows mixing ETL and process automation in a similar fashion.<p>[1] <a href="https://easymorph.com" rel="nofollow">https://easymorph.com</a><p>Disclaimer: I'm the founder.
I too have been off and on writing a Rust shell, though I want it to support `chsh` POSIX compatibility for my Arch setup. It's no where near complete though.<p><a href="https://github.com/nixpulvis/oursh" rel="nofollow">https://github.com/nixpulvis/oursh</a><p>`nu` seems very cool (and I love the name ;), but isn't building atm.
I like how powerful Powershell is, but I absolutely loathe it's verbose syntax and Pascal-Kebab-Case naming (at least for idiomatic Powershell).<p>This seems to have a lot of the power of Powershell, but with a terse, familiar syntax - I really like the look of this!<p>Being cross-platfotm is a huge seller for me too - I mainly work on Windows, but also on Linux and MacOS.
I like this. Very neat with tables; easier to memorise than PowerShells very verbose command names. It feels like a pseudo SQL interface to the OS.
Only area where I think it oversteps the Unix philosophy is the integrated terminal multiplexer. I think the shell should be terminal agnostic.
This seems to address a lot of the shortcomings presented here: <a href="https://mkremins.github.io/blog/unix-not-acceptable-unix/" rel="nofollow">https://mkremins.github.io/blog/unix-not-acceptable-unix/</a>
This looks great. Is the autocomplete as good as fish? If so I may switch from fish to nu. The autocomplete matters a lot. With fish I often repeat commands during development and I only have to type the first letter or two.
Lately I've been paying more attention to alternative CLI tooling, extensions, etc. Our open source community uses git-flow-avh for example and it has been extremely helpful. I can't wait to try this out.
The next step this begs for is for the shell to be like lisp. The results are s expressions and you can write macros to slice dice and pipeline the data to your hearts content.
I really like this conceptually. I feel like with Powershell, you need a bunch of commands that actually pass objects around or act upon them; and something similar in Linux would require writing a lot of extensions first.<p>I've been using fish shell for years and glad I started down that route. It's been incredibly useful.<p>I really want to give this a shot too. I feel like too often I'll open up Python, read a file and then manipulate it when I need to something more complex than simple grep/sed/cut/wc etc... This feels like it really fills in that gap. It starts with something really simple: get your unstructured output in a structure, and then build from it. I could see people writing more custom extensions and objects later on if this gets popular.
It looks to me like they took the concept of “everything as sql” to the extreme.<p>Do you want to live your entire life and everything you do in sql? I don’t.<p>I think I will pass.