The most obvious application of tree-sitter is editors. I wrote a VSCode extension to replace the built-in syntax coloring with tree-sitter-based coloring: <a href="https://marketplace.visualstudio.com/items?itemName=georgewfraser.vscode-tree-sitter" rel="nofollow">https://marketplace.visualstudio.com/items?itemName=georgewf...</a><p>I actually think it would make more sense for the various VSCode language extensions to just bake in tree-sitter for their language. I have had a PR open to do this with golang for a while: <a href="https://github.com/microsoft/vscode-go/pull/2555" rel="nofollow">https://github.com/microsoft/vscode-go/pull/2555</a>
Important recent development in tree sitter was the new query language. Like TextMate or Sublime Grammars, ts in atom did use CSS selectors, but now it has a much more powerful s-expression query language which is useful for more than just syntax highlighting, e.g. static analysis. An application of that is Github's semantic, a haskell tool for code navigation and call graph analysis.<p>Demo and explanation: <a href="https://github.com/tree-sitter/tree-sitter/pull/444" rel="nofollow">https://github.com/tree-sitter/tree-sitter/pull/444</a>
Neovim is aiming to integrate this in the next major release, v0.5: <a href="https://github.com/neovim/neovim/pull/11113" rel="nofollow">https://github.com/neovim/neovim/pull/11113</a>
I've been following tree sitter for a while, as I find the tech super cool and can't wait to see more practical applications.<p>One thing (among many others) that I've found really promising about Dark is its editor. See the hands-on video on their homepage for a demo: <a href="https://darklang.com/" rel="nofollow">https://darklang.com/</a><p>It mostly feels like you're just typing text like in any regular text editor, but your inputs are actually manipulating the AST directly, and the editor itself ensures that your inputs can never result in an invalid program (i.e. there's no such thing as making a syntax error in Dark). It's inspired by tooling in the lisp world like Paredit and Parinfer, but Dark itself doesn't have to _look_ like a lisp because the structure of the AST is maintained by the editor itself instead of by users manually inserting and removing parens. It's an ingenious way to get most of the productivity benefits of a lisp-style syntax and all the structural editing tooling that comes with it, without intimidating new-comers with the super foreign looking parens infested syntax lisps are infamous for.<p>The other day I was actually briefly looking into whether or not it could be possible to replicate something like this in Atom using tree-sitter for some mainstream language like JS, but ended up getting blocked by the fact that Atom doesn't seem to offer an API for plugins to block/replace user input. This is probably for the best, given all the horrible ways this could be abused, but it does mean if I wanted to explore the idea further I'd probably have to either fork Atom to experiment with the idea or build something up from scratch, which is a pretty daunting undertaking given how deceptively complex modern editors can get these days.<p>But maybe I'm missing a different way to accomplish this in Atom with its existing APIs? Or does anyone know if VSCode's extension APIs can support this use case? I realize I've probably barely scratched the surface given how little time I've spent on it so far.
Will tree sitter also stimulate creation of free tools which work on the AST?<p>E.g. it's a mystery to me why we don't have free refactoring tools like the ones in IntelliJ. Like some free library which could extract methods, rename variables, etc. by modyfing the AST. It does not seem too hard.<p>Is it because the current AST parsers are not fast enough or is there some other reason?
So... You write your grammars in Javascript. Which is then serialized to JSON but a parser defined in Rust, so that it can be compiled to C?..<p>That’s... a very roundabout way of doing things.<p><a href="http://tree-sitter.github.io/tree-sitter/implementation" rel="nofollow">http://tree-sitter.github.io/tree-sitter/implementation</a>
The project page is at <a href="https://tree-sitter.github.io/tree-sitter/" rel="nofollow">https://tree-sitter.github.io/tree-sitter/</a>
Discussed at the time: <a href="https://news.ycombinator.com/item?id=18213022" rel="nofollow">https://news.ycombinator.com/item?id=18213022</a>