Tvix: We Are Rewriting Nix

I don&#x27;t know much about Nix, although it has been on my radar for some time. The claim that the language performance is bad pops up time and again, and I&#x27;m sure many people would be happy to have a faster alternative --- so props to them for trying to address this clearly important problem.<p>That being said, in perusing related issues on GitHub such as [1], I have found no serious attempts to even profile the damn thing. The typical attempt seems to be throwing a random idea at the evaluator and crossing fingers. The example of a &quot;performance experiment&quot; [2] from the article is self-described as follows:<p>&gt; Add an alternative impl of the now-abstract Bindings base class that is backed by a std::vector, somewhat similar but stylistically a little superior to the array-backed implementation in upstream nix.<p>I fail to see how that is a performance experiment when it does not mention performance at all.<p>The post also mentions issues with the general Nix design and a desire to modernize it, which I am unqualified to comment on, but seems a worthwhile endeavor in and of itself. I do hope that they won&#x27;t end up writing a new evaluator for the Nix language without a proper understanding of what makes the current one slow and a clear plan not to repeat its mistakes, or the new one risks being just as slow as the old one.<p>[1] : <a href="https:&#x2F;&#x2F;github.com&#x2F;NixOS&#x2F;nix&#x2F;issues&#x2F;2652" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;NixOS&#x2F;nix&#x2F;issues&#x2F;2652</a><p>[2] : <a href="https:&#x2F;&#x2F;cl.tvl.fyi&#x2F;c&#x2F;depot&#x2F;+&#x2F;1123&#x2F;" rel="nofollow">https:&#x2F;&#x2F;cl.tvl.fyi&#x2F;c&#x2F;depot&#x2F;+&#x2F;1123&#x2F;</a>

I really wish there was a statically typed alternative to nix. Nix docs are horrible and incomplete and the language is called the same as the rest of tye ecosystem so it&#x27;s hard to google. I don&#x27;t think there can be a good declarative functional language without static types because how the hell do you debug? I&#x27;ts been really hard whenever I have to do something in a bigger nix environment. I get the same feeling like when I jump into a bigger Python project which isn&#x27;t documented and I have to read around to see what is what, only goto definition doesn&#x27;t work, the language is weird and everything is some specific function which I don&#x27;t know is built in or where it&#x27;s from and googling gives no answers except maybe link to source code

I don&#x27;t know enough about Nix internals to know how much of a problem this is, or the right way to improve it.<p>However, it&#x27;s good to see an ecosystem forming around the derivation data type, with the possibility of reuse across projects that use derivations.<p>Good stuff!<p>I&#x27;d be interested to hear a perspective from the Nix and Guix authors.

What would seem most useful would be a compile-time-typed language (perhaps with some inference propagation, for convenience) that existing Nix code could be translated into, maybe even while emulating execution to discover the actual types to annotate with. You have the advantage that all existing production Nix code is known to terminate in short order.<p>Then, you could generate a new version of the entire 30,000-strong package collection automatically, with the new language swapped in in place of the Nix code, which could then be abandoned.

OT for nix, but that&#x27;s an interesting approach to building an organisation on the internet. They seem to have a good sense of cohesion despite being just a group of people with common interests who hang out on an IRC and are distributed across the world?

Hey Tazjin, i see you are around that one.<p>Quick question. Can you expand on IFD and integrations? That is one thing i would love to know more about your plans.<p>Also i hope it works, the nox code base is imho the biggest impediment to the nix model being more used.<p>Small advise: please consider making it really easy to contribute to through wll the tooling. This is one of the reason Nix codebase rotted

Is tvix&#x2F;nix pretty tightly coupled to the Linux kernel? I know there is some effort to use the nix package manager on OSX, but I&#x27;m not sure if that is a hack or an actual full port to a different kernel. Could it be used on, say, OpenBSD? SeL4?

So this sounds like it is mostly concerned with the derivation part. Is it going to be based on Nickel then, or a new custom parser? Would be interested in the rationale there

There are noble goals to this project to rewriting Nix to improve performance. The reasons feel valid. However I wonder about certain issues:<p>- Nix itself has problems of compatibility with Nix! Let me explain...recently NixOS 21.11 released. It still comes packaged with Nix 2.3.16 instead of Nix 2.4 because of incompatibility issues. When the Nix project itself is not able to deliver backwards compatibility I feel uncertain that Tvix might be able to deliver their promise of full compatibility with all the deep architectural changes it plans. Nix does not have a formal specification: Nix is what its C&#x2F;C++ implementation says. Nix also has deep architectural changes planned going ahead. So Tvix will need to re-architect Nix keeping in mind (and keeping up) with the future changes that are coming. It&#x27;s going to be tough to build this &quot;alternative&quot; implementation. Users are going to run into weird compatibility issues, not have the patience to resolve them and then just go back to the original Nix implementation.<p>- A rewrite to a popular project will find it difficult to succeed. There is simply too much momentum in the parent Nix implementation (I&#x27;m not talking about the package definitions here) that very few people are likely to adopt this rewrite. This is because most people might not really need the speedup. However, it&#x27;s possible that the pain around slow evaluation runs so deep that other companies might adopt this too. I don&#x27;t know enough here.<p>- The rewrite needs to offer more reason to shift. Right now this rewrite seems to be mainly about performance. There is also a proposal to be able to integrate with GNU Guile. To me this is just avoiding one untyped language (Nix) to replace with another slightly better untyped language (Guile Scheme) (GNU Guix uses Guile also AFAIK). What about introducing typing? <a href="https:&#x2F;&#x2F;github.com&#x2F;tweag&#x2F;nickel" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;tweag&#x2F;nickel</a> is an interesting project and it would be great to have a system like Nix with optional typing to make writing the derivations more robust. What about rewriting in a safe programming language (e.g. Rust or a GC-ed language)? These things would be likely to excite would-be-contributors and would make your rewrite viable. In other words, the rewrite needs to offer more than just the promise of performance.<p>Perhaps I have not understood the project goals fully. Maybe the project is all about just being able to leverage the Nix package definitions but have a totally different implementation to instantiate them? But the issue here is that nix-the-language is very intimately tied up with the nix-the-platform and the rewritten implementation will still need to really be extremely compatible to capture all the implicit assumptions in nixpkgs.<p>Once again, improving nix through a rewrite is a noble goal. It&#x27;s a tough problem and it would be awesome if the project succeeds!<p>I have obviously not thought about this as much as you guys and would love to learn how you wish to tackle some of these problems. Maybe some of these issues are not as big as them seem??

How do you pronounce &quot;Tvix&quot;?

Every distribution has the exact same problems that are impossible to prevent. It&#x27;s all about the inherent limitations of how software is developed and run in operating systems today. All this stuff about &#x27;solving dependency hell&#x27; and dependency paths tied up with merkle trees doesn&#x27;t change the fact that you can&#x27;t use two completely different libraries in one application at the same time, nor shell out to two incompatible versions of a binary from the same application. It is only possible if the software itself is designed to do that, using the features of a dynamic linker that allow linking in multiple versions of the same function and specifying which version you&#x27;re calling from what source function. We don&#x27;t actually need a new package manager or distribution at all, because they can&#x27;t solve one of the most fundamental problems - only developing software differently can do that.<p>Sorry, but Nix is a total waste of time.