The very idea of having compilation unit boundaries look like project boundaries squicks me :/... the lightweight compilation unit structure C/C++ has--with the ability to elide information but the requirement that it be prototyped--frankly always seemed like the perfect tradeoff, and it generally means that projects are able to be compiled in massively parallel environments as all of the dependencies are explicit. This is why distcc is such a trivial project to just drop on almost C codebase, and is why I have been able to get our builds doing like 100x parallel compiles on AWS Lambda (and like, before someone tries to claim that header file bloat somehow prevents this: it doesn't, and build engineers do massively parallel build farms for C/C++ all the time, and it is only in some specific corner cases of C++ projects defined entirely as templates where this comes up, and even then there are trivial techniques you can use to help break the translation units apart). Every time a new language comes out and tries to "fix" translation units by adding tons of cross-file information dependencies without the explicit definition vs. declaration difference I kind of roll my eyes as I know there's almost no chance the build is ever going to fast, and these "we need all the code at once to even begin to analyze it" languages are the worst :(.
>In my experience though projects tend to start in a single crate, without great attention to their internal dependency graph, and once compilation time becomes an issue, they have already created a spaghetti dependency graph that is difficult to refactor into smaller crates.<p>This happened to me too. One important problem to overcome is the orphan rule. This means: you can only implement a trait you own or for a type you own. If both the type and the trait are not in your crate, you are out of luck.<p>So my first attempt failed utterly. It was very disappointing. You see, by splitting the project in different crates I put one type and one trait in different crates.<p>After thinking about this problem for a few weeks I accepted that I needed to write wrappers, so called newtypes. But this hurt a lot. I managed to abstract away a lot of the boilerplate by writing macros but still am not very happy about the complexity.<p>Growing pains.
Its not just a Rust problem, I've run into the same thing in Java and I'm sure it exists elsewhere. The only answer I can come up with is to keep your app cleanly divided into modules. There's some cognitive overhead but its usually worthwhile.<p>I recently divided a 100k line Java app into 5 different modules and it compiles 4 times faster. The new boundaries have encouraged better code as well. Suddenly we have things like interfaces and IoC. Before, people just stuffed things wherever.
Just for the record, orphan instances, although possible in Haskell, are regarded as a big problem and something that should be avoided as much as possible. The problem being, you can declare an orphan instance yourself, later discover you need another library, only to discover it implements the orphan instance as well, preventing compilation. Worst case scenario is that the two implementations are semantically different and you no longer have any idea what your code should do.<p>(Orphan instances in apps are horrible, orphan instances in libraries run the risk of making your whole library unusable.)<p>As useful as they sometimes are, I can't help feeling Rust made the right call here.
In Haskell two libraries that define conflicting orphan instances can't be linked together. Given the culture around code sharing and crates.io, this would probably be a big problem for Rust. (As I recall I was the first to propose forbidding orphan instances, although at the time I didn't realize they were called that.)
>In my experience though projects tend to start in a single crate, without great attention to their internal dependency graph, and once compilation time becomes an issue, they have already created a spaghetti dependency graph that is difficult to refactor into smaller crates.<p>Maybe there should be a lint for not having modules be mutually dependent?
The URL of this article was changed to the following (so the one linked on Hacker News as of when I posted this comment is now a 404).<p><a href="https://pingcap.com/blog/Rust-s-Huge-Compilation-Units" rel="nofollow">https://pingcap.com/blog/Rust-s-Huge-Compilation-Units</a>