Josh: Get the advantages of a monorepo with multirepo setups

IANAMU (I Am Not A Monorepo User), but as far as I understand monorepos and their advantages, I&#x27;m not sure what the use-case for this tooling is.<p>Most of the time, when an org chooses to move to having a monorepo (rather than just being left with one by accident of history), the key advantage they&#x27;re striving to attain, is the ability to make changes to cross-cutting concerns across many distinct applications&#x2F;libraries, with single commits&#x2F;PRs. To change an API, and <i>all</i> of its internal callers, atomically, without having to worry about symbolically binding the two together with dependency version constraint resolution.<p>Which is to say, the key advantage of a monorepo comes from having the whole monorepo checked out locally.

From the title I expected it to be a tool for treating multiple separate repos as though they were all just one single monorepo. But from the description in the README, it seems to be for treating subsets of a monorepo as though they were separate repositories.<p>PS: The title, in case it is changed, is currently “josh: Get the advantages of a monorepo with multirepo setups”

Like some people, I was expecting to find a way to have the advantages of a monorepo while having projects in separate repos. This is something Bloomberg is doing, and it&#x27;s very cool. Each project is a separate repo, but they have a central integrated &quot;repo&quot; with all the repos, which is the &quot;source of truth&quot;, and were code is built and deployed from. You can commit changes in your repo, and then you &quot;release&quot; the code into the integrated repo, which will rebuild all the transitive dependencies and run their tests to make sure everything still works. If anything fails, your release of the code is not merged in the repo. I&#x27;m now working with a monorepo, and I much prefer the Bloomberg approach. Cross repo changes can be made atomically (you update the reference in the integration repo for multiple individual repositories at once), and that is usually the big sell point of monorepo. And it doesn&#x27;t have the downsides of the monorepo. The only issue is that it&#x27;s not very ergonomic, and there isn&#x27;t a tool to make that easy. But building such a tool is definitely easier than implementing a virtual FS as it has been done in multiple companies.<p>I&#x27;d love if someone still working there were to write a nice post about that system, it was the first of such a kind I saw.

I&#x27;ve come to learn that git monorepos are becoming quite popular. What I don&#x27;t understand is why people are using git for this kind of workflow. It forces you to actively work against git&#x27;s design goals and implementation. Which then compels the use of several odd workarounds and kludges to kind of seemingly reassemble a half-baked flavor of subversion. Why not just use a tool designed around monorepos and subtrees? I&#x27;m genuinely curious. I assume I&#x27;m missing something.

I think this is quite exciting, as it solves a major unsolved problem for large git monorepos: enabling development or CI&#x2F;CD inside a git monorepo without requiring a large checkout.<p>As monorepos grow huge, this comes to be very costly or even prohibitive, and companies like Google simply don&#x27;t use Git.<p>Here are some problems with alternative approaches that have been mentioned:<p>* VFS for Git: I believe abandonded by MSFT in favor of improved client-side tooling: <a href="https:&#x2F;&#x2F;github.com&#x2F;microsoft&#x2F;VFSForGit&#x2F;blob&#x2F;master&#x2F;docs&#x2F;faq.md#why-are-you-abandoning-vfs-for-git" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;microsoft&#x2F;VFSForGit&#x2F;blob&#x2F;master&#x2F;docs&#x2F;faq....</a> .<p>* Sparse checkout: limits ability to use a build system to dynamically find any dependencies and rebuild them<p>* Submodules: can&#x27;t atomically update both the parent and the child repo, have to manually update the referenced commit of the child repo in the parent repo, and each collaborator must manually update their child repo when the commit changes

How does this compare to partial clones and sparse-checkout? This question was raised as an issue in the project but was closed as &quot;not really an issue&quot;, which I guess is true.<p><a href="https:&#x2F;&#x2F;github.com&#x2F;esrlabs&#x2F;josh&#x2F;issues&#x2F;23" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;esrlabs&#x2F;josh&#x2F;issues&#x2F;23</a>

A lot of the comments here are surprisingly dismissive. I think having the ability to project parts of your git repo (still with a normal git api!) is an incredibly useful feature. Take the example of DefinitelyTyped: the maintainers can just do all the things they want in one repository which _vastly_ reduces the development overhead, but consumers of that code can use it however they please. If I understand correctly, you could have a submodule reference work out of the box but to a subset of that repo that you care about - seems pretty damn cool to me!

This can be kinda achieved with git submodules.<p>I’m currently using a monorepo with submodules and it works really well.<p>Dependency management is not a huge issue too, at least if you don’t have hundreds of submodules.

The problem with trying to force one single history, is that it ignores deployments. And user onboarding&#x2F;behavior change. All of which can be relevant when working on a project.<p>With multi repository projects, this helps some thinking, as it is clear that the changes were not atomic between systems. They are literally separate at all layers, including the commit.<p>I sympathize with wanting a simpler view. I&#x27;m just worried on an inflated value proposition.

I might be reading this wrong, but it seems like this creates a second place where a project would need to express dependencies between modules.<p>You would need to do it for JOSH and whatever your build tool is. You&#x27;d probably need some additional git commit hooks to ensure your build tool of choice config is in sync with josh.

There are a few (re)solutions in the wind. The latest one that I&#x27;ve known is `west` (part of Zephyr-RTOS project), but I haven&#x27;t tried yet.<p>There may have wrong description (FIXME) but a sort list is found here: <a href="https:&#x2F;&#x2F;github.com&#x2F;icy&#x2F;git_xy#why" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;icy&#x2F;git_xy#why</a>

This seems to provide the same functionality as using Git Sub Modules[1].<p>Am I getting the correct impression?<p>[1] <a href="https:&#x2F;&#x2F;git-scm.com&#x2F;book&#x2F;en&#x2F;v2&#x2F;Git-Tools-Submodules" rel="nofollow">https:&#x2F;&#x2F;git-scm.com&#x2F;book&#x2F;en&#x2F;v2&#x2F;Git-Tools-Submodules</a>

I don’t see what problems this solves, but I do see plenty of problems it could introduce. There are tremendous benefits to sticking with the tried and tested approach supported by git rather than introducing yet more tooling.

They definitely need a long FAQ. I&#x27;m sure it&#x27;s different than submodules <i>somehow</i>, but in what ways&#x2F;circumstances&#x2F;purposes, I have no idea.

I have yet to hear anyone give me a coherent explanation of why a monorepo is better.

Before the traditional HN comments &quot;isn&#x27;t this just ...?&quot;, the README already does that for you:<p>&gt; a blazingly-fast, incremental, and reversible implementation of git history filtering

That looks nice. It must have some major drawbacks? Sounds too good to be true…

Having been to the both sides of it I can say there exist exactly 0 advantages of a monorepo setup.

Most of you don’t need a monorepo, the same way most of you don’t need, well, half the shit peddled in the tech Instagram (conferences, meetups, mediums, blogs, hn).<p>You just don’t need that stuff, there’s like 20 of you on a team and at best your app probably sucks and barely has users, and if it does have users, it’s probably some trivial bullshit.<p>You’re all a bunch of ordinary folks, so stop fucking up the workplace with your identity crisis. No, you are not an elite engineer, you are Bob, the guy who goes home every day and watches Netflix&#x2F;plays video games.