While working on PyTorch, I also wrote an equivalent tool (funnily named nearly the same thing) for doing stack diffs (<a href="https://github.com/ezyang/ghstack/" rel="nofollow">https://github.com/ezyang/ghstack/</a>), which most of our team uses for more complicated PRs. The UX for working on commits is a bit different than this tool though; instead of pushing branches individually, you just run "ghstack" on a stack of commits and it will create a PR per commit in the chain (amending each commit so that its commit message records what PR it corresponds to). To update the PRs, just amend or interactive rebase the original commits. Personally, I find this a lot easier to handle than finagling tons of branches.
I don't like this workflow, I see it as needlessly complicated. If you really can't cherry pick your big PR into simple branches to merge directly into master (which I rarely have seen over the years), create a longer lived branch and make small PRs to that. Then merge longer lived branch into master. You should never be going off and work on huge changes without keeping in sync with the base branch. Stacked PRs make all this more difficult.
The article claims that GitHub doesn't support this workflow, but it actually does a pretty good job natively. In particular, if PR2 targets PR1, merging PR1 into master will automatically change the base branch of PR2 to be master, and PR2 will then be able to merge cleanly. I suspect this might not work as cleanly if you rebase or squash as the merge method, since you're destroying the historical information that git would have used to figure out what's going on.
I follow something very similar, though I do merges starting with F1 -> mainline, then rebase F2 on mainline and merge, then rebase F3 on mainline and merge.<p>This is because the earlier feature branches are usually more "done" and I'm still working on the later ones, so don't want to wait until they are all done to submit PRs and merge them from the tail.<p>I also squash my feature branches before posting PRs, so there's only a single commit, which avoids the rebase issue when branches get new commits. If there's PR feedback for an earlier branch, I'll make the changes, squash (or amend) the commit, then force push that branch. I'll then rebase the later branches to propagate the change. If you have re-re enabled the later rebases are simple.<p>This works well for me, usually I don't go past F2 or F3 in practice, which is small enough # of branches to manage all the rebasing by hand.<p>Overall this is a great doc/explanation, and I love the visuals.
This was really well written, and I loved the visualizations too - I wouldn't have grasped it nearly as quickly without them.<p>This very much reminds of of merged-based-rebasing[1] aka psycho-rebasing[2] (which itself became part of git-extras[3]; Have you seen the concept before?<p>--<p>1: <a href="https://tech.people-doc.com/merging-the-right-way.html" rel="nofollow">https://tech.people-doc.com/merging-the-right-way.html</a><p>2: <a href="https://tech.people-doc.com/psycho-rebasing.html" rel="nofollow">https://tech.people-doc.com/psycho-rebasing.html</a><p>3: <a href="https://github.com/tj/git-extras/blob/1894f453f696720111723043f410437df7aa14b5/man/git-psykorebase.md" rel="nofollow">https://github.com/tj/git-extras/blob/1894f453f6967201117230...</a>
Maybe I am just too dumb but I always found pull requests hard for making small commits. Somehow PRs always ended up being on the bigger end. We even tried these stacked PRs but it was quite confusing.<p>In the end we moved to Gerrit and never looked back. We are not super experienced yet but I'd say our commit quality went up by quite a bit.<p>This is a good introduction to the differences of stacked diffs vs PRs: <a href="https://jg.gg/2018/09/29/stacked-diffs-versus-pull-requests/" rel="nofollow">https://jg.gg/2018/09/29/stacked-diffs-versus-pull-requests/</a>
This article is really well written, and does a great job of introducing the problem all the way through to a really simple solution.<p>I feel something like this might be good as a first class citizen in git (it feels like a common problem, and I don't see how it's solved by some sort of Github feature)
There is an even simpler approach that doesn't involve cherry-pick or rebase.<p>Instead of doing this:<p>```<p>checkout f2<p>git rebase f1<p>push -fu origin f2<p>checkout f3<p>git rebase f2<p>push -fu origin f3<p>```<p>Do this instead:<p>```<p>git checkout f2<p>git merge f1<p>git push<p>git checkout f3<p>git merge f2<p>git push<p>```<p>You would follow this same approach if someone merged a PR ahead of yours into master:<p>```<p>git checkout master<p>git pull<p>git checkout f1<p>git merge master<p>git push<p>```<p>I stack PRs everyday because of the points mentioned in the article, but haven't found a tool which will automatically rollup merges yet (haven't looked either).<p>(edit: formatting)
I dislike the “merge from top to bottom of the stack” flow proposed as the “standard way”.<p>I find that the bottom PRs get unwieldy; you can end up merging a multi-thousand line behemoth f1 containing f2, f3, etc. which is hard to verify as correct.<p>Instead, merge f1 including any changes f1’ you may have force-pushed to that branch. Then do a ‘git rebase <sha> f2 —onto master’ where <sha> is the old f1 commit that you originally targeted your f2 PR against.<p>You’ll need to resolve merge conflicts f1..f1’ into your f2. You can now force push that f2 and retarget your f2 PR onto master.<p>Repeat for f3, f4, ..., though I generally find more than 2-3 stacked PRs to be risky; the further away from master you get, the more likely something is going to change drastically in a PR that you’re branching off.<p>Anyway, this requires a bit more rebase-fu, but I think it leaves a better paper trail and is easier to work with if you do grok rebase fully.
I've considered building systems that would speculatively run builds using forward projections of all pending pull requests, as well as run multiple parallel scenarios on merge order + rebase such that merge conflicts are minimized. Maybe sprinkle in some monte carlo if the numbers get a little intense.<p>In practice, I feel it is much easier to assign work and organize your codebase so that logically-different business functions can be managed in isolation.<p>We rarely have merge conflicts these days, and we are also on 1 big monorepo. It mostly boils down to discipline and architecture.
First, never use --force, but use --force-with-lease -- this will fail to push in case someone make remote changes. Useful if you are collaborating on branches (or develop from more than 1 computers).<p>> It can't pause when a conflict occurs, so you have to fix the conflict and (somehow) re-run it from the point it stopped at, which is fiddly at best.<p>There is better way:
1. I use `git rebase -i` from the top of the stack -- it opens a vim with list of changes it's going to do.
2. I have script (<a href="https://github.com/mic47/git-tools/blob/master/GitStackTodo.hs" rel="nofollow">https://github.com/mic47/git-tools/blob/master/GitStackTodo....</a> ) that process this and inserts commands to backup branch and move branch to new location to TODO list. At this point, I can even modify the TODO list to rip out commits I don't want, or squash commits i want. Or you can reorder commits (I usually do code review fix at top of the stack and then reorder commits -- at least if I am reasonably sure there won't be conflicts for this fixes).
3. At this point, you can insert more things, like run tests after each branch, or commit (and pause rebase in case of failure, so you can fix it).
4. When I close this file, rebase starts. In case of conflict, rebase pauses, let you fix it, and when you continue rebase, it will finish the TODO file.
5. After, I have script that removes backup branches.
6. I have script that runs command on each branch, so at the end, I do this to push my changes `git stack-foreach master git push --force-with-lease `<p>What if you can't resolve conflict and you are in the middle of the rebase? You can run `git rebase --abort` and it will restore top of your commit. Only drawback is that branches that were rebased are not restored, but hence my script also create backup branches so I can fix that manually and move branches back.
I highly suggest reading my article on How we should be using Git. It covers a Git Patch Stack workflow, where it originated from and the tooling we built around it.<p>It has important ties to how the Linux Kernel and Git dev teams work as well as breaks down the benefits in relation to CI as a methodology.<p><a href="https://upte.ch/blog/how-we-should-be-using-git/" rel="nofollow">https://upte.ch/blog/how-we-should-be-using-git/</a>
Don't love it. Would much rather:<p><pre><code> git push origin f1:f1
git checkout f2
git merge f1
git push origin f2:f2
git checkout f3
git merge f2
git push origin f3:f3
</code></pre>
Merge is generally less effort to handle conflicts too. I see cherry-pick as a last-resort option for fixing certain complicated scenarios.
Great visuals and nice write up!<p>As others have noted already, I like the git-ps approach (<a href="https://upte.ch/blog/how-we-should-be-using-git/" rel="nofollow">https://upte.ch/blog/how-we-should-be-using-git/</a>)
I'm still reading the article, but it looks like there's a typo where it refers to:<p><pre><code> Merge in the stack from top to bottom - in this case we're
going to merge Feature 3's PR, Feature 2's PR, and Feature
1's PR, in that order, using GitHub's UI.
</code></pre>
But the "top to bottom" would be merging F1, then F2, then F3 (and you'd have to rebase & re-upstream each as you go).
Nice article. Reminds me of a similar approach and tool called git-ps - <a href="https://github.com/uptech/git-ps" rel="nofollow">https://github.com/uptech/git-ps</a> .<p>Write-up:
<a href="https://upte.ch/blog/how-we-should-be-using-git/" rel="nofollow">https://upte.ch/blog/how-we-should-be-using-git/</a>
I really like seeing efforts being made in this direction. For the last few years I've been using git-town.com, which works quite well but I often end up having to work around it (especially when I have diamond-shaped branch dependencies, e.g. to refactor two independent things before implementing a small feature on top). I'll have to try this instead and compare!
> Changes that build on the original either go in the same PR, or have to wait until the PR is merged so a second PR can be created.<p>For feature branches feature-2 that's based on feature-1, why not create a PR from feature-2 onto feature-1 to get the review started? Once feature-1 is merged, simply rebase feature-2 and update the PR. Pipeline that shit, baby!
I've been using stgit[1] for this type of workflow for last 10 years. It allows to think in terms of atomic patches and transparently rebases all changes for you. Also it plays nicely with Gerrit.<p>[1] <a href="https://stacked-git.github.io/" rel="nofollow">https://stacked-git.github.io/</a>
I use <a href="https://git.sr.ht/~krobelus/git-branchless" rel="nofollow">https://git.sr.ht/~krobelus/git-branchless</a>