The issue is with how GitHub (and most PR / CI systems...) relies on stale CI results.
Master/Main changed between the tests running for that PR, and it merging. Why should those test results be considered a useful indicator for what happens after the PR merges anymore?
GitHub does have a setting you can enable to help with this, essentially, when any PR merges, it forces the developer to rebase any open PRs. A huge waste of effort, so nobody turns it in.
There are far better systems out there, e.g. zuul-ci.org which was designed to handle this exact problem at scale. Others have mentioned GitLab merge trains, though I've never used them - so hard to tell if they really solve this :)
You can actually clone/checkout a "merged" PR provided by Github:
git checkout --force refs/remotes/pull/119/merge
They generate it async behind the scenes, so it may not be immediately available after your PR is opened, and changes to master may be delayed.
Maybe the difference is whether you consider each atom to be a releasable candidate, or is there is another release layer (a-la good old “a successful git branching model”)
I’m confused because this isn’t stated clearly in the comments, but shouldn’t you create a merge commit (for the feature branch) that combines both master and feature branch code before running the tests? And then change master to point to your newly created merge commit once tests pass?
Actually one should treat changes as totally ordered before merging them.
But that brings a choke point into your workflow:
* Change A developed and good for master
* Change B developed and good for master
* Change A merged into master
* Change B not good for master any more, gets pushed back
* Change C developed and good for master
* ...
In principle, a change can be put back forever, depending on your practical situation.
You could use a merge queue, that does an rebase on latest master, then tests your code and merges.
Another point in favor of rebase not merge.
> Should CI create a merged branch behind-the-scenes and run tests on that before allowing the branch to be merged?
This is surprisingly difficult to get right, especially with projects with lots of concurrent changes. Gitlab merge trains[1] really help here.
1. https://docs.gitlab.com/ee/ci/merge_request_pipelines/pipeli...