The current CI requires rebasing to main far too often

[potiuk]

Body

I was wondering why people have to rebase their PRs so often now, this was not the case for quite a while, but it seems that now, the need to rebase to latest main is far more frequent than it should be. I think also we have a number of cases where PR build succeeds but it breaks main after merging. This also did not use to be that often as we observe now..

I looked a bit closer and I believe the problem was #15944 PR streamlined some of the code of our CI I just realised it also removed an important feature of the previous setup - namely using the MERGE commit rather than original commit from the PR.

Previously, when we built the image, we have not used the original commit from the incoming PR, but the merge commit that GitHub generates. Whenever there is no conflict, GitHub performs automatic merge with main and by default PR during the build uses that 'merge' PR and not the original commit.

This means that all the PRs - even if they can be cleanly rebased - are using the original commit now and they are built "as if they were built using original branch point".

Unfortunately as far as I checked, there is no "merge commit hash" in the "pull_request_target" workflow. Previously, the "build image" workflow used my custom "get_workflow_origin" action to find the merge commit via GitHub API. This has much better user experience because the users do not have to rebase to main nearly as often as they do now.

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.

[potiuk]

cc: @ashb