-
Notifications
You must be signed in to change notification settings - Fork 16.3k
Fix automatic backport workflow race condition #58705
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
c273fff to
a91c2de
Compare
jscheffl
approved these changes
Nov 26, 2025
Contributor
jscheffl
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sleep is not cool but also no better idea. Thanks for the investigation! Wow this is complex :-D
Contributor
Author
|
@jscheffl I think I can also get it done by using retries in the API calls. But that will probably be more complex and time consuming to do |
Lee-W
approved these changes
Nov 26, 2025
vatsrahul1001
approved these changes
Nov 26, 2025
github-actions bot
pushed a commit
that referenced
this pull request
Nov 26, 2025
(cherry picked from commit 1aa9b7a) Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
github-actions bot
pushed a commit
to aws-mwaa/upstream-to-airflow
that referenced
this pull request
Nov 26, 2025
) (cherry picked from commit 1aa9b7a) Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
potiuk
pushed a commit
that referenced
this pull request
Nov 27, 2025
ephraimbuddy
pushed a commit
that referenced
this pull request
Dec 3, 2025
itayweb
pushed a commit
to itayweb/airflow
that referenced
this pull request
Dec 6, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The automatic backport workflow was intermittently skipping PRs that should be backported, even when they had
backport-to-*labels. After some investigation, I could see a race condition with GitHub's API.Examples:
For successful runs:
Run actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea with: script: const { data: pullRequest } = await github.rest.repos.listPullRequestsAssociatedWithCommit({ owner: context.repo.owner, repo: context.repo.repo, commit_sha: process.env.GITHUB_SHA }); if (pullRequest.length > 0) { const pr = pullRequest[0]; const backportBranches = pr.labels .filter(label => label.name.startsWith('backport-to-')) .map(label => label.name.replace('backport-to-', '')); console.log(`Commit ${process.env.GITHUB_SHA} is associated with PR ${pr.number}`); console.log(`Backport branches: ${backportBranches}`); core.setOutput('branches', JSON.stringify(backportBranches)); } else { console.log('No pull request found for this commit.'); core.setOutput('branches', '[]'); } github-token: *** debug: false user-agent: actions/github-script result-encoding: json retries: 0 retry-exempt-status-codes: 400,401,403,404,422 env: COMMIT_SHA: c03fc7943009b02072ad275dd31e54f927bff889 GITHUB_TOKEN: *** Commit c03fc7943009b02072ad275dd31e54f927bff889 is associated with PR 57853 Backport branches: v3-1-testFor failed runs:
1s Run actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea with: script: const { data: pullRequest } = await github.rest.repos.listPullRequestsAssociatedWithCommit({ owner: context.repo.owner, repo: context.repo.repo, commit_sha: process.env.GITHUB_SHA }); if (pullRequest.length > 0) { const pr = pullRequest[0]; const backportBranches = pr.labels .filter(label => label.name.startsWith('backport-to-')) .map(label => label.name.replace('backport-to-', '')); console.log(`Commit ${process.env.GITHUB_SHA} is associated with PR ${pr.number}`); console.log(`Backport branches: ${backportBranches}`); core.setOutput('branches', JSON.stringify(backportBranches)); } else { console.log('No pull request found for this commit.'); core.setOutput('branches', '[]'); } github-token: *** debug: false user-agent: actions/github-script result-encoding: json retries: 0 retry-exempt-status-codes: 400,401,403,404,422 env: COMMIT_SHA: caafc704462cc15bedde67ef416d870ceabd13e8 GITHUB_TOKEN: *** No pull request found for this commit.The root cause seems to be that when a PR is merged to
main, the workflow immediately queries GHlistPullRequestsAssociatedWithCommitAPI to find the PR and check for backport labels. However, this API call seems to need time to process the merge and associate the commit with the PR.Performed analysis on couple of PRs and this is my finding:
To solve this, I have two options:
Add some sort of retry mechanism while calling the API. Complex implementation but could work.
[Chosen Now] Added a
15second delay before querying the API. This gives the backend sufficient time to process the merge commit and establish the PR association, ensuring backport labels are consistently detected.Why 15 seconds?
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.