Should PR testruns expire after a while?

[iritkatriel]

We had a PR that was left open for a few months, and once it was merged the tests failed and the build was broken: python/cpython#22033

I think the PR was deemed by the bot as good-to-merge on the basis of successful testruns from a few months ago.

Should something be done in the bots to prevent this happening again? Perhaps expire the testruns after a while?

[vstinner]

The developer who merges a PR should pay attention to when the CI run for the last time.

Breaking buildbots became really rare. In https://bugs.python.org/issue26053 case, buildbots were fixed quickly, so it's ok. I don't think that we need to change the workflow just because the CI breaks once a while.

It would be great to automatilly re-run the CI with the PR rebased before merging, but it is non trivial to implement. The https://mergify.io/ may be an option for that. Moreover, our CI is not really reliable :-( For example, test_asyncio random failures are sadly common enough for developers to notice.

[hugovk]

At https://github.com/python/cpython/settings if you turn on this:

It adds this optional button to PRs:

If you click that it will merge main into the PR and re-run CI.

Or you can do rebase:

[terryjreedy]

Update with merge commit would be nice. Update with rebase would not.

[JelleZijlstra]

@hugovk's suggestion was implemented, further discussion in #434.