Multiple links to the same PR

[Mariatta]

This was opened in python/core-workflow#224, but it is more appropriate here:

🍝 (copy pasta)

In bpo-32856 you can see that the bot created two links to the same PR python/cpython#5695. I suppose this happened because I fixed a typo in the title just after creating the PR. But I wondering if there is a race condition which can cause more several errors.

[JulienPalard]

Happened again here: https://bugs.python.org/issue35579 with 4 links to the same PR.

[Mariatta]

It could be related to the bpo maintenance recently, where Ernest had to replay all of the failed webhook deliveries. python/core-workflow#301

[terryjreedy]

If you hit a link [edit] button, you can unlink, but I don't know if just 1 or all 4 would be affected. The duplicate merge reports can be deleted 1 by 1 if you want to clean the issue for future viewers.

[serhiy-storchaka]

Just 1 will be unlinked. You need to unlink them 1 by 1. I did this many times, and it is tiresome.

[ned-deily]

AFAICT, this now seems to be happening for all PRs with b.p.o references, with 3 identical PR links being created. See, for example, https://bugs.python.org/issue35614. @ewdurbin @ezio-melotti

[gpshead]

fwiw, it isn't always three, i believe I've seen four happen today.

[ewdurbin]

It appears to be a race condition on these statements: https://bitbucket.org/python/roundup/src/45440a8172b155875eb4daecd44104abe03b49d3/roundup/github.py#lines-200:201

https://bitbucket.org/python/roundup/src/45440a8172b155875eb4daecd44104abe03b49d3/roundup/github.py#lines-228

If the handler is slow, and multiple web hooks are being processed simultaneously for the same PR they each helpfully create the object in the database.

Investigating if there is a reasonable way to handle this.

[Mariatta]

Is this something that will go away if we start using GitHub issues?

[tirkarthi]

I guess this also affects keywords. After a PR is added to the issue every message repeats the keyword patch thrice per message in the activity log. Example : https://bugs.python.org/issue35603 .

[ewdurbin]

This is a rough attempt at resolving this: https://bitbucket.org/python/roundup/pull-requests/1/attempt-to-handle-pr-race-condition/diff

[vstinner]

I will follow this issue since the bug is annoying. It sends me useless email notifications and not all duplicated PRs get updated when a PR is merged or rejected.

[tirkarthi]

Is this possibly due to the extra workers that were spin up during the maintenance causing duplicate PRs to be created always since the race condition exists for a very long time? Subscribing to python-bugs-list sends a lot of duplicate notifications and emails for PRs sometimes causes actual messages to be overlooked if there are multiple backports created at the same time for the issue.

[pitrou]

@ewdurbin If pull_request.setkey('number') creates a unique key (I'm not acquainted with the Roundup DB layer), then the solution looks reasonable on the principle (and pretty much idiomatic, IMHO). But you probably shouldn't re-do prs.append(pr) if the PR was already there?