Cron job to close stale PRs

[brettcannon]

I'm classifying "stale" as someone who created a PR but has not signed the CLA or someone who received a code review requesting changes and and never followed up with more commits or review comments. These two scenarios specifically make sure that the PR submitter is the hold up and not any core dev(s). (A PR that can't be merged probably shouldn't be classified as such since they shouldn't have to perpetually maintain a PR if they don't get a review from a core dev).

It would probably be nice to receive a warning that a PR will be closed within a week, but re-opening a PR I don't think is hard either so it should be a big deal to just close either if it's too much work to give the warning (that would need to be verified, though, that a PR submitter and re-open their own PR before making a decision about this).

[matrixise]

+1. on today, there are 17 open PR where the CLA is not signed.

[souravsingh]

@Mariatta I am interested in working on this.

[Mariatta]

@souravsingh Sure! I think this is not a straight forward task, and it has some fun challenges :)

My initial thoughts about this:

A PR is considered "stale" if:

• the last activity on it has a timestamp older than X days (1 month? 2 months? 3 months?)

and

• it has CLA not signed label, or
• it has awaiting changes label

The workflow:

1. bedevere applies the stale label to PRs with above criteria
2. bedevere leaves a comment saying that this PR will be closed unless changes are made.
3. if a PR has the stale label and there has been no change 7 days after the label has been applied, then bedevere will just close the PR. Not sure how to detect this. Perhaps by retrieving all the comments from the PR, and see the timestamp when the comment from step 2 was made?

Side task, I think it can be a separate task. It will be a webhook instead of a cron.

• Remove the stale label when awaiting changes or CLA not signed label has been removed.

I think the code can be added to bedevere, so we don't need to create any new bot.

[brettcannon]

My one worry with adding this to Bedevere is GitHub quota. Going through 453 issues to find out details about them is going to eat through quota rather quickly and might be too much with Bedevere's current quota usage.

[Mariatta]

Hm, the rate limit is something I haven't considered.
If we were to create a new app/bot for this, will it also be at risk of hitting the rate limit?

[brettcannon]

Quite possibly. We might not know until we do the queries and see what the count comes out to. This might require smearing the work across the day, e.g. start at midnight, find out how many pages of issues there are, and then do a page per hour (where hopefully there are less than 24 pages 😉 ).

[Mariatta]

For the first task of applying the stale label, (is stale the right term?), the bot won't be going through all open PRs, only the ones with CLA not signed label (2 pages) or awaiting changes (1 page) labels.

The API calls will be:
https://api.github.com/repos/python/cpython/issues?labels=CLA%20not%20signed&sort=updated&direction=asc&state=open
and
https://api.github.com/repos/python/cpython/issues?labels=awaiting%20changes&sort=updated&direction=asc&state=open

[brettcannon]

Nice catch on the GET arguments! So it looks like 40 issues in total ATM are either labeled "CLA not signed" or "awaiting changes". But the problem is that out of 19 pages worth of issues, only the first 5 have any "awaiting" label really applied to them. So that means the 17 issues that are awaiting changes could be extrapolated to be about 68 (although possibly more since those PRs are probably/hopefully languishing for a reason), which puts it at 90 issues or more (aside: maybe we should write a script that we run once that goes through and does a back-fill of all issues without an "awaiting" label? We could have a custom message when we set "awaiting changes" saying "we're totally guessing here, so please say 'I didn't expect the Spanish Inquisition' if we're wrong" or something).

So let's worst-case it to 200 issues. If a PR is now considered stale/inactive, then there will be a comment and an added label (so 2 requests). If a PR has passed it's one week grace period, then it will be a comment and closure (2 requests). Add in pagination on the issue list and we're looking at worst-case 400+ requests out of our (I believe) 5000 request quota, so we're approaching 10%.

I have opened python/bedevere#64 so that we can see what kind of margin we currently have so we don't have to keep guessing.

[chetankm-cs]

Hi,
I am working on this, I have added the code for labelling stale pr.
I need help on setting a cron for this.