The action gets stuck checking existing stale PRs without getting to new ones

[mezz]

Describe your issue

I have a long "stale" time of 90 days, and then 30 days to close.
The stale bot uses up all my API actions on iterating over existing stale commits and checking them for comments, and it never gets to apply the stale label to everything that needs it.
I am left with a handful of issues marked stale, but many more need the label.

Your stale action configuration

https://github.com/mezz/JustEnoughItems/blob/def1bfdcba3af124de6fe78fd23ed39c7f1d41a6/.github/workflows/stale.yml#L14-L28

Further context

I think the bot needs a strategy that lets it apply stale labels first, and then check random stale commits for their comments. If the action can persist some data between runs, it would be nice for it to track which issue it left off on so that it can start from there on the next run.

[steveluscher]

Oh no. And that's with only ~200 issues and ~20 pull requests? I'm definitely going to run into this problem then, having ~1000 issues and ~300 pull requests.

[dsame]

Hello @mezz , i thought about different strategies and none of them seems perfect to me. But what do you think about just descending sorting the issues by updated_at every time? This way the issues that were not be labeled go first and are updated to be checked next time last. Do you see any disadvantages of this approaches?

[mezz]

Hello @dsame ,
It is a bit difficult to predict but I think that may at least get things labeled.
I can imagine that the oldest stale issues will not get closed with this strategy.
If you can create a sorting option, I am happy to try it out and let you know if things improve though.

Are you able to store any state between runs, so that it can "resume" when it runs out of actions?

[jklymak]

We are discussing trying this in Matplotlib (>1600 open issues and > 300 open PRs). It's a little unclear what "operations-per-run" refers to - will we be making >1900 operations just to get the state of each Issue/PR? Or is it 20 operations in batches of 100? After that, is each label/close an operation? Sorry if this should be clear - not super experienced with what GH Actions consider a call or not.... Note that https://docs.github.com/en/rest/overview/resources-in-the-rest-api?apiVersion=2022-11-28#rate-limits-for-requests-from-github-actions claims Actions get 1000/h, so maybe we don't need to limit to anything as low as 30... Thanks for any help; hopefully these clarifications are not hijacking this issue.

[dsame]

@mezz in theory it is possible to use the same storage that is used for caches to store the state but it will require extra API calls just to "get into the previous state" - that i want to avoid.

[mezz]

I believe the first priority for the system design should be correctness, and from there we can optimize if we can.
Unless there is a better solution that comes to mind, I am not sure how else this bot can work correctly.
My stale bot has maxed out its API calls for months, unfortunately wasting them without making much progress, so requiring extra API calls seems like a low price.

[worldomonation]

Chiming in with the repo where I mainly work in for my company is again running into this issue. This problem arises when folks use this to mark issues as stale, but not necessarily let the action close it.

I asked a similar question back in early 2021 here: #329.

Numbers:
We have ~4600 active issues, of which 1000 or so are legitimate targets for this action (after taking into account label exemptions).

In each day's action run, it starts with the oldest issue (which has been marked stale for 1+ year now), gradually consuming API calls checking the same issues each time, then halts at around issue 32000. The newest issue is now at ~77xxx, so at the pace we're closing issues (which is manually, not using the action's auto-close feature) the bot will never catch up with the issues being filed.

I'm beginning to close stale issues as it's marked by this action (after a manual review) so it would be interesting to see if that would help the action go further.

[dsame]

Hello @mezz , can you please try to run the action on the main branch to make sure it solves the problem?

- uses: actions/stale@main

[mezz]

Hi @dsame, here are the results of my run with actions/stale@main: https://github.com/mezz/JustEnoughItems/actions/runs/5558711782
It seems to be working correctly, thank you. I can let you know if I run into rate limits with it again.

What has changed in the newer version?

[mezz]

I see a strange error actually, I think things may not be quite right:
https://github.com/mezz/JustEnoughItems/actions/runs/5558711782/jobs/10154121794#step:2:1252

  [#2380] Marking this pull request as stale
  Error: [#2380] Error when adding a label: Resource not accessible by integration
  [#2380] This pull request is now stale
  [#2380] This pull request is already stale

[dsame]

Hello @mezz , has the problem happened again? The error message says about the server-side problem, usually with the permissions, but it does not seem to be a case. Was not it sporadic server issue?

[mezz]

I have this set up to run every hour automatically.
A list of all runs is available here: https://github.com/mezz/JustEnoughItems/actions/workflows/stale.yml

You can see the most recent run here:
https://github.com/mezz/JustEnoughItems/actions/runs/5613747199/job/15210436590
It looks like it still has the same issue present:

  [#2380] This pull request should be stale based on the last update date the 21-05-2022 (2022-05-21T06:22:54Z)
  [#2380] This pull request should be marked as stale based on the option days-before-stale (​[https://github.com/actions/stale#days-before-stale​)](https://github.com/actions/stale#days-before-stale%E2%80%8B)) (90)
  [#2380] Marking this pull request as stale
  Error: [#2380] Error when adding a label: Resource not accessible by integration
  [#2380] This pull request is now stale
  [#2380] This pull request is already stale

[dsame]

@mezz thank you for the quick response. Now i see the workflow lacks pull-requests: write permission, this why it can not modify PR.

Recommended permissions are:

permissions:
  contents: write # only for delete-branch option
  issues: write
  pull-requests: write

Did it help?

[dsame]

Hello @mezz, i am closing the issue due to inactivity, but feel free to reopen it or create the new one in case if the problem still exists.

[timhirsh]

@dsame, this is awesome, thank you for implementing! I would love to start leveraging statefulness. When do you expect the next release to be cut?

[kamaradclimber]

That would indeed be great to get a new release 🙏

[95-martin-orion]

Adding my support to the requests for a fresh release. The top-level README advertises this feature but it is not present in the latest release, which caused issues when I tried to use it.