-
-
Notifications
You must be signed in to change notification settings - Fork 84
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GitHub rate limit triggered when syncing a lot of repositories #49
Comments
Hey @grahammcculloch, thanks for creating the Issue!
The GitHub PAT was probably rate-limited.
What would the We could also automatically throttle the number of requests when we encounter the rate limit. I will have to take a closer look at GitHub's Rate Limits, maybe they return a value indicating when we can make a request again. I don't think we would need the What do you think? |
@BetaHuhn yeah, I agree with your analysis. The GitHub docs on 'dealing with abuse rate limits' are quite useful with their suggestions. I guess you could handle repos serially instead of concurrently. Or you could enqueue all GitHub requests through a queue that throttles them to wait at least a second between requests. I guess this is an edge-case. But presumably, others will have similar issues in due course. |
Thanks for taking a closer look @grahammcculloch! It might be an edge case, but as you mentioned more people are probably going to run into it. This definitely needs to be fixed. I will take a look at your proposed solutions and will implement one of them as soon as I have time this weekend. |
Awesome - appreciate it @BetaHuhn. Do ping me if you have something you want me to test. |
I implemented a fix on the develop branch. The action now uses plugin-throttling.js to throttle API requests based on the mentioned recommended request throttling best practices. I also refactored (ec2d589) the GitHub part of the action to simplify the code, but that shouldn't have any effects on the action itself. @grahammcculloch you can try it out by using the action like this: uses: BetaHuhn/repo-file-sync-action@develop I don't have that many repos available right now to test this myself, so please report back if you are still hitting any rate limits. |
@BetaHuhn awesome - I will test this out today or tomorrow and get back to you with the results! |
@BetaHuhn no problems running with the develop branch - although I can't prove that it hit the rate limit and backed off :-) |
Great, I will merge it into master and release a new version! |
🎉 This issue has been resolved in version 1.7.1 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
🐞 Describe the bug
When executing the
repo-file-sync-action
in my workflow, it shows as successfully having completed but the logs show multiple instances of the following error:Each time this error is reported, the PR fails to be created. For example:
No PR is created on the
jellyfish-queue
repo.📚 To Reproduce
Create a sync.yml file that should result in PRs being created on a lot (about 20) of repos.
💡 Expected behavior
No errors logged, all expected PRs successfully created.
🖼️ Screenshots
N/A
⚙️ Environment
📋 Additional context
I'm not sure if this means the GitHub user associated with
GH_PAT
has been rate-limited or whetherrepo-file-sync-action
or the workflow itself has been rate-limited?Either way, I wonder if it would help to add a
RATE_LIMIT
option or similar that could be used to throttle the action's requests to the GitHub API to mitigate this error?The text was updated successfully, but these errors were encountered: