API rate limit exceeded for installation #644

gr2m · 2023-06-02T19:31:38Z

I was hoping that rate limiting is now properly taken care of since we moved to Ocotkit via #487, but the error popped up again

https://github.com/semantic-release/commit-analyzer/actions/runs/5159013164/jobs/9293389281#step:6:126

We can tell from the user agent that the correct @semantic-release/github version was used

      'user-agent': '@semantic-release/github v8.1.0 octokit-core.js/4.2.1 Node.js/18.16.0 (linux; x64)',

What's odd: the request option does not have a retries key, it looks like it didn't attempt to retry at all:
https://github.com/semantic-release/commit-analyzer/actions/runs/5159013164/jobs/9293389281#step:6:138

I will look into it. Sorry, I'm not sure why this is happening and rate limit errors are hard to reproduce reliably, it will take me a moment to get to the bottom of this. If anyone is inclined to help out, you are very welcome to

The text was updated successfully, but these errors were encountered:

gr2m · 2023-06-02T19:33:38Z

One thing I want to look into is to avoid using search in the first place, we can look up associated pull requests based on commits using GraphQL now, there is no need to use search for that.

Kampfmoehre · 2023-06-05T07:53:41Z

If you need any example failed pipeline you can look into this one.

dhensby · 2023-09-08T22:36:13Z

I got hit by this issue earlier today and have done some digging to see what is going on.

So far there I've found 1 reason that is potentially causing retries to fail: https://github.com/semantic-release/github/blob/v9.0.4/lib/octokit.js#L25-L48

In the setup of the Octokit class, the retry config is passed in via RETRY_CONF, where RETRY_CONF is:

{
  // By default, Octokit does not retry on 404s.
  // But we want to retry on 404s to account for replication lag.
  doNotRetry: [400, 401, 403, 422],
}

But in the onRetry function, RETRY_CONFIG.retries is used and is undefined, this means the onRetry function will never return true and so retries are not attempted (I assume).

I tried to write a test to cover off the rate limit scenario, but I also noticed that the TestOctokit doesn't use SemanticReleaseOctokit, so the test suite doesn't test the logic as it is when run outside the test suite (ie: no retry, no throttle checks, etc). I suspect this has been done to improve speed of the test suite as when I did use the SemanticReleaseOctokit the test suite did take much longer to complete.

Kampfmoehre · 2024-03-11T07:50:15Z

Here is another failed pipeline, I saw in the logs that the search request was already limited. We use dependabot to keep deps up to date, resulting in a lot of small merge requests. Combined with not so regularly "manual" updates this results in a lot of issues having to be updated after release. But before at least some issues got updated, now it seems it doesn't even update one issue because searching for affecting issues already fails.

https://github.com/droidsolutions/semantic-release-update-file/actions/runs/8229131488/job/22499768584

Since logs vanish after some time, here is the request from the log

   status: 403,
  response: {
    url: 'https://api.github.com/search/issues?q=repo%3Adroidsolutions%2Fsemantic-release-update-file+type%3Apr+is%3Amerged+203258a9db965ddd4c498d1256ba4609809d1c61+c0b8a0c51a5efab6453fdd7175ed3250301c8196+0e56bf1bb4622b6cb28b58afe8426d2c591eb194+bd8e51d5ebaae09faff3310464a865586daa70a8',
    status: 403,
    headers: {
      'access-control-allow-origin': '*',
      'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset',
      'content-encoding': 'gzip',
      'content-security-policy': "default-src 'none'",
      'content-type': 'application/json; charset=utf-8',
      date: 'Mon, 11 Mar 2024 07:30:29 GMT',
      'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin',
      server: 'GitHub.com',
      'strict-transport-security': 'max-age=31536000; includeSubdomains; preload',
      'transfer-encoding': 'chunked',
      vary: 'Accept-Encoding, Accept, X-Requested-With',
      'x-accepted-oauth-scopes': 'repo',
      'x-content-type-options': 'nosniff',
      'x-frame-options': 'deny',
      'x-github-media-type': 'github.v3; format=json',
      'x-github-request-id': 'E20E:33A877:280331C:48F2596:65EEB313',
      'x-oauth-scopes': 'repo, workflow',
      'x-ratelimit-limit': '30',
      'x-ratelimit-remaining': '0',
      'x-ratelimit-reset': '1710142229',
      'x-ratelimit-resource': 'search',
      'x-ratelimit-used': '30',
      'x-xss-protection': '0'
    },
    data: {
      message: 'API rate limit exceeded for user ID 14077931. If you reach out to GitHub Support for help, please include the request ID E20E:33A877:280331C:48F2596:65EEB313.',
      documentation_url: 'https://docs.github.com/rest/overview/rate-limits-for-the-rest-api'
    }
  },
  request: {
    method: 'GET',
    url: 'https://api.github.com/search/issues?q=repo%3Adroidsolutions%2Fsemantic-release-update-file+type%3Apr+is%3Amerged+203258a9db965ddd4c498d1256ba4609809d1c61+c0b8a0c51a5efab6453fdd7175ed3250301c8196+0e56bf1bb4622b6cb28b58afe8426d2c591eb194+bd8e51d5ebaae09faff3310464a865586daa70a8',
    headers: {
      accept: 'application/vnd.github.v3+json',
      'user-agent': '@semantic-release/github v9.0.2 octokit-core.js/4.2.1 Node.js/20.11.1 (linux; x64)',
      authorization: 'token [REDACTED]'
    },
    request: { agent: undefined, hook: [Function: bound bound register] }
  },
  pluginName: '@semantic-release/github'
}

dhensby · 2024-03-11T09:24:12Z

@Kampfmoehre your version of @semantic-release/github is v9.0.2, but fixes have come in since then (eg: my fix was released in v9.0.5.

Upgrade your version of the package and I suspect it won't happen anymore (I've not experienced any issues since v9.0.5)

Kampfmoehre · 2024-03-11T09:28:20Z

@dhensby thx, I'll give it a try. We don't have the plugin as extra dep though, was previously using the one that comes with semantic-release package so you might want to update the min version there.

gr2m · 2024-03-11T15:55:19Z

using the one that comes with semantic-release package

The semantic-release package defines a range, you should get all updates automatically. If you have it as a dev dependency, make sure to update your lock file. We recommend to run it with just npx semantic-release without adding it as dependency

ab-arao · 2024-05-29T19:05:04Z

I think I'm running into the same problem as @Kampfmoehre and would appreciate some help getting the dependencies upgraded properly.

First, here's the error we are seeing:

[7:44:55 PM] [semantic-release] [@semantic-release/changelog] › ℹ  Create /__w/repo/repo/repo/CHANGELOG.md
[7:44:55 PM] [semantic-release] › ✔  Completed step "prepare" of plugin "@semantic-release/changelog"
[7:44:58 PM] [semantic-release] › ✔  Created tag v4.1.1
[7:44:58 PM] [semantic-release] › ℹ  Start step "publish" of plugin "@semantic-release/github"
[7:44:59 PM] [semantic-release] [@semantic-release/github] › ℹ  Published GitHub release: https://github.com/ourorg/repo/releases/tag/v4.1.1
[7:44:59 PM] [semantic-release] › ✔  Completed step "publish" of plugin "@semantic-release/github"
[7:44:59 PM] [semantic-release] › ℹ  Start step "success" of plugin "@semantic-release/github"
[7:45:00 PM] [semantic-release] [@semantic-release/github] › ℹ  Skip commenting on issues and pull requests.
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
[7:49:00 PM] [semantic-release] › ✘  Failed step "success" of plugin "@semantic-release/github"
[7:49:00 PM] [semantic-release] › ✘  An error occurred while running semantic-release: RequestError [HttpError]: You have exceeded a secondary rate limit. Please wait a few minutes before you try again. If you reach out to GitHub Support for help, please include the request ID 2840:295B44:3868737:64A2183:6232352C.
    at /usr/lib/node_modules/semantic-release/node_modules/@octokit/request/dist-node/index.js:124:21
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async requestWithGraphqlErrorHandling (/usr/lib/node_modules/semantic-release/node_modules/@octokit/plugin-retry/dist-node/index.js:71:20)
    at async Job.doExecute (/usr/lib/node_modules/semantic-release/node_modules/bottleneck/light.js:405:18) {
  status: 403,
  response: {
    url: 'https://api.github.com/search/issues?q=in%3Atitle+repo%3Aourorg%2Frepo+type%3Aissue+state%3Aopen+The%20automated%20release%20is%20failing%20%F0%9F%9A%A8',
    status: 403,
    headers: {
      'access-control-allow-origin': '*',
      'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset',
      'content-encoding': 'gzip',
      'content-security-policy': "default-repo 'none'",
      'content-type': 'application/json; charset=utf-8',
      date: 'Tue, 28 May 2024 19:49:00 GMT',
      'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin',
      server: 'GitHub.com',
      'strict-transport-security': 'max-age=31536000; includeSubdomains; preload',
      'transfer-encoding': 'chunked',
      vary: 'Accept-Encoding, Accept, X-Requested-With',
      'x-content-type-options': 'nosniff',
      'x-frame-options': 'deny',
      'x-github-api-version-selected': '2022-11-28',
      'x-github-media-type': 'github.v3; format=json',
      'x-github-request-id': '2840:295B44:3868737:64A2183:6232352C',
      'x-ratelimit-limit': '30',
      'x-ratelimit-remaining': '30',
      'x-ratelimit-reset': '1716925800',
      'x-ratelimit-resource': 'search',
      'x-ratelimit-used': '1',
      'x-xss-protection': '0'
    },
    data: {
      documentation_url: 'https://docs.github.com/free-pro-team@latest/rest/overview/rate-limits-for-the-rest-api#about-secondary-rate-limits',
      message: 'You have exceeded a secondary rate limit. Please wait a few minutes before you try again. If you reach out to GitHub Support for help, please include the request ID 2840:295B44:3868737:64A2183:6232352C.'
    }
  },
  request: {
    method: 'GET',
    url: 'https://api.github.com/search/issues?q=in%3Atitle+repo%3Aourorg%2Frepo+type%3Aissue+state%3Aopen+The%20automated%20release%20is%20failing%20%F0%9F%9A%A8',
    headers: {
      accept: 'application/vnd.github.v3+json',
      'user-agent': '@semantic-release/github v9.2.6 octokit-core.js/5.0.2 Node.js/21',
      authorization: 'token [REDACTED]'
    },
    request: {
      agent: undefined,
      hook: [Function: bound bound register],
      retryCount: 3
    }
  },
  pluginName: '@semantic-release/github'
}

As you can see, it says we are using @semantic-release/github v9.2.6.

However, in the build container we use the dependencies are the following:

$ docker run --rm -it build-container:latest npm view semantic-release dependencies
{
  '@semantic-release/commit-analyzer': '^12.0.0',
  '@semantic-release/error': '^4.0.0',
  '@semantic-release/github': '^10.0.0',
  '@semantic-release/npm': '^12.0.0',
  '@semantic-release/release-notes-generator': '^13.0.0',
  'aggregate-error': '^5.0.0',
  cosmiconfig: '^9.0.0',
  debug: '^4.0.0',
  'env-ci': '^11.0.0',
  execa: '^9.0.0',
  figures: '^6.0.0',
  'find-versions': '^6.0.0',
  'get-stream': '^6.0.0',
  'git-log-parser': '^1.2.0',
  'hook-std': '^3.0.0',
  'hosted-git-info': '^7.0.0',
  'import-from-esm': '^1.3.1',
  'lodash-es': '^4.17.21',
  marked: '^12.0.0',
  'marked-terminal': '^7.0.0',
  micromatch: '^4.0.2',
  'p-each-series': '^3.0.0',
  'p-reduce': '^3.0.0',
  'read-package-up': '^11.0.0',
  'resolve-from': '^5.0.0',
  semver: '^7.3.2',
  'semver-diff': '^4.0.0',
  signale: '^1.2.1',
  yargs: '^17.5.1'
}

We also execute semantic-release with npx as recommended in a previous comment.

Given that semantic-release should be using at least version 10.0.0 of @semantic-release/github, I am not sure how v9.2.6 is actually responsible for making the request.

I'd appreciate any guidance on what to upgrade in order to use a newer version that has the rate-limiting fix.

Thank you!

dhensby · 2024-05-30T08:28:42Z

@ab-arao you've not run into the same problem as @Kampfmoehre. They ran into the canonical issue of this thread (which is that if the API limit was hit then the job failed immediately).

What was meant to happen is that a secondary back-off was meant to take place to try to "wait out" the API limit resetting. Your job has done that (hence the Request quota exhausted for request GET /search/issues which didn't happen for @Kampfmoehre), and so is working "as expected" even though it's not perfect.

I think it's time this issue was closed, as I believe the issue is fixed. It may be worth a new issue opened about the secondary rate-limit and a more reliable way to wait-out the API limit even more gracefully.

ab-arao · 2024-05-30T15:23:41Z

Got it, thanks for clarifying @dhensby! I agree it makes sense to treat this as a separate issue since now the secondary rate limit is the issue.

yogithesymbian · 2024-08-23T08:12:10Z

[8:09:24 AM] [semantic-release] › ✔  Completed step "publish" of plugin "@semantic-release/github"
[8:09:24 AM] [semantic-release] › ℹ  Start step "success" of plugin "@semantic-release/github"
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues

zachspar · 2024-08-29T16:22:38Z

@babblebey seems like this is still a problem with the latest semantic-release/github plugin:

Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
Request quota exhausted for request GET /search/issues
...
    headers: {
      accept: 'application/vnd.github.v3+json',
      'user-agent': '@semantic-release/github v10.1.7 octokit-core.js/6.1.2 Node.js/20.12.2 (linux; x64)',
      authorization: 'token [REDACTED]'
    },
    request: {
      agent: undefined,
      hook: [Function: bound bound register],
      retryCount: 3
    }
  },
...
[4:10:07 PM] [semantic-release] › ✘  An error occurred while running semantic-release: RequestError [HttpError]: You have exceeded a secondary rate limit. Please wait a few minutes before you try again.

I thought I remember reading that the version is now supposed to use GraphQL to do the issue search instead of the GH API?

EDIT: I see the new issue, any update on when a fix will be merged?

babblebey · 2024-08-29T16:44:44Z

I thought I remember reading that the version is now supposed to use GraphQL to do the issue search instead of the GH API?

Yea, we changed consumption of the searchAPI in the request to fetch for associatedPRs on commits and that works great.

But, we still use the searchAPI when fetching for semantic-realease created failing release issues, we do this search so we can close the issue on successful release.

The searchAPI is very troublesome haha, painful to experience but, we're currently working on a fix here #907 where we're relegating its usage to remove in future release and replace with GraphQL

gr2m pinned this issue Jun 2, 2023

gr2m mentioned this issue Jun 12, 2023

How to execute "success" step manually after its failure? #651

Open

dhensby mentioned this issue Sep 8, 2023

fix: define retry limit in retry config #693

Merged

babblebey self-assigned this Jun 12, 2024

babblebey mentioned this issue Jun 13, 2024

fix: replace github search api with graphql in success lifecycle method #857

Merged

gr2m closed this as completed in #857 Jul 1, 2024

babblebey unpinned this issue Jul 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

API rate limit exceeded for installation #644

API rate limit exceeded for installation #644

gr2m commented Jun 2, 2023

gr2m commented Jun 2, 2023

Kampfmoehre commented Jun 5, 2023 •

edited

Loading

dhensby commented Sep 8, 2023

Kampfmoehre commented Mar 11, 2024

dhensby commented Mar 11, 2024

Kampfmoehre commented Mar 11, 2024

gr2m commented Mar 11, 2024

ab-arao commented May 29, 2024 •

edited

Loading

dhensby commented May 30, 2024

ab-arao commented May 30, 2024

yogithesymbian commented Aug 23, 2024

zachspar commented Aug 29, 2024 •

edited

Loading

babblebey commented Aug 29, 2024 •

edited

Loading

API rate limit exceeded for installation #644

API rate limit exceeded for installation #644

Comments

gr2m commented Jun 2, 2023

gr2m commented Jun 2, 2023

Kampfmoehre commented Jun 5, 2023 • edited Loading

dhensby commented Sep 8, 2023

Kampfmoehre commented Mar 11, 2024

dhensby commented Mar 11, 2024

Kampfmoehre commented Mar 11, 2024

gr2m commented Mar 11, 2024

ab-arao commented May 29, 2024 • edited Loading

dhensby commented May 30, 2024

ab-arao commented May 30, 2024

yogithesymbian commented Aug 23, 2024

zachspar commented Aug 29, 2024 • edited Loading

babblebey commented Aug 29, 2024 • edited Loading

Kampfmoehre commented Jun 5, 2023 •

edited

Loading

ab-arao commented May 29, 2024 •

edited

Loading

zachspar commented Aug 29, 2024 •

edited

Loading

babblebey commented Aug 29, 2024 •

edited

Loading