Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MAX_NODE_LIMIT_EXCEEDED #911

Closed
travi opened this issue Sep 3, 2024 · 3 comments · Fixed by #912
Closed

MAX_NODE_LIMIT_EXCEEDED #911

travi opened this issue Sep 3, 2024 · 3 comments · Fixed by #912
Assignees
Labels

Comments

@travi
Copy link
Member

travi commented Sep 3, 2024

i just hit this error when releasing a package that hasnt had activity in quite some time other than renovate dependency update PRs. the last release of this package was in 2021, so i imagine there are lots of dependency update commits and associated PRs since then. i imagine this is another scenario where pagination could be necessary.

error output:

✘  An error occurred while running semantic-release: Error: This query requests up to 1,010,000 possible nodes which exceeds the maximum limit of 500,000.
    at file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/aggregate-error/index.js:23:26
    at Array.map (<anonymous>)
    at new AggregateError (file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/aggregate-error/index.js:16:19)
    at file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/semantic-release/lib/plugins/pipeline.js:55:13
    at async pluginsConfigAccumulator.<computed> [as success] (file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/semantic-release/lib/plugins/index.js:87:11)
    at async run (file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/semantic-release/index.js:218:3)
    at async Module.default (file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/semantic-release/index.js:278:22)
    at async default (file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/semantic-release/cli.js:55:5) {
  type: 'MAX_NODE_LIMIT_EXCEEDED',
  pluginName: '@semantic-release/github'
}
AggregateError: 
    Error: This query requests up to 1,010,000 possible nodes which exceeds the maximum limit of 500,000.
        at Array.map (<anonymous>)
        at file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/semantic-release/lib/plugins/pipeline.js:55:13
        at async pluginsConfigAccumulator.<computed> [as success] (file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/semantic-release/lib/plugins/index.js:87:11)
        at async run (file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/semantic-release/index.js:218:3)
        at async Module.default (file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/semantic-release/index.js:278:22)
        at async default (file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/semantic-release/cli.js:55:5)
    at file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/semantic-release/lib/plugins/pipeline.js:55:13
    at async pluginsConfigAccumulator.<computed> [as success] (file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/semantic-release/lib/plugins/index.js:87:11)
    at async run (file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/semantic-release/index.js:218:3)
    at async Module.default (file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/semantic-release/index.js:278:22)
    at async default (file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/semantic-release/cli.js:55:5) {
  errors: [
    Error: This query requests up to 1,010,000 possible nodes which exceeds the maximum limit of 500,000.
        at file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/aggregate-error/index.js:23:26
        at Array.map (<anonymous>)
        at new AggregateError (file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/aggregate-error/index.js:16:19)
        at file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/semantic-release/lib/plugins/pipeline.js:55:13
        at async pluginsConfigAccumulator.<computed> [as success] (file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/semantic-release/lib/plugins/index.js:87:11)
        at async run (file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/semantic-release/index.js:218:3)
        at async Module.default (file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/semantic-release/index.js:278:22)
        at async default (file:///home/runner/.npm/_npx/03d6da4377eeea29/node_modules/semantic-release/cli.js:55:5) {
      type: 'MAX_NODE_LIMIT_EXCEEDED',
      pluginName: '@semantic-release/github'
    }
  ]
}
@babblebey
Copy link
Member

We are already paginating and in fact we split requests into chunks, the issue here is that we've got a query that possibly break the GitHub GraphQL API Node Limit... this one's my bad 🫣

But, 1,010,000 possible nodes! How did we get here??? 🤔

Examine our Query and Calculate

query getAssociatedPRs {
  repository([name], [owner]) {
    commit1: object(oid: [sha]) {
      ...Fields
    }

    // ....other commits minimized

    commit100: object(oid: [sha]) {
      ...Fields
    }
  }
}

fragment Fields on Commit {
  oid
  associatedPullRequests(first: 100) {
    pageInfo {
      endCursor
      hasNextPage
    }
    nodes {
      //...otherBaseFields minimized
      labels(first: 100) {
        nodes {
          id
          url
          name
          color
        }
      }
      mergeable
      canBeRebased
      changedFiles
      mergedAt
      isDraft
      mergedBy {
        login
        avatarUrl
        url
      }
      commits {
        totalCount
      }
    }
  }
}

100 = 100 Commits Nodes
+
100 x 100 = 10,000 AssociatedPRs Nodes
+
100 x 100 x 100 = 1,000,000 AssociatedPRs Label Nodes

= 1,010,000 Total Possible Nodes

Explainer

  • We are making request to 100 commits in one call, this mean the API connects to a total of each of the 100 commits and that calculate: 100 Commits Nodes
  • We are getting 100 associatedPRs per commits so this means for all the 100 commits, this means the API is requesting a total of: 10,000 AssociatedPRs Nodes
  • We are also getting 100 labels per associatedPRs (on each commits), the API is requesting a total of: 1,000,000 AssocatedPRs Label Nodes
  • Total Requested Nodes: 1,010,000

Proposed Fix

We currently split the context.commit into chunks of 100 and make a parallel call to the graphQL API; this process involves building the graphql query (writing the commit1 - commit100 fragment in one call) for each chunks and making the call.

In order to accommodate the <= 500,000 node limits, I will reduce the label requested first value to 40 this will help bring the requested nodes per call to 410,000.

😉

@AntonyF-Andreani
Copy link

@ldamici

Copy link

github-actions bot commented Sep 3, 2024

🎉 This issue has been resolved in version 10.3.1 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants