Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extract directly to the toolcache #2631

Merged
merged 14 commits into from
Dec 4, 2024

Conversation

henrymercer
Copy link
Contributor

Adding the CodeQL Bundle to the toolcache seems to take a surprisingly long amount of time. We haven't collected large scale telemetry yet, but here's some anecdotal numbers from looking at the logs on PR checks:

  • Linux: ~10–15 secs
  • macOS: ~15–20 secs
  • Windows: ~2 minutes (!)

Therefore in this PR, we add a feature flag that extracts the CodeQL bundle directly to the toolcache, eliminating the copy to toolcache step completely.

I've tested the change by adding a PR check, temporarily modifying this to running 10 runs on each platform, and running this a bunch of times. Next steps are dogfooding this change internally using the feature flag.

Merge / deployment checklist

  • Confirm this change is backwards compatible with existing workflows.
  • Confirm the readme has been updated if necessary.
  • Confirm the changelog has been updated if necessary.

@henrymercer henrymercer requested a review from a team as a code owner December 3, 2024 18:29
dbartol
dbartol previously approved these changes Dec 3, 2024
Copy link
Contributor

@dbartol dbartol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Looks like a relatively simple change, plus some plumbing to pass the destination directory through to all the places the write the files.

On a GitHub-hosted runner, it looks like we'll still add the bundle to the toolcache, even though the cache will disappear after the job. Is this just because we have to put it somewhere, and we might as well put it in the toolcache for consistency with the less-common self-hosted runner scenario?

@henrymercer
Copy link
Contributor Author

On a GitHub-hosted runner, it looks like we'll still add the bundle to the toolcache, even though the cache will disappear after the job. Is this just because we have to put it somewhere, and we might as well put it in the toolcache for consistency with the less-common self-hosted runner scenario?

Yes, pretty much. It's also consistent with the situation where the GitHub-hosted runner images have caught up to the latest version of CodeQL and the tools we want are already installed. Plus if anyone's still using a hardcoded toolcache path to find CodeQL, that'll still work.

Copy link
Member

@NlightNFotis NlightNFotis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

Thank you for this change!

src/tools-download.ts Show resolved Hide resolved
src/tools-download.ts Show resolved Hide resolved
@henrymercer henrymercer merged commit 3096afe into main Dec 4, 2024
274 checks passed
@henrymercer henrymercer deleted the henrymercer/extract-direct-to-toolcache branch December 4, 2024 11:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants