Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Offline cache for github worker #29925

Open
karalabe opened this issue Jun 4, 2024 · 5 comments
Open

Offline cache for github worker #29925

karalabe opened this issue Jun 4, 2024 · 5 comments
Labels

Comments

@karalabe
Copy link
Member

karalabe commented Jun 4, 2024

We have a github action and a dedicated worker for it to run tests fast... but the thing caches Go build files into GitHub (uploading and downlaoding 3.5GB), so it takes 8 minutes to start the tester, 20 secs to test and 8 minutes to stop the tester. Crazy. We need to make the tester cache locally, since our dedicated builder is a unique snowflake.

Also should enable race detector runs on it.

@sisco0
Copy link

sisco0 commented Jun 23, 2024

Is there any specific Github Actions Workflow YAML file that you would like to consider? I could only find go.yml.
I would suggest modifying the Workflow YAML file. Default paths of interest are ~/.cache/go-build and $GOPATH/pkg/mod. Might be more paths are of interest.

For developers who are not maintainers of this repository, I would suggest performing a personal fork and test over it.

I attach my timing results in the following screenshot for a Fork of the current project, where go.yml is executed using my own machine.
Kindly observe the first iteration to be approximately 4 minutes duration. However, for iterations after it, times seem to be optimized.
image

@gballet
Copy link
Member

gballet commented Oct 15, 2024

@karalabe @parithosh had something in the pipe to solve this. Not sure what the current state is.

Basically, the cache is big because it's never cleared. It should be cleared more often.

@gballet
Copy link
Member

gballet commented Oct 15, 2024

also FYI it is a local cache, it's just that copying it takes forever

@parithosh
Copy link
Member

cc @elasticroentgen , How/when do we want to migrate the geth github actions runner into your infra?

@elasticroentgen
Copy link
Member

as stated on our devops ticket https://github.com/ethereum/devops/issues/1507 i've set up a potential solution for this, but never received feedback on it. If it works as expected (see the linked issue) i can assist in making the switch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants