Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hook up GOPATH to named_caches? #13390

Open
Eric-Arellano opened this issue Oct 28, 2021 · 1 comment
Open

Hook up GOPATH to named_caches? #13390

Eric-Arellano opened this issue Oct 28, 2021 · 1 comment
Labels
backend: Go Go backend-related issues

Comments

@Eric-Arellano
Copy link
Contributor

Downloading third-party dependencies is proving really slow with Go. Over two minutes for https://github.com/kubernetes-sigs/external-dns

Our current implementation of third-party packages does not preserve the benefits of downloads across Pants runs. While we cache a particular resolution, if you make one change to go.mod or go.sum, you have to redo the entire download without leveraging already downloaded modules from earlier.

I don't know how to fix this because we probably want to use named_caches, but we do need to populate the chroot with the contents. When populating the chroot with contents, we also would like to symlink a la #12716.

@Eric-Arellano Eric-Arellano added the backend: Go Go backend-related issues label Oct 28, 2021
@Eric-Arellano
Copy link
Contributor Author

Eric-Arellano commented Oct 28, 2021

but we do need to populate the chroot with the contents

More concretely, we have two uses cases for this:

  • Third-party package code needs to extract out the relevant subset so that we can compile the code later:

all_digest_subset_gets.append(
Get(
Digest,
DigestSubset(
stripped_result_digest,
PathGlobs(
[os.path.join(dir_path, "*")],
glob_match_error_behavior=GlobMatchErrorBehavior.error,
description_of_origin=f"downloading {import_path}",
),
),
)
)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend: Go Go backend-related issues
Projects
None yet
Development

No branches or pull requests

1 participant