Work around COPY --link limitations by pre-creating full filesystem tree
#521
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… tree Without this, `COPY --link` insists on creating `/usr` and `/usr/local` for us, and does so with non-reproducible timestamps, defeating the purpose of our carefully crafted reproducible `/usr/local/go` -- this combats that by pre-creating a full `/target` directory that includes `/target/usr/local/go` so we can `COPY --link /target/ /` and get a *properly* reproducible layer. I've also added more sanity checks to validate our reproducibility assumptions (namely that our detected `SOURCE_DATE_EPOCH` value is older than our build/wall clock and that no files in our final tree are newer than our `SOURCE_DATE_EPOCH`).
|
Anyone following along at home can do something like the following to verify this: $ docker buildx --builder foo build https://github.com/docker-library/golang.git#refs/pull/521/head:1.22/bookworm --output type=oci --quiet | tar -tv | grep 060aaf7efd0676cdf56165fe26e63a047d7f3c483ab1043d530db9370e6c28e7
-r--r--r-- 0/0 69345548 1969-12-31 16:00 blobs/sha256/060aaf7efd0676cdf56165fe26e63a047d7f3c483ab1043d530db9370e6c28e7Replacing |
$ docker buildx --builder foo build https://github.com/docker-library/golang.git#refs/pull/521/head:1.22/bookworm --output type=oci --quiet | tar -x --to-stdout blobs/sha256/060aaf7efd0676cdf56165fe26e63a047d7f3c483ab1043d530db9370e6c28e7 | tar -tvz | grep -E ' usr(/local(/go)?)?/$'
drwxr-xr-x 0/0 0 2024-05-30 12:26 usr/
drwxr-xr-x 0/0 0 2024-05-30 12:26 usr/local/
drwxr-xr-x 0/0 0 2024-05-30 12:26 usr/local/go/vs: $ crane blob golang@sha256:69828e165440b00c6a6cf1cc039b1812b75b8604568728dccd4d39573d405e26 | tar -tvz | grep -E ' usr(/local(/go)?)?/$'
drwxr-xr-x 0/0 0 2024-06-13 11:14 usr/
drwxr-xr-x 0/0 0 2024-06-13 11:14 usr/local/
drwxr-xr-x 0/0 0 2024-05-30 12:26 usr/local/go/ |
yosifkit
approved these changes
Jun 13, 2024
docker-library-bot
added a commit
to docker-library-bot/official-images
that referenced
this pull request
Jun 13, 2024
Changes: - docker-library/golang@1a33f8b: Merge pull request docker-library/golang#521 from infosiftr/copy-link-redux - docker-library/golang@81c0d31: Work around `COPY --link` limitations by pre-creating full filesystem tree
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Without this,
COPY --linkinsists on creating/usrand/usr/localfor us, and does so with non-reproducible timestamps, defeating the purpose of our carefully crafted reproducible/usr/local/go-- this combats that by pre-creating a full/targetdirectory that includes/target/usr/local/goso we canCOPY --link /target/ /and get a properly reproducible layer.I've also added more sanity checks to validate our reproducibility assumptions (namely that our detected
SOURCE_DATE_EPOCHvalue is older than our build/wall clock and that no files in our final tree are newer than ourSOURCE_DATE_EPOCH).See also #505