Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include source stage cache key in cache key for COPY commands using --from #883

Conversation

cvgw
Copy link
Contributor

@cvgw cvgw commented Nov 25, 2019

Address one part of the issue in #870

Description

Include the source stage cache key in the cache key for COPY commands which use --from. This will insure when the source stage for the COPY command changes the cache for said COPY command is invalidated.

FROM a as a
RUN break-the-cache
FROM b as b
RUN something  <-- this will still be cached
COPY --from a a/file   <--here the cache is disabled

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

  • Includes unit tests
  • Adds integration tests if needed.

See the contribution guide for more details.

Reviewer Notes

  • The code flow looks good.
  • Unit tests and or integration tests added.

Release Notes

Describe any changes here so maintainer can include it in the release notes, or delete this block.

- Improve cache invalidation for COPY commands using the --from option

@cvgw cvgw force-pushed the u/cvgw/include_source_stage_digest_in_cache_key_copy_from_commands branch 4 times, most recently from 042771f to eace4e7 Compare November 26, 2019 00:23
@cvgw cvgw changed the title Work in progress Include source stage digest in cache key for COPY commands using --from Nov 26, 2019
@cvgw cvgw force-pushed the u/cvgw/include_source_stage_digest_in_cache_key_copy_from_commands branch 3 times, most recently from 69c8794 to 6b88c13 Compare November 26, 2019 01:11
@mcfedr mcfedr mentioned this pull request Nov 26, 2019
4 tasks
@cvgw cvgw force-pushed the u/cvgw/include_source_stage_digest_in_cache_key_copy_from_commands branch 3 times, most recently from 36200a8 to a8bfba2 Compare November 28, 2019 18:38
@cvgw cvgw changed the title Include source stage digest in cache key for COPY commands using --from Include source stage cache key in cache key for COPY commands using --from Dec 2, 2019
@cvgw
Copy link
Contributor Author

cvgw commented Dec 2, 2019

This PR also requires #899 to be fixed in order to completely address the issue

@cvgw cvgw force-pushed the u/cvgw/include_source_stage_digest_in_cache_key_copy_from_commands branch from a8bfba2 to 58919ba Compare December 10, 2019 00:27
Include the digest of the stage specified in the --from argument for
COPY commands which use --from
@cvgw cvgw force-pushed the u/cvgw/include_source_stage_digest_in_cache_key_copy_from_commands branch 2 times, most recently from 561db28 to db76b60 Compare December 10, 2019 02:52
* use the cachekey of the src stage rather than the digest
for COPY --from commands as they are reproducible unlike digests
* track digest to cache keys and stage indexes to digest
* add extra debug logging for troubleshooting cachekey building issues
* convert Sha256 hashes to hex encoded strings rather than plain strings
for easier human reading
@cvgw cvgw force-pushed the u/cvgw/include_source_stage_digest_in_cache_key_copy_from_commands branch from db76b60 to b19214a Compare December 10, 2019 16:45
@cvgw cvgw merged commit f8507eb into GoogleContainerTools:master Dec 10, 2019
@cvgw cvgw deleted the u/cvgw/include_source_stage_digest_in_cache_key_copy_from_commands branch December 10, 2019 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants