--label causes cache invalidation in later stages
#3088
Comments
QuanZhang-William
added
kind/bug
|
I also suspect this is my issue when using |
|
In our case it's even worse, we removed all the labels yet still run into the same issue, the reason, the
Every time the base-image gets built, a new image with the identical layers is created, because it contains an updated timestamp. This is not tragic, but kaniko-cache is sensitive to the image-sha, not the sha of the layers. So when this new image is used as a base, the entire cache is lost, even though nothing changed in the base image. note that we pass the base image as a build-arg: FROM upstream/image:version AS build
ARG BUILD_IMAGE=build
FROM $BUILD_IMAGE AS productionbut the arg does not contain the shasum, so nothing should change in the environment. also note that I'm aware of the |
|
i guess the issue is here |
|
for my case at least this could be enough? + sourceImageNoTimestamps, err := mutate.CreatedAt(sourceImage, v1.Time{})
+ if err != nil {
+ return nil, err
+ }
+ digest, err := sourceImageNoTimestamps.Digest()
- digest, err := sourceImage.Digest()of course this could be extended to also get rid of labels etc. |
|
that seems to fix the timestamp issue on my side, I think I can go ahead and open a PR, maybe also fix the labels issue, although I think that should be two separate, dependent PRs, as the maintainers might want independent judgement on whether to include labels or not. |
|
this fixes the issues with the labels causing cache misses in my setup, @euven could you please verify? + cf, err := sourceImage.ConfigFile()
+ if err != nil {
+ return nil, err
+ }
+ cfg := cf.DeepCopy()
+ cfg.Created = v1.Time{}
+ cfg.Config.Labels = map[string]string{}
+ sourceImageReproducible, err := mutate.ConfigFile(sourceImage, cfg)
+ if err != nil {
+ return nil, err
+ }
+
+ digest, err := sourceImageReproducible.Digest()
- digest, err := sourceImage.Digest() |
Actual behavior
A change in a
--labelvalue, causes cache invalidation for stages after the first one.Expected behavior
Caches are used as expected for a RUN command all stages, if there has been no changes to it.
To Reproduce
Steps to reproduce the behavior:
--cache --target stage-2 --label somelabel=1`docker run -v "/tmp/config.json:/kaniko/.docker/config.json" -v $PWD:/workspace2 gcr.io/kaniko-project/executor:debug --dockerfile /workspace2/docker/ubuntu-base-images/Dockerfile --no-push --context /workspace2/ --cache --cache-repo gitlab.catalyst.net.nz:4567/eugene/docker-images/kaniko-testing-cache --target stage-2 --label somelabel=6
2. run step 1 again and observe the cache being used for both stages:
--label somelabel=2, then run the build again. You'll note the cache not being used for theRUNin the second stage:Triage Notes for the Maintainers
--cacheflagThe text was updated successfully, but these errors were encountered: