Image sizes increased dramatically since merging #289

[lubo]

Since merging #289, Kaniko seems to create layers containing not only the files copied by COPY instruction, but also files from the previous layers.

[bobcatfish]

That's not good! Thanks for reporting this @lubo - do you happen to have an example of how to reproduce this?

[lubo]

Sure. The Dockerfiles I've been testing Kaniko with look like this:

FROM image as installer
RUN <install files into /install>
COPY more files /install

FROM image
COPY --from=installer /install /

So, I use multi-stage builds. In the first stage, /install is created and then it's copied to the second stage. Now, I'd expect the layer created by COPY --from=installer /install / to only contain files from /install in the first stage. I haven't investigated this thoroughly, but the layer definitely contains files that it should not and it seems like they come from the previous layers of the second stage (or from the base image image in this particular case).

[bobcatfish]

Thanks @lubo ! So it looks like this could be specifically related to multi-stage builds 🤔 That logic definitely may have changed in #289 (not intentionally tho!). I'll take a look!

[priyawadhwa]

I used this test Dockerfile and was able to repro this @ HEAD:

FROM gcr.io/google-appengine/debian9 as installer
RUN mkdir /install
RUN echo "hey" > /install/hey
COPY Makefile hack /install/

FROM gcr.io/google-appengine/debian9
COPY --from=installer /install /

I used container-diff with the --type file --type layer arguments to diff images by layer, and kaniko built the image correctly.

I actually don't think this is a bug. #289 changed the logic here to better determine if we want to snapshot the entire filesystem or just certain files after each command.

Before #289 we defaulted to snapshotting the entire filesystem after the last command, so you wouldn't have seen a huge log message detailing the exact files to snapshot. Now that the logic is correct, since you're copying to root (/), we snapshot all of the files at root and you get a giant log message listing all of those files, which is expected behaviour.

We could change the code to not print out all the files we're snapshotting and instead just say something like (Snapshotting files...) though, so that the logs aren't super long and unhelpful!

[priyawadhwa]

Whoops, please disregard my above comment, I just tried again with the same Dockerfile and it does seem like there's some sort of bug 😰

Update: The COPY command gets the files to snapshot by getting all files copied to the destination here, and in this case the destination is root (/) so we include all files to be snapshotted. Now that #289 forces snapshotting of specific files (Add instead of MaybeAdd) all of these files end up in the layer.

@bobcatfish , WDYT would make sense here?

[priyawadhwa]

Hey @lubo , this should have been fixed by #319, please open another issue if you encounter any more problems!