Fix caching for multi-step builds. #441
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
priyawadhwa
reviewed
Nov 8, 2018
| img, err := layerCache.RetrieveLayer(ck) | ||
| if err != nil { | ||
| logrus.Infof("No cached layer found for cmd %s", command.String()) | ||
| break |
There was a problem hiding this comment.
Sorry if I'm missing something, but won't the same problem from the issue happen here?
For this Dockerfile:
WORKDIR /dir
RUN something
we won't find the cached layer for WORKDIR so it'll break and never check the cache for RUN?
priyawadhwa
reviewed
Nov 8, 2018
dlorenc
force-pushed
the
fixcache
branch
2 times, most recently
from
5093ade
to
6a91162
Compare
|
Looks like the cache test failed in kokoro 😞 |
xoen
reviewed
Nov 9, 2018
| @@ -18,3 +18,5 @@ | |||
|
|
|||
| FROM gcr.io/google-appengine/debian9@sha256:1d6a9a6d106bd795098f60f4abb7083626354fa6735e81743c7f8cfca11259f0 | |||
| RUN apt-get update && apt-get install -y make | |||
| COPY context/bar /context | |||
There was a problem hiding this comment.
Kind of lurking here so apologies if it's a stupid suggestion but it may help to add a WORKDIR command to this test Dockerfile as it's something many Dockerfile out in the wild will have.
|
Hello @dlorenc - first of all, thanks so much for the quick fix ❤️ I've checked out this branch and built the executor docker image locally and tried again with our template I tried both with the original I still see that it pushes more stuff to the To be clear, these are very small images (AWS ECR says "< 0.01 MiB") but over time it can hit the default AWS ECR limit of 1,000 images per repository, especially if using one single cache repository per class of applications - unless of kaniko has some kind of built-in cache cleanup mechanism which I'm not aware of. |
Thanks - I do think this is a further optimization we can make, and I need to dig into where it's coming from still. |
It looks like that was a bug with container-diff. I changed the Dockerfile to not triggger it. |
xoen
added a commit
to ministryofjustice/analytics-platform-common-concourse-tasks
that referenced
this pull request
Nov 9, 2018
> kaniko is a tool to build container images from a Dockerfile [...] > [...] > kaniko doesn't depend on a Docker daemon and executes each > command within a Dockerfile completely in userspace. Kaniko can cache docker layers by pushing these to a special docker repository and reusing them later on which is ideal in the context of Concourse where each task runs in an independent container. There is a bug in the kaniko caching mechanism at the moment but there is a PR which seems to solve it (I tested it locally and it's now using the cached layers). This task is using a resource which was already there, we'll see if this works (resource code looks OK to me) Resourse: https://github.com/lxfontes/kaniko-resource Kaniko: https://github.com/GoogleContainerTools/kaniko Kanino cache bugfix: GoogleContainerTools/kaniko#441
xoen
added a commit
to ministryofjustice/analytics-platform-common-concourse-tasks
that referenced
this pull request
Nov 9, 2018
> kaniko is a tool to build container images from a Dockerfile [...] > [...] > kaniko doesn't depend on a Docker daemon and executes each > command within a Dockerfile completely in userspace. Kaniko can cache docker layers by pushing these to a special docker repository and reusing them later on which is ideal in the context of Concourse where each task runs in an independent container. There is a bug in the kaniko caching mechanism at the moment but there is a PR which seems to solve it (I tested it locally and it's now using the cached layers). This task is using a resource which was already there, we'll see if this works (resource code looks OK to me) Resourse: https://github.com/lxfontes/kaniko-resource Kaniko: https://github.com/GoogleContainerTools/kaniko Kanino cache bugfix: GoogleContainerTools/kaniko#441
xoen
added a commit
to ministryofjustice/analytics-platform-common-concourse-tasks
that referenced
this pull request
Nov 9, 2018
> kaniko is a tool to build container images from a Dockerfile [...] > [...] > kaniko doesn't depend on a Docker daemon and executes each > command within a Dockerfile completely in userspace. Kaniko can cache docker layers by pushing these to a special docker repository and reusing them later on which is ideal in the context of Concourse where each task runs in an independent container. There is a bug in the kaniko caching mechanism at the moment but there is a PR which seems to solve it (I tested it locally and it's now using the cached layers). This task is using a resource which was already there, we'll see if this works (resource code looks OK to me) Resourse: https://github.com/lxfontes/kaniko-resource Kaniko: https://github.com/GoogleContainerTools/kaniko Kanino cache bugfix: GoogleContainerTools/kaniko#441
xoen
added a commit
to ministryofjustice/analytics-platform-common-concourse-tasks
that referenced
this pull request
Nov 9, 2018
> kaniko is a tool to build container images from a Dockerfile [...] > [...] > kaniko doesn't depend on a Docker daemon and executes each > command within a Dockerfile completely in userspace. Kaniko can cache docker layers by pushing these to a special docker repository and reusing them later on which is ideal in the context of Concourse where each task runs in an independent container. There is a bug in the kaniko caching mechanism at the moment but there is a PR which seems to solve it (I tested it locally and it's now using the cached layers). This task is using a resource which was already there, we'll see if this works (resource code looks OK to me) Resourse: https://github.com/lxfontes/kaniko-resource Kaniko: https://github.com/GoogleContainerTools/kaniko Kanino cache bugfix: GoogleContainerTools/kaniko#441
xoen
added a commit
to ministryofjustice/analytics-platform-common-concourse-tasks
that referenced
this pull request
Nov 9, 2018
> kaniko is a tool to build container images from a Dockerfile [...] > [...] > kaniko doesn't depend on a Docker daemon and executes each > command within a Dockerfile completely in userspace. Kaniko can cache docker layers by pushing these to a special docker repository and reusing them later on which is ideal in the context of Concourse where each task runs in an independent container. There is a bug in the kaniko caching mechanism at the moment but there is a PR which seems to solve it (I tested it locally and it's now using the cached layers). This task is using a resource which was already there, we'll see if this works (resource code looks OK to me) Resourse: https://github.com/lxfontes/kaniko-resource Kaniko: https://github.com/GoogleContainerTools/kaniko Kanino cache bugfix: GoogleContainerTools/kaniko#441
xoen
added a commit
to ministryofjustice/analytics-platform-common-concourse-tasks
that referenced
this pull request
Nov 9, 2018
> kaniko is a tool to build container images from a Dockerfile [...] > [...] > kaniko doesn't depend on a Docker daemon and executes each > command within a Dockerfile completely in userspace. Kaniko can cache docker layers by pushing these to a special docker repository and reusing them later on which is ideal in the context of Concourse where each task runs in an independent container. There is a bug in the kaniko caching mechanism at the moment but there is a PR which seems to solve it (I tested it locally and it's now using the cached layers). This task is using a resource which was already there, we'll see if this works (resource code looks OK to me) Resourse: https://github.com/lxfontes/kaniko-resource Kaniko: https://github.com/GoogleContainerTools/kaniko Kanino cache bugfix: GoogleContainerTools/kaniko#441
priyawadhwa
approved these changes
Nov 9, 2018
This change fixes that by properly "replaying" the Dockerfile and mutating the config when calculating cache keys. Previously we were looking at the wrong cache key for each command when there was more than one.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
#397 broke caching for Dockerfiles that have more than a single command.
This change fixes that by properly "replaying" the Dockerfile and mutating the config when
calculating cache keys. Previously we were looking at the wrong cache key for each command
when there was more than one.
I think this should fix #410