Dockerfiles should not be derived from a rocker image on Docker Hub #754
Comments
eitsupi
added
enhancement
|
@eitsupi Can you outline the advantages of this proposal? I think this would have significant disadvantages.
Yes, I share the frustration that it is difficult to test the changes to one image, especially a very downstream image like |
|
@cboettig The layers are shared in my proposal as well. # rocker/r-ver like base image
# Cached layers
FROM aaa
# New layers
COPY /bbb /bbb
RUN ./bbb
COPY /ccc /cccand # rocker/rstudio like image
# Cached layers
FROM aaa
COPY /bbb /bbb
RUN ./bbb
# New layers
COPY /ddd /ddd
RUN ./ddd
COPY /ccc /ccc# rocker/tidyverse like image
# Cached layers
FROM aaa
COPY /bbb /bbb
RUN ./bbb
COPY /ddd /ddd
RUN ./ddd
# New layers
COPY /eee /eee
RUN ./eee
COPY /ccc /cccThe current implementation uses rocker scripts in # rocker/rstudio like image
# Cached layers
FROM rocker/r-ver
# New layers
RUN ./dddEven though only # rocker/tidyverse like image
# Cached layers
FROM rocker/rstudio
# New layers
RUN ./eee |
|
@eitsupi Ha, okay I see now that my mental model of caching from old docker build is not quite up to date, buildx caching is more 'clever' in this respect and can identify that those lines are identical and thus can be used from the cache. That's only true if both images are built on the same machine of course, and I don't think used to be true at all on older docker.
I think historically So I agree this won't break caching, but it still seems like a substantial change. If we simply kept the FROM lines as they are, but had all images copy over only the Either way, I think there are implications to think through. I see why from a build perspective it is rather annoying that changing The whole original idea of having a |
In fact, it is not. Since the bake file is written to pull the cache from the container registry (or can be specified in the CLI options), the cache is used across machines. Like rocker-versioned2/bakefiles/4.3.2.docker-bake.json Lines 83 to 85 in 1828862 rocker-versioned2/.github/workflows/r-build-test.yml Lines 38 to 46 in 1828862 |
|
that's clever! buildx / bake system is still a whole new world to me. In any event, I agree that this preserves the cache (and thus also preserves the matching layers, which impacts the end user and not just the build times). I just also think that preserving the layering and cache would be accomplished by changing COPY pattern alone. I'm still not sure that we want rocker/r-ver image to be shipping with a potentially different copy of a build script like |
|
My suggestion is to copy all of the rocker scripts directories at the very end of each Dockerfile. rocker-versioned2/dockerfiles/r-ver_4.3.2.Dockerfile Lines 12 to 19 in 1828862 Since this is only done in r-ver for now, |
|
Ah, I follow, yes, with the COPY at the end of each they all get updated copies of all the scripts even if other layers are captured from cache. I hadn't followed that part before. I think that nicely avoids the images getting out-of-sync with what scripts they contain due to caching. If the images inherit from each-other linearly (the current use of FROM), I guess that causes build cache to break. But using an identical FROM and repeated lines, buildx can leverage cache. That's really different, but I like it! In a way it is nice that Dockerfiles are more self-contained, a user looking at the Dockerfile for Ok, I'm convinced! Sorry I was so slow on this. buildx is more of an adjustment to my mental model than I anticipated! |
From #750 (comment)
e.g.
instead of
The text was updated successfully, but these errors were encountered: