Use ARG in stage name

[Bidski]

In relation to #2618

How does caching work for stages in a multi-stage build?

If I have a Dockerfile similar to this one and I change the value of ARG how does that affect the caching of the stages?

ARG variant

FROM archlinux:latest AS base-image
RUN some things

FROM base-image AS base-image-variant1
RUN some variant specific things (dependent on ${variant})

FROM base-image-variant1 AS base-image-variant1-sub-image-1
RUN some more things (transitively dependent on ${variant} via base-image-variant1)

FROM base-image-variant1 AS base-image-variant1-sub-image-2
RUN some more things (transitively dependent on ${variant} via base-image-variant1)

The contents of base-image-variant1 will vary based on the value of ${variant} and will also cause the contents of base-image-variant1-sub-image-1 and base-image-variant1-sub-image-2 to change as well.

Does changing the value of a global argument completely invalidate any cache for the stages? Does the value of any global arguments get included as part of the cache so that all stages are cached uniquely between builds based on the values of any global arguments?

[tonistiigi]

In your example, changing the ARG will never have any effect on the cache because you have only defined the arg in the global scope, meaning none of the RUN statements can see it. https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact

If instead you meant to write.

FROM base-image AS base-image-variant1
ARG variant
RUN some variant specific things (dependent on ${variant})

then only the commands that use the value of the ARG get invalidated when it is changed. RUN some things remains cached when you changed the value of variant.

[Bidski]

So what are the caching rules for stages?

If I have something like this, under what conditions would the cache for base-image-variant-sub-stage-1 and base-image-variant-sub-stage-2 be invalidated?

Apart from the value of the ARG changing, what would invalidate the cache for base-image-variant?

FROM archlinux:latest AS base-image
RUN some things

FROM base-image AS base-image-variant
ARG variant
RUN some variant specific things (dependent on ${variant})

FROM base-image-variant AS base-image-variant-sub-stage-1
RUN some more things (transitively dependent on ${variant} via base-image-variant)

FROM base-image-variant AS base-image-variant-sub-stage-2
RUN some more things (transitively dependent on ${variant} via base-image-variant)

FROM base-image-variant AS final-stage
COPY --from=base-image-variant-sub-stage-1 ......
COPY --from=base-image-variant-sub-stage-2 ......

I'm not sure if I said this well enough before, so I will repeat it.

base-image-variant directly uses the value of ${variant} and generates certain files which are dependent on this value. So changing the value of the ARG will cause a direct change in the content of certain generated files.

base-image-variant-sub-stage-1 and base-image-variant-sub-stage-2 don't use the ARG directly. Rather the content of the files that are generated in these stages is influenced by the content of the generated files in base-image-variant.

[tonistiigi]

Apart from the value of the ARG changing, what would invalidate the cache for base-image-variant?

Updates to archlinux:latest layers/env. That's it.

If you want to make changes to base-image-variant without invalidating cache for base-image-variant-sub-stage-1 then that stage can't depend on the base stage. Instead for example it should also be FROM archlinux:latest and results merged together with base-image-variant using COPY --from.