Bake target dependencies

[asonawalla]

One notable missing feature of the high-level build construct offered by the bake command is the ability to express dependencies between targets. Compose provides a valid "depends on" parameter, but from my reading it looks like this is currently dropped when compose files are read by bake.

My use case is a good example; I have a Go codebase that produces lots of Go binaries. Many of those binaries get stuffed into containers to run as long-lived daemons, many get used in other various container images also built by my repo, and some wind up in both. I have a go_container.dockerfile that looks approximately like this:

FROM golang:1.12 as go_builder
<do stuff>
RUN go install ./...

FROM alpine as go_container
ARG cmd
COPY --from=go_builder /go/bin/${cmd} /opt/gobin/${cmd}

Which I'm then invoking using one target with the correct arg per daemon-container I want to build, plus once as a generic go_builder target, just to get the image with all the built binaries in the local registry. I also have a toolbox.dockerfile that looks approximately like this:

FROM go_builder as go_builder
FROM ubuntu as toolbox
<do stuff>
COPY --from=go_builder /go/bin/<somecmd> /usr/local/bin/
<do more stuff>

And I have tens of dockerfiles that follow a similar pattern to this one. The trick is that go_builder needs to be built before we can start executing the toolbox image.

Currently, I'm solving for this by invoking bake twice: once for the core go containers and once for everything else, though that limits the expressiveness of my dependencies and leads to less-than-perfect utilization of my compute power. A working depends-on parameter would solve for both concerns.

[FernandoMiguel]

Afaik there is no depends on at build time, only run time, for compose

[FernandoMiguel]

I too have not two but 3 layers of builds for compose and buildkit.
I was actually curious to see if dependy build would work differently on buildkit but ran out of time before holidays

[teohhanhui]

docker-compose builds images in depends_on order as a side effect. It's nice for caching when not doing parallel builds, but I don't think buildx bake should replicate the same behaviour.

[FernandoMiguel]

@teohhanhui docker-compose has no support for depends_on at build time.
only at run time

[teohhanhui]

@FernandoMiguel:

docker-compose has no support for depends_on at build time.
only at run time

There is indeed no guarantee about that, but in practice it does work: docker/compose#6332 (comment)

[FernandoMiguel]

not sure what that link is supposed to add?
nothing there says it supports it.
plus, anyone using compose, will probably use parallel which is native on BuildX

until it does offer that option via targets, what I do is have multiple YAML
https://github.com/FernandoMiguel/BuildKit/blob/master/docker-compose.stage-2.yml

[skaldesh]

Is there no plan to support such a feature?
We have exactly the same use case as OP, and isn't it pretty common to have base containers that you need in subsequent builds?
I would like to start one bake that builds my whole collection of containers.

[maxheld83]

If I'm not mistaken, this is now possible, via the contexts = field.

I think this can be closed.

[asonawalla]

If I'm not mistaken, this is now possible, via the contexts = field.

I think this can be closed.

Thanks @maxheld83, this seems to be what I was looking for!

[kvalev]

Updated documentation link: https://docs.docker.com/build/bake/build-contexts/#using-a-result-of-one-target-as-a-base-image-in-another-target