Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker-image does not yet support --target #13934

Closed
vputz opened this issue Dec 21, 2021 · 12 comments · Fixed by #13956
Closed

docker-image does not yet support --target #13934

vputz opened this issue Dec 21, 2021 · 12 comments · Fixed by #13956

Comments

@vputz
Copy link

vputz commented Dec 21, 2021

(pants 2.8)

The new docker-image functionality is excellent, but does not currently support the --target argument of docker build which allows for multistage builds (https://docs.docker.com/develop/develop-images/multistage-build/). In our application for example the containers can be build from a single Dockerfile using --target dev to build a the "development version" of a container. It would be very useful to have this!

ie

docker_image(
  dependencies=[...]
  target="my-target"
)

Many thanks!

@kaos
Copy link
Member

kaos commented Dec 21, 2021

Thanks!

One question, would it make sense to be able to provide what target to build for globally (i.e. on the command line) or is it always the same target for each docker_image?

@kaos
Copy link
Member

kaos commented Dec 21, 2021

Also, I'm thinking it may be a good idea to use a little more verbose, but less ambiguous field name, perhaps target_stage to avoid overloading the meaning of target in the Pantsbuild domain.

@vputz
Copy link
Author

vputz commented Dec 21, 2021

Possibly so. With regard to the command line, possibly yes? It would eliminate the need for multiple targets (meaning pants targets, so that validates your second comment :) ). I had started doing this in preparation:

docker_image(dependencies=[...], target="dev", tag="dev-containers")
docker_image(dependencies=[...same...], target="prod", tag="prod-containers")

...which is a bit of unnecessary duplication, but allows the mass build of one type or another with the tags. I'm honestly undecided on whether that's too much duplication or if passing the target through with the command line would be better.

In our case, we have a "local development pipeline" which builds everything with the "dev" target, and then a separate one for the production images, so we don't do it by-image on the command line typically. But if it's easy to add it as a command-line global arg, that would eliminate duplication in the BUILD files and you could "build all things tagged as containers, but only to the dev stage".

@kaos
Copy link
Member

kaos commented Dec 21, 2021

[...] build all things tagged as containers, but only to the dev stage

Yea, that's kind of what I imagined it could be used for. Will do that, as I think it's worth saving in on those duplications when possible.

@Eric-Arellano
Copy link
Contributor

Alternatively, is this an example of the need for parametrization? Currently that's achieved via multiple targets for the same entity, but #13882 speculates how we could have better first class support.

@kaos
Copy link
Member

kaos commented Dec 21, 2021

is this an example of the need for parametrization?

Yes and no, I think. Wouldn't it mean it builds all variations each time? In this case, that's not desirable, unless the output image also uses the target_stage in the image_tags in some way, to distinguish them.

Also from the described use case by @vputz

we have a "local development pipeline" which builds everything with the "dev" target, and then a separate one for the production images

Indicates two distinct runs, one for each target stage, so specifying which to build at run time would be needed.

@kaos
Copy link
Member

kaos commented Dec 21, 2021

@vputz speaking of image tags.. would it be desirable to have a way to tag the images with the stage being built?.. perhaps something like:

docker_image(
  image_tags=["1.2-{target_stage}"],
  ...
)

@Eric-Arellano
Copy link
Contributor

Wouldn't it mean it builds all variations each time?

Parametrization probably needs a way to only run one of the cases. But yeah, to be designed.

@vputz
Copy link
Author

vputz commented Dec 22, 2021

@kaos -- parameterization on tagging would indeed be useful, although not just for the target stage--we also currently tag by date of build and current git hash (given that it's a monorepo, this isn't optimal because two containers could be identical if an unrelated directory of the git tree changes, but would require a new hash and upload because the git hash is different. Tagging by some pants-inferred uniqueness hash only dependent on dependencies would be lovely but that's beyond my brain at the moment.

@kaos
Copy link
Member

kaos commented Dec 22, 2021

Thanks for those tidbits of usefulness. Worth considering. There is #13724 regarding presenting git data to rules. The "dependencies hash" for a target is interesting as well, will ponder on that for a bit.

@kaos
Copy link
Member

kaos commented Dec 22, 2021

Made a poc of the dep hash thing in #13959.
It uses the build args, environment and input sources for the hash value, so if any of those change, the image would change and thus has a different hash.

@vputz
Copy link
Author

vputz commented Dec 22, 2021

Fascinating! Hmm, that could be really handy for us; thanks!

kaos added a commit that referenced this issue Dec 27, 2021
Introduce a new `target_stage` field for `docker_image` as well as the `[docker].build_target_stage` option.

Fixes #13934
kaos added a commit to kaos/pants that referenced this issue Dec 27, 2021
Introduce a new `target_stage` field for `docker_image` as well as the `[docker].build_target_stage` option.

Fixes pantsbuild#13934

[ci skip-rust]

[ci skip-build-wheels]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants