dockerfile2llb: experimental parser directive for op metadata

[AkihiroSuda]

e.g.

# org.mobyproject.buildkit.experimental = "{WorkerConstraint: ["labels.org.mobyproject.buildkit.worker=host01"]}"
FROM foo
RUN echo I'm on host01
# org.mobyproject.buildkit.experimental = "{NoCache: true}"
RUN echo I'm still on host01, but cache is disabled

# org.mobyproject.buildkit.experimental = "{WorkerConstraint: ["labels.org.mobyproject.buildkit.worker=host02"]}"
FROM foo
RUN echo I'm on host02

[AkihiroSuda]

@tonistiigi @vdemeester @dnephin

Any thought?
These directives won't be available in Docker products, but would be useful during development.
For Docker products, Docker, Inc. should define more human-friendly and simpler directive/instruction.

[vdemeester]

@AkihiroSuda I guess there is already way to pass "op metadata" in the interface between frontends and buildkit right ?

I think we could experiment with that in the frontend in the form of a comment as your example. It doesn't ties us to anything for now while letting us play with workerconstraint, etc… so I'm 👍

[AkihiroSuda]

I guess there is already way to pass "op metadata" in the interface between frontends and buildkit right ?

Not yet (and we need that) 😅

[dnephin]

As long as there is an easy way to disable them it seems fine, otherwise it could make integration more difficult.

[tonistiigi]

I guess there is already way to pass "op metadata" in the interface between frontends and buildkit right ?

Not yet (and we need that) 😅

What's missing? Dockerfile frontend does set the op metadata to track the original command atm.

Any thought?

I'm not sure we should add any new Dockerfile features in this repo. It is easy to get into split head that makes replacing docker build backend hard. I do think we should embrace the fact that BuildKit doesn't require a compiled-in implementation and can use any custom version of a frontend that could be based on the source of any repo. And of course, we should try to work together on all these possible new features, maybe just not in this repo. There are lot of other features like this that have proposals in moby/moby, (EXPORT, RUN --MOUNT, heredoc, secrets, INCLUDE etc.). To manage this we should first decide how we will manage new features so it is a good user experience for our users (my recommendation would be to add a shebang directive for specifying the frontend reference and agree on a versioning scheme).

[AkihiroSuda]

What's missing?

At least, we don't have CLI or something else that human users can interact with to set op metadata.
That's why I opened this issue.

I'm not sure we should add any new Dockerfile features in this repo

OK, and on second thought, we might be able to use CLI flags for setting stage-granularity metadata.

Stage-granularity is not ideal but should cover most usecases.
At least, it should be enough for worker constraints, although not useful for setting ignoreCache.

CLI would be like --frontend-opt stage=0,ignoreCache=true,worker.hostname=ws01.
Or maybe just put .buildkitmeta.toml to the build context and the Dockerfile frontend picks it up when --frontend-opt meta=.buildkitmeta.toml is specified.

my recommendation would be to add a shebang directive for specifying the frontend reference and agree on a versioning scheme

You meant something like #!/usr/bin/env dockerfile-foobar-v42?
Why does it start with shebang? Is it expected to be exec(2)-uted?

[tonistiigi]

CLI would be like --frontend-opt stage=0,ignoreCache=true,worker.hostname=ws01.
Or maybe just put .buildkitmeta.toml to the build context and the Dockerfile frontend picks it up when --frontend-opt meta=.buildkitmeta.toml is specified.

Tbh, I don't know what the best solution for this problem is. I was only commenting about if we should do it in this repo, do #163 first, or find some other repo where we can experiment with this kind of stuff. For ignore-cache I think cli flags per stage are better. Whether to do a fast build or slower build that may be more precise is a decision for the person invoking the builder(cli-flags). At the same time, dockerfile author best knows what sections should be made configurable for this. It is not ideal as all stages that user wants to reevaluate need to be set with a flag separately and dockerfile author needs to split all possible commands that user may want to reevaluate to separate stages. I think ideally dockerfile author would define a build arg (boolean) and how setting that build arg changes the cache behavior but I don't have a good syntax for that atm. For the constraints I think we should be much more careful. There is a proposal for FROM --platform for lcow. We should approach this to solve use cases not to just clone all low-level capabilities LLB has in Dockerfile.

If you are going for cli syntax for stages, I'd recommend. ignore-cache-all=true, ignore-cache=stage0,stage1.

You meant something like #!/usr/bin/env dockerfile-foobar-v42?
Why does it start with shebang? Is it expected to be exec(2)-uted?

#!docker.io/tonistiigi/dockerfile:version Ignore the shebang, it may as well be # version=docker.io/tonistiigi/dockerfile:version if that is simpler to understand. I haven't thought through the version scheme yet. Something where you could opt-in to either stable channels that don't get backward incompatible changes but are maintained for bugfixes. Or choose a set of experimental bleeding edge features that will never get removed but you will not get any other updates(apart from maybe p0 security).

[AkihiroSuda]

This could be a custom frontend image
e.g.

# syntax = foo

FROM bar AS baz WORKERCONSTRAINT oci-runtime=nvidia-oci-runtime

or

RUN --runtime
RUN --entitlement

Also looks like we would have a bunch of usecases in addition to distributed mode

• GPU
• cgroups?
  ...

[AkihiroSuda]

Closing as outdated