Feature request: ability to enable Docker Buildkit for sam build

[ghost]

Describe your idea/feature/enhancement

I wish SAM CLI would enable me to enable Docker Buildkit when running sam build for a function with PackageType: Image.

This would enable me to use BuildKit features such as cache mounts for faster and more efficient builds.

Proposal

A very specific approach would be to add a new option along the lines of --use-docker-buildkit or something like that.

That said, something more general might be a good idea.

For example, I tried to enable buildkit by running export DOCKER_BUILDKIT=1 && sam build but it didn’t work.

IIRC docker build supports various other environment variables, so it might be a good idea to have a general purpose mechanism to set environment variables that’d be exposed to the docker build process.

One way to do this might be to simply forward/inherit the environment vars from the user’s shell to/into the docker build process.

Another way could be to add a CLI option like --docker-build-env DOCKER_BUILDKIT=1,FOO=bar

Additional Details

This will not require any updates to the SAM Spec.

[ghost]

Huh, I just found docker/docker-py#2230 and it looks like bad news… 😞

[qingchm]

I know it's going to be an interesting feature to support but since our builds are based on docker API for python, unless docker supports buildkit on python api I don't see a quick solution to this!

[ghost]

Makes sense. Thanks.

Maybe it’d help if maintainers of downstream libs would comment on that issue? Or otherwise somehow convey to the maintainers of docker-py that support for Buildkit would be great?

Thanks!

[qingchm]

@aviflat Yep I would assume opening an issue against their repository would be the way to go!

[ghost]

Well, since docker/docker-py#2230 already exists, I was suggesting that one of the maintainers of this project post a comment to that issue, voicing support for the feature.

[mildaniel]

Since there isn't much we can do on our side, I will close this for now but please re-open when this is actionable for us.

[astahlman]

The lack of BuildKit caching is currently our biggest pain point with SAM. Given that it's been >3.5 years with no progress on the upstream issue in docker-py, would you be open to a PR that replaces the docker-py build call with the docker build CLI command? FWIW, the maintainers recommend using the CLI for builds. This is the approach that docker-compose project took.

Or if you'd rather make the proposed behavior opt-in we could toggle whether we use docker-py or the CLI via a command line flag like --use-docker-buildkit.

[remidebette]

Hi,
Alternatively, for using the docker cli without reimplementing the wheel, the project https://github.com/gabrieldemarmiesse/python-on-whales readily provides a python wrapper on the CLI.

It works fine with buildx