Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for buildx #131

Merged
merged 2 commits into from
Mar 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -168,12 +168,22 @@ docker / imageNames := Seq(

Use the key `docker / buildOptions` to set build options.

Example:
#### cross-platform
The `platforms` parameter enables the cross-platform build.
With valuing this parameter the docker image will build using `buildx` command and the host environment should already been set up for.

⚠️ For using the cross builds you need QEMU binaries
```shell
docker run --privileged --rm tonistiigi/binfmt --install all
```

#### Example:
```scala
docker / buildOptions := BuildOptions(
cache = false,
removeIntermediateContainers = BuildOptions.Remove.Always,
pullBaseImage = BuildOptions.Pull.Always,
platforms = List("linux/arm64/v8"),
additionalArguments = Seq("--add-host", "127.0.0.1:12345", "--compress")
)
```
Expand Down
2 changes: 1 addition & 1 deletion project/build.properties
Original file line number Diff line number Diff line change
@@ -1 +1 @@
sbt.version=1.6.2
sbt.version=1.8.0
14 changes: 12 additions & 2 deletions src/main/scala/sbtdocker/DockerBuild.scala
Original file line number Diff line number Diff line change
Expand Up @@ -113,19 +113,23 @@ object DockerBuild {
var lines = Seq.empty[String]

def runBuild(buildKitSupport: Boolean): Int = {
val buildX = if (buildOptions.platforms.isEmpty) Nil else List("buildx")
val load = if (buildOptions.platforms.isEmpty) Nil else List("--load")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The --load parameter is needed for buildx to create the image

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't the --load flag only work for single-platform builds?
From docker docs:

Currently, multi-platform images cannot be exported with the docker export type. The most common usecase for multi-platform images is to directly push to a registry (see registry).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pushing directly to a registry makes breaking changes in this library. Feel free to create a PR that supports multiple platforms in one build

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure that's currently possible when the docker buildx build command doesn't support it.

val buildOptionFlags = generateBuildOptionFlags(buildOptions)
val buildKitFlags = if (buildKitSupport) List("--progress=plain") else Nil
val buildArgumentFlags = buildArguments.toList.flatMap {
case (key, value) =>
Seq(s"--build-arg", s"$key=$value")
}
val command: Seq[String] = dockerPath ::
buildX :::
"build" ::
buildOptionFlags :::
buildKitFlags :::
buildArgumentFlags :::
load :::
"--file" ::
dockerfilePath.name ::
dockerfileAbsolutePath.getPath ::
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using an absolute path makes the command more readable and portable

dockerfileAbsolutePath.getParentFile.getPath ::
Nil
log.debug(s"Running command: '${command.mkString(" ")}'")
Expand Down Expand Up @@ -187,18 +191,24 @@ object DockerBuild {
}
"--pull=" + value
}
val platformsFlag: List[String] = buildOptions.platforms match {
case Seq() => Nil
case platforms => List(s"--platform=${platforms.mkString(",")}")
}

cacheFlag :: removeFlag :: pullFlag :: buildOptions.additionalArguments.toList
cacheFlag :: removeFlag :: pullFlag :: platformsFlag ::: buildOptions.additionalArguments.toList
}

private val SuccessfullyBuilt = "^Successfully built ([0-9a-f]+)$".r
private val SuccessfullyBuiltBuildKit = ".* writing image sha256:([0-9a-f]+) .*\\bdone$".r
private val SuccessfullyBuiltBuildx = ".* exporting config sha256:([0-9a-f]+) .*\\bdone$".r
private val SuccessfullyBuiltPodman = "^([0-9a-f]{64})$".r

private[sbtdocker] def parseImageId(lines: Seq[String]): Option[ImageId] = {
lines.collect {
case SuccessfullyBuilt(id) => ImageId(id)
case SuccessfullyBuiltBuildKit(id) => ImageId(id)
case SuccessfullyBuiltBuildx(id) => ImageId(id)
case SuccessfullyBuiltPodman(id) => ImageId(id)
}.lastOption
}
Expand Down
2 changes: 2 additions & 0 deletions src/main/scala/sbtdocker/models.scala
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,14 @@ object BuildOptions {
* @param cache Use cache when building the image.
* @param removeIntermediateContainers Remove intermediate containers after a build.
* @param pullBaseImage Always attempts to pull a newer version of the base image.
* @param platforms Allows cross platform builds. Make sure that you already set docker's buildx up on the machine that uses this feature
* @param additionalArguments Provide any other arguments to the `docker build` task, see reference at https://docs.docker.com/engine/reference/commandline/build/#options. For example `Seq("--add-host", "127.0.0.1:12345", "--compress")`.
*/
final case class BuildOptions(
cache: Boolean = true,
removeIntermediateContainers: BuildOptions.Remove.Option = BuildOptions.Remove.OnSuccess,
pullBaseImage: BuildOptions.Pull.Option = BuildOptions.Pull.IfMissing,
platforms: Seq[String] = Seq.empty,
additionalArguments: Seq[String] = Seq.empty
)

Expand Down
20 changes: 20 additions & 0 deletions src/test/scala/sbtdocker/DockerBuildSpec.scala
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,14 @@ class DockerBuildSpec extends AnyFreeSpec with Matchers {
flags should contain("--pull=true")
}

"Add platform argument for cross build" in {
val options = BuildOptions(platforms = List("linux/amd64", "linux/arm64"))
val flags = DockerBuild.generateBuildOptionFlags(options)

flags should contain("--platform=linux/amd64,linux/arm64")

}

"Custom arguments" in {
val options = BuildOptions(additionalArguments = Seq("--add-host", "127.0.0.1:12345", "--compress"))
val flags = DockerBuild.generateBuildOptionFlags(options)
Expand All @@ -117,6 +125,18 @@ class DockerBuildSpec extends AnyFreeSpec with Matchers {
DockerBuild.parseImageId(lines) shouldEqual Some(ImageId("353fcb84af6b"))
}

"Docker buildx output" in {
val lines = Seq(
"#7 exporting layers 0.4s done",
"#7 exporting manifest sha256:427ff04564194e7d44a5790d5739465789861cb5ee6db60ccdb15388865dfd64 0.0s done",
"#7 exporting config sha256:d1d7dbb3987417e91239032f2a36a2ede76e62276849ebf362004c14d6fc82ac",
"#7 exporting config sha256:d1d7dbb3987417e91239032f2a36a2ede76e62276849ebf362004c14d6fc82ac 0.0s done",
"#7 sending tarball"
)
DockerBuild.parseImageId(lines) shouldEqual Some(ImageId("d1d7dbb3987417e91239032f2a36a2ede76e62276849ebf362004c14d6fc82ac"))

}

"Docker build output version 20.10.10" in {
val lines = Seq(
"#6 exporting to image",
Expand Down