chore: create packages

[crazy-max]

Follow-up to our discussion about releasing buildx as a separate package.

I first started to implement this on the docker-ce-packaging repo but the Go version is not the one we want for buildx. I think we should first merge docker/cli#3258.

Creating packages here allows for easier maintenance. In a follow-up we could upload pkgs and update repodata on download.docker.com (cc @StefanScherer).

Test it:

$ docker buildx bake "https://github.com/crazy-max/buildx.git#deb-pkg" pkg

$ dpkg-deb -I bin/docker-buildx_0.7.0~9-g1250d945.m_amd64.deb 
 new Debian package, version 2.0.
 size 12219872 bytes: control archive=419 bytes.
       1 bytes,     0 lines      conffiles
     306 bytes,    10 lines      control
      81 bytes,     1 lines      md5sums
 Package: docker-buildx
 Version: 0:0.7.0~9-g1250d945.m
 Section: default
 Priority: optional
 Architecture: amd64
 Maintainer: Docker <support@docker.com>
 Vendor: Docker
 Installed-Size: 46008
 Homepage: https://github.com/docker/buildx
 Description: Docker Buildx plugin extends build capabilities with BuildKit.

$ rpm -qip bin/docker-buildx-0.7.0~9-g1250d945.m.x86_64.rpm 
Name        : docker-buildx
Epoch       : 0
Version     : 0.7.0~9-g1250d945.m
Release     : 1
Architecture: x86_64
Install Date: (not installed)
Group       : Tools/Docker
Size        : 47112192
License     : Apache-2.0
Signature   : (none)
Source RPM  : docker-buildx-0.7.0~9-g1250d945.m-1.src.rpm
Build Date  : Thu Dec 16 11:31:29 2021
Build Host  : buildkitsandbox
Relocations : (not relocatable)
Packager    : Docker <support@docker.com>
Vendor      : Docker
URL         : https://github.com/docker/buildx
Summary     : Docker Buildx plugin extends build capabilities with BuildKit.
Description :
Docker Buildx plugin extends build capabilities with BuildKit.

cc @thaJeztah @tonistiigi

Signed-off-by: CrazyMax crazy-max@users.noreply.github.com

[thaJeztah]

So IIUC, the idea is for the buildx repository to provide the "recipe" for building the .deb and .rpm packages, and to update the docker-ce-packaging pipeline, to run it as part of releases, correct?

I had a look at what it produces, and noticed that this currently produces static binaries (packaged as .deb / .rpm).

ldd /usr/libexec/docker/cli-plugins/docker-buildx
	linux-vdso.so.1 (0x00007fffc1dc7000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa7e9f0a000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa7e9b19000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fa7ea129000)

I know the current buildx binaries we ship in the docker-ce-cli packages are dynamically linked, and perhaps that's not an issue, but I do recall that @justincormack didn't want to be using static binaries (and I also recall some binaries could not be shipped as static binaries on Docker Desktop, related to some GPL code, but not sure if that's the case for these).

As to naming of the package, I know we currently use docker-scan-plugin (for docker scan); given that these debs/rpms do install a cli plugin, I'm inclined to name them accordingly; both for consistency, and because I'm not sure if there's much value in "obscuring" the fact that it is a plugin. I'll leave that choice to @justincormack though.

[thaJeztah]

As to integrating it with the docker-ce-packaging pipeline; we have per-distro channels at download.docker.com, so there's a couple of things we need to look into on how we'll be integrating this;

• need to look how to integrate it into the "matrix" of distro/architectures we build for, so that the .deb and .rpm files land in the expected directories.
  • We need to double-check if there's differences in the .deb / .rpm metadata that may be different between distro versions. Admitted; the install for this plugin is more straightforward than (e.g.) dockerd or containerd, but I know at some point our release-pipeline tried to build packages using a single distro, and there were incompatibilities between .deb packages built for newer distros when installed on older versions of the distro (see deb pkgs: restart after upgrade TAR-1285 containerd-packaging#113 and [TAR-1285] deb: Replace all instances of try-restart with restart containerd-packaging#115).
  • Depending on whether or not these will be dynamically linked, we need to either build for each distro+distro-version+architecture, or build once, and duplicate the packages for each one.
  • Have a way to generate the right filename for each distro/distro-version (see Debian/Ubuntu package versions contain codename which "break" dist upgrades for-linux#1315)
• man pages are currently not included in the packages; are we planning to include those as well? (or does that depend on Man support cli-docs-tool#14 ?)

Given that buildx is currently stashed in the docker-ce-cli package, and this will move it to a separate package, we need to add the correct metadata for this, so that they cannot be installed side-by-side (which would overwrite each other's binaries);

• For .deb packages, this would be scenario #7 or #8 outlined in https://wiki.debian.org/PackageTransition (combination of Conflicts, Breaks, Depends (or Recommends)).
• For .rpm packages, I think this would be a Conflicts: docker-ce-cli <= <current version> in the buildx package, and a Requires: in the docker-ce-cli package (to force the buildx plugin to be installed).

[thaJeztah]

Depending on whether or not these will be dynamically linked, we need to either build for each distro+distro-version+architecture, or build once, and duplicate the packages for each one.

Hmmm.. so "build once and copy" won't work, as the version (including distro etc) is also part of the package's metadata, so I guess we'll have to build for each distro (or at least package for each distro). Of course; not sure if that's an actual issue if the binary is exactly the same for all of them (i.e., upgrading distro would be fine with keeping the "old" package).

Here's what's in the control file in side the package (compared between the buildx package and the docker-ce-cli) - I haven't checked yet for RPMs;

cat ./bin/docker-buildx_0.7.0~9-g1250d945_amd64/control/control
Package: docker-buildx
Version: 0:0.7.0~9-g1250d945
Section: default
Priority: optional
Architecture: amd64
Maintainer: Docker <support@docker.com>
Vendor: Docker, Inc
Installed-Size: 51408
Homepage: https://github.com/docker/buildx
Description: Docker Buildx plugin extends build capabilities with BuildKit.

cat ./bin/docker-ce-cli_20.10.11~3-0~ubuntu-focal_amd64/control/control
Package: docker-ce-cli
Source: docker-ce
Version: 5:20.10.11~3-0~ubuntu-focal
Architecture: amd64
Maintainer: Docker <support@docker.com>
Installed-Size: 153865
Depends: libc6 (>= 2.4)
Recommends: docker-scan-plugin
Conflicts: docker (<< 1.5~), docker-engine, docker-engine-cs, docker.io, lxc-docker, lxc-docker-virtual-package
Breaks: docker-ce (<< 5:0)
Replaces: docker-ce (<< 5:0)
Section: admin
Priority: optional
Homepage: https://www.docker.com
Description: Docker CLI: the open-source application container engine
 Docker is a product for you to build, ship and run any application as a
 lightweight container
 .
 Docker containers are both hardware-agnostic and platform-agnostic. This means
 they can run anywhere, from your laptop to the largest cloud compute instance and
 everything in between - and they don't require you to use a particular
 language, framework or packaging system. That makes them great building blocks
 for deploying and scaling web apps, databases, and backend services without
 depending on a particular stack or provider.

[crazy-max]

@thaJeztah

So IIUC, the idea is for the buildx repository to provide the "recipe" for building the .deb and .rpm packages, and to update the docker-ce-packaging pipeline, to run it as part of releases, correct?

Yes that's correct

I had a look at what it produces, and noticed that this currently produces static binaries (packaged as .deb / .rpm).

ldd /usr/libexec/docker/cli-plugins/docker-buildx
	linux-vdso.so.1 (0x00007fffc1dc7000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa7e9f0a000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa7e9b19000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fa7ea129000)

I know the current buildx binaries we ship in the docker-ce-cli packages are dynamically linked, and perhaps that's not an issue, but I do recall that @justincormack didn't want to be using static binaries (and I also recall some binaries could not be shipped as static binaries on Docker Desktop, related to some GPL code, but not sure if that's the case for these).

Not sure but I think it depends on the situation. For a cli plugin like buildx it doesn't make much sense and in the contrary adds a lot of overhead imo.

As to naming of the package, I know we currently use docker-scan-plugin (for docker scan); given that these debs/rpms do install a cli plugin, I'm inclined to name them accordingly; both for consistency, and because I'm not sure if there's much value in "obscuring" the fact that it is a plugin. I'll leave that choice to @justincormack though.

Sure let me know if it needs to be changed: https://github.com/crazy-max/buildx/blob/d24a0cdae3563b04d7f1a4e3cee93efa0605b2c1/hack/nfpm.yml#L1

• We need to double-check if there's differences in the .deb / .rpm metadata that may be different between distro versions. Admitted; the install for this plugin is more straightforward than (e.g.) dockerd or containerd, but I know at some point our release-pipeline tried to build packages using a single distro, and there were incompatibilities between .deb packages built for newer distros when installed on older versions of the distro (see

For a cli plugin pkg which only "extracts" without using any triggers or pre(post)install scripts it doesn't seem we can hit this case.

• or build once, and duplicate the packages for each one.

Don't think it would work as the distro ver would be included in the release.

• Have a way to generate the right filename for each distro/distro-version

We can follow the Debian version scheme sure. Smth like docker-buildx_0.7.0~deb11u0 or docker-buildx_0.7.0-el8 for rpm pkgs.

Given that buildx is currently stashed in the docker-ce-cli package, and this will move it to a separate package, we need to add the correct metadata for this, so that they cannot be installed side-by-side (which would overwrite each other's binaries)

Good point

Hmmm.. so "build once and copy" won't work, as the version (including distro etc) is also part of the package's metadata, so I guess we'll have to build for each distro (or at least package for each distro). Of course; not sure if that's an actual issue if the binary is exactly the same for all of them (i.e., upgrading distro would be fine with keeping the "old" package).

I can add a specific ARG to change the release based on the distro.

Here's what's in the control file in side the package (compared between the buildx package and the docker-ce-cli) - I haven't checked yet for RPMs;

deb

$ dpkg-deb -I bin/docker-buildx_0.7.0~9-g1250d945.m_amd64.deb 
 new Debian package, version 2.0.
 size 12219872 bytes: control archive=419 bytes.
       1 bytes,     0 lines      conffiles
     306 bytes,    10 lines      control
      81 bytes,     1 lines      md5sums
 Package: docker-buildx
 Version: 0:0.7.0~9-g1250d945.m
 Section: default
 Priority: optional
 Architecture: amd64
 Maintainer: Docker <support@docker.com>
 Vendor: Docker
 Installed-Size: 46008
 Homepage: https://github.com/docker/buildx
 Description: Docker Buildx plugin extends build capabilities with BuildKit.

Compared with docker-scan-plugin pkg:

$ dpkg-deb -I bin/docker-scan-plugin_0.12.0~debian-bullseye_amd64.deb 
 new Debian package, version 2.0.
 size 3518216 bytes: control archive=556 bytes.
     340 bytes,    11 lines      control
     164 bytes,     2 lines      md5sums
 Package: docker-scan-plugin
 Source: docker-ce (5:20.10.12~3-0~debian-bullseye)
 Version: 0.12.0~debian-bullseye
 Architecture: amd64
 Maintainer: Docker <support@docker.com>
 Installed-Size: 12946
 Enhances: docker-ce-cli
 Section: admin
 Priority: optional
 Homepage: https://github.com/docker/scan-cli-plugin
 Description: Docker scan cli plugin.

rpm

$ rpm -qip bin/docker-buildx-0.7.0~9-g1250d945.m.x86_64.rpm 
Name        : docker-buildx
Epoch       : 0
Version     : 0.7.0~9-g1250d945.m
Release     : 1
Architecture: x86_64
Install Date: (not installed)
Group       : Tools/Docker
Size        : 47112192
License     : Apache-2.0
Signature   : (none)
Source RPM  : docker-buildx-0.7.0~9-g1250d945.m-1.src.rpm
Build Date  : Thu Dec 16 11:31:29 2021
Build Host  : buildkitsandbox
Relocations : (not relocatable)
Packager    : Docker <support@docker.com>
Vendor      : Docker
URL         : https://github.com/docker/buildx
Summary     : Docker Buildx plugin extends build capabilities with BuildKit.
Description :
Docker Buildx plugin extends build capabilities with BuildKit.

Compared with docker-scan-plugin pkg:

$ rpm -qip bin/docker-scan-plugin-0.12.0-3.el8.x86_64.rpm 
warning: bin/docker-scan-plugin-0.12.0-3.el8.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 621e9f35: NOKEY
Name        : docker-scan-plugin
Epoch       : 0
Version     : 0.12.0
Release     : 3.el8
Architecture: x86_64
Install Date: (not installed)
Group       : Tools/Docker
Size        : 13247224
License     : ASL 2.0
Signature   : RSA/SHA512, Mon Dec 13 15:21:53 2021, Key ID c52feb6b621e9f35
Source RPM  : docker-scan-plugin-0.12.0-3.el8.src.rpm
Build Date  : Mon Dec 13 12:45:11 2021
Build Host  : c4ca28c21cdf
Relocations : (not relocatable)
Packager    : Docker <support@docker.com>
Vendor      : Docker
URL         : https://github.com/docker/scan-cli-plugin/
Summary     : Docker Scan plugin for the Docker CLI
Description :
Docker Scan plugin for the Docker CLI.

[crazy-max]

@thaJeztah

Hmmm.. so "build once and copy" won't work, as the version (including distro etc) is also part of the package's metadata, so I guess we'll have to build for each distro (or at least package for each distro). Of course; not sure if that's an actual issue if the binary is exactly the same for all of them (i.e., upgrading distro would be fine with keeping the "old" package).

I can add a specific ARG to change the release based on the distro.

I added the possibility to define a release version for each distro. See https://github.com/crazy-max/buildx/blob/23d8eaffa41e2748b0b7894bb239290352745c4b/docker-bake.hcl#L10-L15

12220824 Dec 16 18:46 docker-buildx-0.7.0~9-gd24a0cda.m-centos7.x86_64.rpm
12220824 Dec 16 18:46 docker-buildx-0.7.0~9-gd24a0cda.m-centos8.x86_64.rpm
12220836 Dec 16 18:46 docker-buildx-0.7.0~9-gd24a0cda.m-fedora33.x86_64.rpm
12220836 Dec 16 18:46 docker-buildx-0.7.0~9-gd24a0cda.m-fedora34.x86_64.rpm
12220836 Dec 16 18:47 docker-buildx-0.7.0~9-gd24a0cda.m-fedora35.x86_64.rpm
12220812 Dec 16 18:47 docker-buildx-0.7.0~9-gd24a0cda.m-ol8.x86_64.rpm
12220824 Dec 16 18:47 docker-buildx-0.7.0~9-gd24a0cda.m-rhel7.x86_64.rpm
12220824 Dec 16 18:47 docker-buildx-0.7.0~9-gd24a0cda.m-rocky8.x86_64.rpm
12219878 Dec 16 18:46 docker-buildx_0.7.0~9-gd24a0cda.m-debian10_amd64.deb
12219874 Dec 16 18:46 docker-buildx_0.7.0~9-gd24a0cda.m-debian11_amd64.deb
12219880 Dec 16 18:47 docker-buildx_0.7.0~9-gd24a0cda.m-raspbian10_amd64.deb
12219880 Dec 16 18:47 docker-buildx_0.7.0~9-gd24a0cda.m-raspbian11_amd64.deb
12219878 Dec 16 18:46 docker-buildx_0.7.0~9-gd24a0cda.m-ubuntu1804_amd64.deb
12219880 Dec 16 18:46 docker-buildx_0.7.0~9-gd24a0cda.m-ubuntu2004_amd64.deb
12219878 Dec 16 18:47 docker-buildx_0.7.0~9-gd24a0cda.m-ubuntu2104_amd64.deb
12219878 Dec 16 18:47 docker-buildx_0.7.0~9-gd24a0cda.m-ubuntu2110_amd64.deb

[crazy-max]

@thaJeztah Last commit publish pkgs to docker/buildx-pkg. Will look like this: https://hub.docker.com/r/crazymax/buildx-pkg/tags

Structure inside the scratch image:

[crazy-max]

Changed pkg name to docker-buildx-plugin as discussed. PTAL @thaJeztah @tonistiigi.