Merge Docker manifests from different runners

[diehard2]

Description

I've followed the steps here - https://docs.docker.com/build/ci/github-actions/multi-platform/
but I've been unable to get it to work for a custom registry. Its also extremely difficult to both understand and debug

adding an option in this action such as

'merge-manifests' that replaces all of that code would make this far simpler. With more runners allowing for ARM builds, this would let people avoid QEMU emulation and parallelize their build configurations far more easily

[crazy-max]

I've followed the steps here - https://docs.docker.com/build/ci/github-actions/multi-platform/
but I've been unable to get it to work for a custom registry.

Can you create a bug report with workflow and logs so we can take a look?

Its also extremely difficult to both understand and debug

Yes this has been raised in #671 as well. We also want to ease distributed builds by using subactions for both build-push-action and bake-action.

'merge-manifests' that replaces all of that code would make this far simpler.

Can you explain how this merge-manifests input would work?

this would let people avoid QEMU emulation

This will not avoid QEMU on GitHub hosted runners, it will just build on a dedicated runner.

[junghans]

I ran into a similar issue now that the arm64 linux runners ( ubuntu-24.04-arm) are online. Previously arm64 image were out of the question as qemu was too slow and MacOS M1 runner cannot build docker containers.

Using Noelware/docker-manifest-action seems to work.

workflow.yml

jobs:
  build:
    strategy:
      matrix:
        config:
          - {arch: 'arm64'}
          - {arch: 'amd64'}
    runs-on: ${{ matrix.config.arch == 'arm64' && 'ubuntu-24.04-arm' || 'ubuntu-latest' }}
    steps:
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      - name: Checkout code    
        uses: actions/checkout@v4
      - name: Build and Push Docker images for all Container Registries
        uses: docker/build-push-action@v6
        with:
          tags: |
            ghcr.io/${{ github.repository }}/build:latest-${{matrix.config.arch}}
          file: Dockerfile 
          push: true
  merge-docker-manifest:
    runs-on: ubuntu-latest
    needs: build
    steps:
      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      - name: Create and push manifest images
        uses: Noelware/docker-manifest-action@master # or use a pinned version in the Releases tab
        with:
          inputs: ghcr.io/${{ github.repository }}/build:latest
          images: ghcr.io/${{ github.repository }}/build:latest-amd64,ghcr.io/${{ github.repository }}/build:latest-arm64           
        push: true

[crazy-max]

now that the amd64 linux runners ( ubuntu-24.04-arm) are online

I guess you meant arm64 and yes we are going to update our docs related to these new runners but will not be up until these runners are GA.

Using Noelware/docker-manifest-action seems to work.

Your workflow creates multiple tags with architecture suffix which is not the desired behavior. In our case https://docs.docker.com/build/ci/github-actions/multi-platform/#distribute-build-across-multiple-runners we are pushing by digest to avoid that. Don't think Noelware/docker-manifest-action supports merging these manifests though as it's using docker manifest command: https://github.com/Noelware/docker-manifest-action/blob/a34eba526b9e4f02939cd3385b52f9cdef9bdf99/src/index.ts#L69

[bo0tzz]

Your workflow creates multiple tags with architecture suffix which is not the desired behavior.

I don't think that's the case, but that the arguments are very unclearly named. As I understand it, the architecture suffix tags are the inputs that get merged into one final manifest without a specific arch suffix.

[junghans]

now that the amd64 linux runners ( ubuntu-24.04-arm) are online

I guess you meant arm64 and yes we are going to update our docs related to these new runners but will not be up until these runners are GA.

Yes, typo!

Using Noelware/docker-manifest-action seems to work.

Your workflow creates multiple tags with architecture suffix which is not the desired behavior. In our case https://docs.docker.com/build/ci/github-actions/multi-platform/#distribute-build-across-multiple-runners we are pushing by digest to avoid that. Don't think Noelware/docker-manifest-action supports merging these manifests though as it's using docker manifest command: https://github.com/Noelware/docker-manifest-action/blob/a34eba526b9e4f02939cd3385b52f9cdef9bdf99/src/index.ts#L69

Yeah, I would like to avoid having the extra tags, but right now I can live with that.

[luigi311]

I dont have a method for creating both via a single job but if you are ok with a more verbose method you can split it out into 3 jobs.

1. The first job is a docker-amd64 which runs on the normal ubuntu-24.04 runner and builds as the name suggests amd64 images to your normal tags
2. The second job is a docker-arm64 which runs on the new ubuntu-24.04-arm runner and builds your arm64 images to your tags but with a suffix of arm64
3. The third job will then run after the two previous jobs are completed and combine the manifests together.

I only push 1 and 2 to the github container registry and then in step 3 i push the combined manifests to dockerhub. This lets me keep dockerhub clean as the arm64 tags wont exist since they arent needed individually.

Here is the code i use to combine the manifests

for TAG in $(jq -r '.tags[]' <<< "$DOCKER_METADATA_OUTPUT_JSON"); do
  echo "Creating manifest tag $TAG"
  # Replace ghcr.io/${{ github.actor }} with docker.io/${{ secrets.DOCKER_USERNAME }} if DOCKER_USERNAME is set
  if [ "${{ secrets.DOCKER_USERNAME }}" != "" ]; then
    DOCKERHUB_TAG=$(echo "--tag $TAG" | sed "s/ghcr.io\/${{ github.actor }}/${{ secrets.DOCKER_USERNAME }}/")
  else
    DOCKERHUB_TAG=""
  fi
  
  docker buildx imagetools create --append "${TAG}-arm64" --tag "${TAG}" ${DOCKERHUB_TAG}
done

You can look at my full github action here
https://github.com/luigi311/tanoshi-builder/blob/d2d3e89df3d08d5e967befbf30cf04091e08e5a9/.github/workflows/ci.yml

I've been doing this for a while with a self hosted runner since qemu building the images were taking forever. This is pretty verbose though so hopefully theres a better way to do this in the future with a single matrix or something.

[luigi311]

No real reason other than I didn't know it existed lol. I'll see if I can swap over to that to make it hopefully easier to understand.

[ro0NL]

@luigi311 👍 we have it working now using https://github.com/Noelware/docker-manifest-action, but it was pita to refactor :')