Skip to content

image already exists error when building multiple targets with same tag #3576

@orgads

Description

@orgads

Contributing guidelines

I've found a bug and checked that ...

  • ... the documentation does not mention anything about my problem
  • ... there are no open or closed issues that are related to my problem

Description

I have a sophisticated bake file with multiple targets. 2 of these targets have the same tags, one for local docker (type=docker,load=true) with native platform, and another target that inherits the first one and is multi-platform for the registry (type=registry, platforms = ...).

Once in a while, I'm getting an error "image ... already exists":

ERROR: target final-local: failed to solve: image "myreg.azurecr.io/foo:1.2.3": already exists

It looks like both targets try to export the image at the same time.

Expected behaviour

Avoid this collision. Maybe detect targets with same tags and execute "exporting to image" sequentially for them.

Actual behaviour

They sometimes collide.

Buildx version

github.com/docker/buildx v0.30.1 9e66234

Docker info

Client: Docker Engine - Community
 Version:    29.1.3
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.30.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v5.0.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 9
 Server Version: 29.1.3
 Storage Driver: overlayfs
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: dea7da592f5d1d2b7755e3a161be07f43fad8f75
 runc version: v1.3.4-0-gd6d73eb8
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.14.0-1017-azure
 Operating System: Ubuntu 24.04.3 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 7.753GiB
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: orgads
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
  ::1/128
 Live Restore Enabled: false
 Firewall Backend: iptables

Builders list

NAME/NODE        DRIVER/ENDPOINT                   STATUS    BUILDKIT   PLATFORMS
multiarch*       docker-container                                       
 \_ multiarch0    \_ unix:///var/run/docker.sock   running   v0.26.2    linux/amd64 (+3), linux/arm64, linux/arm (+2), linux/ppc64le, (4 more)
default          docker                                                 
 \_ default       \_ default                       running   v0.26.2    linux/amd64 (+3), linux/arm64, linux/arm (+2), linux/ppc64le, (4 more)

Configuration

I cannot reproduce with this minimal example, but it should be similar to my project.

variable "TAG" {
  default = "1.0.0"
}

platforms_val = ["linux/amd64", "linux/arm64"]

target "local" {
  dockerfile-inline = <<-EOF
FROM scratch
ENV TAG=${TAG}
EOF
  output = ["type=docker,load=true"]
  tags = ["orgads/test:${TAG}"]
}

target "push" {
  inherits  = ["local"]
  platforms = platforms_val
  output    = ["type=registry"]
}

group "default" {
  targets = ["local", "push"]
}

Build logs

11:02:45 #21 [final-test] exporting to image
11:02:45 #21 exporting layers
11:02:45 #21 exporting layers 1.3s done
11:02:45 #21 exporting manifest sha256:6cf0dadb9ab50c461899ad2552625cbe837d5d2b78fe3dc1e3f081d25d865597 0.1s done
11:02:45 #21 exporting config sha256:acfe570a219af400821e7f8425b6ca288a0f8891b24c7f80e5c41ff637b0c710 0.0s done
11:02:45 #21 exporting attestation manifest sha256:6f5ffdd73ea006d572f4fc7fbaf4f0e7adbd933c99e5da0d26c917044dc06024
11:02:46 #21 exporting attestation manifest sha256:6f5ffdd73ea006d572f4fc7fbaf4f0e7adbd933c99e5da0d26c917044dc06024 0.2s done
11:02:46 #21 exporting manifest list sha256:fac09caac58a4be9f492968291b10a6a7dc9763139e3682d36cbe10ce93ecf5d
11:02:46 #21 exporting manifest list sha256:fac09caac58a4be9f492968291b10a6a7dc9763139e3682d36cbe10ce93ecf5d 0.1s done
11:02:46 #21 naming to orgads/test:1.0.0
11:02:46 #21 ...
11:02:46 
11:02:46 #22 [final-local-test] exporting to image
11:02:46 #22 exporting layers 1.3s done
11:02:46 #22 exporting manifest sha256:6cf0dadb9ab50c461899ad2552625cbe837d5d2b78fe3dc1e3f081d25d865597 0.1s done
11:02:46 #22 exporting config sha256:acfe570a219af400821e7f8425b6ca288a0f8891b24c7f80e5c41ff637b0c710 0.1s done
11:02:46 #22 exporting attestation manifest sha256:83a6563a64c8773ee7e126e4f4b20dadf97c94990d11a9f17c7cbad1a56d9fa1 0.2s done
11:02:46 #22 exporting manifest list sha256:f78d2ab95ee6746e1af5ea2c080d6fd75c534e8e4079e05257d6bb45cff981bc 0.1s done
11:02:46 #22 naming to orgads/test:1.0.0 done
11:02:46 #22 ERROR: image "orgads/test:1.0.0": already exists
11:02:46 
11:02:46 #21 [final-test] exporting to image
11:02:46 #21 naming to orgads/test:1.0.0 0.0s done
11:02:46 #21 unpacking to orgads/test:1.0.0
11:02:46 #21 unpacking to orgads/test:1.0.0 0.5s done
11:02:46 #21 CANCELED
11:02:46 ------
11:02:46  > [final-local-test] exporting to image:
11:02:46 ------
11:02:46 ERROR: target final-local-test: failed to solve: image "orgads/test:1.0.0": already exists

Additional info

Can you maybe suggest another way to have multi-platform image pushed and native one stored locally with the same tag?

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions