Of two amd64 images (identical in content, different in metadata) only one can be pulled

[niklassemmler]

/kind bug

Description:

I am trying to pull the docker image for Apache Flink on my Macbook Pro with Apple M1 chip. The image is available in two versions. From the Apache Flink maintainers under apache/flink and as official docker hub image under flink. Both images are built for amd64. The images are identical apart from some meta data.

For some reason, I can only pull apache/flink, but not flink. One striking difference is that flink contains the platform in its manifest and apache/flink does not:

❯ diff <(docker manifest inspect flink:1.14.2) <(docker manifest inspect apache/flink:1.14.2)
2,14c2,70
<    "schemaVersion": 2,
<    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
<    "manifests": [
<       {
<          "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
<          "size": 2840,
<          "digest": "sha256:cc231b650e6e8c238ad9a5e9005deff2535aefb46df996eab3f7c5dcf4b5fca8",
<          "platform": {
<             "architecture": "amd64",
<             "os": "linux"
<          }
<       }
<    ]
---
> 	"schemaVersion": 2,
> 	"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
> 	"config": {
> 		"mediaType": "application/vnd.docker.container.image.v1+json",
> 		"size": 9558,
> 		"digest": "sha256:319b46b3fabd39fb7c6a5a92a491c28978415ee35ef431ad68ec6b44be872919"
> 	},
> 	"layers": [
[...]
> 	]

Is one of these picked up by qemu-user-static to decide that it cannot use the image? I had the same problem with docker desktop. So maybe this problem is caused by the specification?

Steps to reproduce the issue:

Preparation

1. Download docker configuration for limavm
2. Add qemu-user-static and binfmt-support
3. Start a limavm: limactl start --tty=false ./docker.yaml
4. export environment export DOCKER_HOST=unix://$HOME/.lima/docker/sock/docker.sock

Pull images

❯ docker pull apache/flink:1.14.2
1.14.2: Pulling from apache/flink
4c25b3090c26: Pull complete
1acf565088aa: Pull complete
b95c0dd0dc0d: Pull complete
d25bb2549fe9: Pull complete
548d32f71092: Pull complete
dca843ed1d58: Pull complete
f2fd5413b2d6: Pull complete
64fee6d6c14a: Pull complete
c716a49da9cb: Pull complete
34a99ed3da5b: Pull complete
8c6dd6e071c1: Pull complete
22ee51cf0ba0: Pull complete
Digest: sha256:3c2ccf6cdb35df7e0f9e94fa2e17e2c1777639d4c80148abb082cf7a6ac0c2ea
Status: Downloaded newer image for apache/flink:1.14.2
docker.io/apache/flink:1.14.2
❯ docker pull flink:1.14.2
1.14.2: Pulling from library/flink
no matching manifest for linux/arm64/v8 in the manifest list entries

Describe the results you received:

Only apache/flink can be pulled.

Describe the results you expected:

Both images can be pulled.

Environment:

• QEMU version: 6.2.0
• Container application: Lima-VM
• Lima-VM version: 0.8.1
• Macbook with M1 chip

Output of docker version

Client: Docker Engine - Community
 Version:           20.10.12
 API version:       1.41
 Go version:        go1.17.5
 Git commit:        e91ed5707e
 Built:             Sun Dec 12 06:28:24 2021
 OS/Arch:           darwin/arm64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.12
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.12
  Git commit:       459d0df
  Built:            Mon Dec 13 11:43:40 2021
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.4.12
  GitCommit:        7b11cfaabd73bb80907dd23182b9347b4245eb5d
 runc:
  Version:          1.0.2
  GitCommit:        v1.0.2-0-g52b36a2
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0