Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publish containers to docker.io #114

Merged
merged 4 commits into from
Nov 11, 2024
Merged

Publish containers to docker.io #114

merged 4 commits into from
Nov 11, 2024

Conversation

uatuko
Copy link
Owner

@uatuko uatuko commented Nov 9, 2024

There seems to be an issue with ghcr.io where published containers can't be pulled ('manifest unknown' error), so moving to docker.io.

Update: docker/build-push-action#820 (comment) suggested the issue could be due to cross-compiling/how multi-arch images are created. After updating the builds to use g++ cross-compile (instead of using QEMU to create multi-arch containers) it seems docker can pull ghcr.io images now.

Notes

10th November, 2024

Inspecting the builds produced by https://github.com/uatuko/ruek/actions/runs/11767516276 seems to show both ghcr.io and docker.io having the same build manifests.

ghcr.io/uatuko/ruek
$ docker buildx imagetools inspect ghcr.io/uatuko/ruek
Name:      ghcr.io/uatuko/ruek:latest
MediaType: application/vnd.oci.image.index.v1+json
Digest:    sha256:ef793b77c8d85473ad96fadb3309bbad2a847ef58d9c931cfaba5b2b682c5320
           
Manifests: 
  Name:        ghcr.io/uatuko/ruek:latest@sha256:c9835b44612c7004bbcd2b2e99ae5083374547aceb6c8725ce45e2979077cc6a
  MediaType:   application/vnd.oci.image.manifest.v1+json
  Platform:    linux/amd64
               
  Name:        ghcr.io/uatuko/ruek:latest@sha256:938d19eed563953d922a5182803d41c4f948256b61987e710a10b80fffb33532
  MediaType:   application/vnd.oci.image.manifest.v1+json
  Platform:    linux/arm64
               
  Name:        ghcr.io/uatuko/ruek:latest@sha256:68f30d71fa085f00e50515336eb7bda17d9ec06455e417c03ad24debb629a3aa
  MediaType:   application/vnd.oci.image.manifest.v1+json
  Platform:    unknown/unknown
  Annotations: 
    vnd.docker.reference.digest: sha256:c9835b44612c7004bbcd2b2e99ae5083374547aceb6c8725ce45e2979077cc6a
    vnd.docker.reference.type:   attestation-manifest
               
  Name:        ghcr.io/uatuko/ruek:latest@sha256:50eb440b657ed3ddd06b0098ca4413441e70dc254942e0c230aaf22c110bb4fe
  MediaType:   application/vnd.oci.image.manifest.v1+json
  Platform:    unknown/unknown
  Annotations: 
    vnd.docker.reference.digest: sha256:938d19eed563953d922a5182803d41c4f948256b61987e710a10b80fffb33532
    vnd.docker.reference.type:   attestation-manifest
docker.io/uatuko/ruek
$ docker buildx imagetools inspect uatuko/ruek
Name:      docker.io/uatuko/ruek:latest
MediaType: application/vnd.oci.image.index.v1+json
Digest:    sha256:ef793b77c8d85473ad96fadb3309bbad2a847ef58d9c931cfaba5b2b682c5320
           
Manifests: 
  Name:        docker.io/uatuko/ruek:latest@sha256:c9835b44612c7004bbcd2b2e99ae5083374547aceb6c8725ce45e2979077cc6a
  MediaType:   application/vnd.oci.image.manifest.v1+json
  Platform:    linux/amd64
               
  Name:        docker.io/uatuko/ruek:latest@sha256:938d19eed563953d922a5182803d41c4f948256b61987e710a10b80fffb33532
  MediaType:   application/vnd.oci.image.manifest.v1+json
  Platform:    linux/arm64
               
  Name:        docker.io/uatuko/ruek:latest@sha256:68f30d71fa085f00e50515336eb7bda17d9ec06455e417c03ad24debb629a3aa
  MediaType:   application/vnd.oci.image.manifest.v1+json
  Platform:    unknown/unknown
  Annotations: 
    vnd.docker.reference.digest: sha256:c9835b44612c7004bbcd2b2e99ae5083374547aceb6c8725ce45e2979077cc6a
    vnd.docker.reference.type:   attestation-manifest
               
  Name:        docker.io/uatuko/ruek:latest@sha256:50eb440b657ed3ddd06b0098ca4413441e70dc254942e0c230aaf22c110bb4fe
  MediaType:   application/vnd.oci.image.manifest.v1+json
  Platform:    unknown/unknown
  Annotations: 
    vnd.docker.reference.type:   attestation-manifest
    vnd.docker.reference.digest: sha256:938d19eed563953d922a5182803d41c4f948256b61987e710a10b80fffb33532

However, while docker.io images can be pulled ghcr.io images gives "manifest unknown" error.

❯ podman version
Client:       Podman Engine
Version:      5.0.2
API Version:  5.0.2
Go Version:   go1.22.2
Built:        Thu Jan  1 01:00:00 1970
OS/Arch:      darwin/arm64

Server:       Podman Engine
Version:      5.0.2
API Version:  5.0.2
Go Version:   go1.21.9
Built:        Wed Apr 17 01:00:00 2024
OS/Arch:      linux/arm64
❯ podman pull ghcr.io/uatuko/ruek
Error: copying system image from manifest list: determining manifest MIME type for docker://ghcr.io/uatuko/ruek:latest: reading manifest sha256:938d19eed563953d922a5182803d41c4f948256b61987e710a10b80fffb33532 in ghcr.io/uatuko/ruek: manifest unknown
❯ podman pull docker.io/uatuko/ruek
Trying to pull docker.io/uatuko/ruek:latest...
Getting image source signatures
Copying blob sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
Copying blob sha256:83d624c4be2db5b81ae220b6b10cbc9a559d5800fd32556f4020727098f71ed0
Copying blob sha256:433e9a8d63bc93d3fa1577ea8dd83d5ae2b459fc6769dc02712b11c964bd517a
Copying blob sha256:4de672e29a23ae2e35a92b07b9bf8dec9e9f9b3a50a92f926539c6c83a19b30a
Copying config sha256:de489992aed217550d385f377fd9f66407d5d79d99d31c6723cb2ff44bf9b066
Writing manifest to image destination
de489992aed217550d385f377fd9f66407d5d79d99d31c6723cb2ff44bf9b066

Update - 1: It used to be both podman and docker gave the same "manifest unknown" result. However, after inspecting images using docker and pulling from docker.io first, I can now pull ghcr.io/uatuko/ruek image as well (could be a coincident?)

$ docker version
Client: Docker Engine - Community
 Version:           27.3.1
 API version:       1.47
 Go version:        go1.22.7
 Git commit:        ce12230
 Built:             Fri Sep 20 11:40:59 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          27.3.1
  API version:      1.47 (minimum version 1.24)
  Go version:       go1.22.7
  Git commit:       41ca978
  Built:            Fri Sep 20 11:40:59 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.22
  GitCommit:        7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c
 runc:
  Version:          1.1.14
  GitCommit:        v1.1.14-0-g2c9f560
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
$ docker pull ghcr.io/uatuko/ruek
Using default tag: latest
latest: Pulling from uatuko/ruek
a480a496ba95: Pull complete 
17343da46eca: Pull complete 
4de672e29a23: Pull complete 
4f4fb700ef54: Pull complete 
Digest: sha256:ef793b77c8d85473ad96fadb3309bbad2a847ef58d9c931cfaba5b2b682c5320
Status: Downloaded newer image for ghcr.io/uatuko/ruek:latest
ghcr.io/uatuko/ruek:latest

Update - 2: Well, it looks like after https://github.com/uatuko/ruek/actions/runs/11768032663 even podman now works. But the docker.io image was pulled before the ghcr.io image.

❯ podman pull docker.io/uatuko/ruek
Trying to pull docker.io/uatuko/ruek:latest...
Getting image source signatures
Copying blob sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
Copying blob sha256:4fbb5dc64dfa7c0fffc8c2ff6b3569d2a08f5175a0f673f53fec8a34de981616
Copying blob sha256:2ce94b48a51d5ad0d5daada3ca8e061912e32c4cf8163d027b3262a74b2ee451
Copying blob sha256:83d624c4be2db5b81ae220b6b10cbc9a559d5800fd32556f4020727098f71ed0
Copying config sha256:a9179c9cc06fde60cf246638c980a0a7cb3212a75d34f9b40873cefa00de0a5e
Writing manifest to image destination
a9179c9cc06fde60cf246638c980a0a7cb3212a75d34f9b40873cefa00de0a5e
❯ podman pull ghcr.io/uatuko/ruek
Trying to pull ghcr.io/uatuko/ruek:latest...
Getting image source signatures
Copying blob sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
Copying blob sha256:83d624c4be2db5b81ae220b6b10cbc9a559d5800fd32556f4020727098f71ed0
Copying blob sha256:4fbb5dc64dfa7c0fffc8c2ff6b3569d2a08f5175a0f673f53fec8a34de981616
Copying blob sha256:2ce94b48a51d5ad0d5daada3ca8e061912e32c4cf8163d027b3262a74b2ee451
Copying config sha256:a9179c9cc06fde60cf246638c980a0a7cb3212a75d34f9b40873cefa00de0a5e
Writing manifest to image destination
a9179c9cc06fde60cf246638c980a0a7cb3212a75d34f9b40873cefa00de0a5e

There seems to be an issue with ghcr.io where published containers can't
be pulled ('manifest unknown' error), so moving to docker.io.
@uatuko uatuko force-pushed the feature/publish-docker-io branch 2 times, most recently from 22039ea to 9c86b1c Compare November 9, 2024 22:53
Copy link

codecov bot commented Nov 9, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.88%. Comparing base (96d61f7) to head (c2c7a87).
Report is 5 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #114   +/-   ##
=======================================
  Coverage   93.88%   93.88%           
=======================================
  Files          21       21           
  Lines        1406     1406           
  Branches      168      168           
=======================================
  Hits         1320     1320           
  Misses         59       59           
  Partials       27       27           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@uatuko uatuko marked this pull request as ready for review November 9, 2024 23:15
@uatuko uatuko marked this pull request as draft November 10, 2024 01:02
@uatuko uatuko force-pushed the feature/publish-docker-io branch 3 times, most recently from 9f722f2 to 7fba022 Compare November 10, 2024 18:26
Publish containers to both docker.io and ghcr.io since using g++
cross-compile seems to have resolved the "manifest unknown" issue for
`docker` (`podman` v5.0.2 still has the issue).
@uatuko uatuko force-pushed the feature/publish-docker-io branch 3 times, most recently from dae0419 to c2c7a87 Compare November 10, 2024 20:01
@uatuko uatuko marked this pull request as ready for review November 10, 2024 20:02
@uatuko uatuko merged commit 996692d into main Nov 11, 2024
4 checks passed
@uatuko uatuko deleted the feature/publish-docker-io branch November 11, 2024 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant