Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: Disable SLSA Provenance metadata for our images #144

Merged
merged 1 commit into from
May 3, 2023

Conversation

tsibley
Copy link
Member

@tsibley tsibley commented Apr 26, 2023

Provenance metadata was newly enabled by default in buildx 0.10.0¹ and entails switching from older, Docker image manifest lists (application/vnd.docker.distribution.manifest.list.v2+json) to the newer, but roughly equivalent-in-concept, OCI image indexes (application/vnd.oci.image.index.v1+json).²

This switch happened automatically for us between these two builds:

nextstrain/base:build-20230119T003940Z
nextstrain/base:build-20230321T203820Z

due to an upgrade of buildx from 0.9.1 → 0.10.0 on the GitHub Actions runners we use for CI.

Unfortunately, Singularity doesn't support OCI image indexes (i.e. multi-platform images) until 3.6.0.³ While we'll likely need to require a newer Singularity version at some point anyway⁴, disabling the provenance metadata for now should restore compatibility with Singularity back to its 2.6 series. We don't need the provenance anyhow.

¹ https://github.com/docker/buildx/releases/tag/v0.10.0

² Provenance metadata is attached as additional entries in the image
index alongside the normal entries for each platform image.

³ nextstrain/cli#267
nextstrain/cli#274

Resolves nextstrain/cli#267.

Testing

  • Inspect media type of the image built for this PR
  • Test it works with older Singularity
  • Checks pass

Provenance metadata was newly enabled by default in buildx 0.10.0¹ and
entails switching from older, Docker image manifest lists
(application/vnd.docker.distribution.manifest.list.v2+json) to the
newer, but roughly equivalent-in-concept, OCI image indexes
(application/vnd.oci.image.index.v1+json).²

This switch happened automatically for us between these two builds:

    nextstrain/base:build-20230119T003940Z
    nextstrain/base:build-20230321T203820Z

due to an upgrade of buildx from 0.9.1 → 0.10.0 on the GitHub Actions
runners we use for CI.

Unfortunately, Singularity doesn't support OCI image indexes (i.e.
multi-platform images) until 3.6.0.³  While we'll likely need to require
a newer Singularity version at some point anyway⁴, disabling the
provenance metadata for now should restore compatibility with
Singularity back to its 2.6 series.  We don't need the provenance
anyhow.

¹ <https://github.com/docker/buildx/releases/tag/v0.10.0>

² Provenance metadata is attached as additional entries in the image
  index alongside the normal entries for each platform image.

³ <nextstrain/cli#267>
⁴ <nextstrain/cli#274>
@tsibley
Copy link
Member Author

tsibley commented Apr 26, 2023

Media type looks good:

$ docker run --rm -it -v ~/.docker/config.json:/config.json:ro quay.io/skopeo/stable inspect --raw docker://docker.io/nextstrain/base:branch-trs-disable-provenance | jq -r .mediaType
application/vnd.docker.distribution.manifest.list.v2+json

@tsibley
Copy link
Member Author

tsibley commented Apr 26, 2023

Works with singularity build on 3.5.3:

$ ./built/bin/singularity --version
singularity version 3.5.3

$ ./built/bin/singularity build --disable-cache tmp.sif docker://nextstrain/base:build-20230321T203820Z
INFO:    Starting build...
FATAL:   While performing build: conveyor failed to get: Error initializing source docker://nextstrain/base:build-20230321T203820Z: Error reading manifest build-20230321T203820Z in docker.io/nextstrain/base: manifest unknown: OCI index found, but accept header does not support OCI indexes

$ ./built/bin/singularity build --disable-cache tmp.sif docker://nextstrain/base:branch-trs-disable-provenance
INFO:    Starting build...
2023/04/26 16:27:56  info unpack layer: sha256:26c5c85e47da3022f1bdb9a112103646c5c29517d757e95426f16e4bd9533405
2023/04/26 16:27:57  info unpack layer: sha256:9e79879be9c7cd8edaba21c88fa4be66ea65e7a3f56f98e7820bcb7dedac3b00
…

and also via Nextstrain CLI:

$ PATH="$HOME/src/singularity/built/bin:$PATH" NEXTSTRAIN_SINGULARITY_IMAGE=docker://nextstrain/base:build-20230417T170517Z nextstrain setup singularity
Setting up singularity…
Updating Singularity image docker://nextstrain/base:build-20230417T170517Z…

INFO:    Starting build...
FATAL:   While performing build: conveyor failed to get: Error reading manifest build-20230417T170517Z in docker.io/nextstrain/base: manifest unknown: OCI index found, but accept header does
not support OCI indexes

Set up failed!


$ PATH="$HOME/src/singularity/built/bin:$PATH" NEXTSTRAIN_SINGULARITY_IMAGE=docker://nextstrain/base:branch-trs-disable-provenance nextstrain setup singularity
Setting up singularity…
Updating Singularity image docker://nextstrain/base:branch-trs-disable-provenance…

INFO:    Starting build...
Getting image source signatures
Copying blob 26c5c85e47da done
Copying blob 9e79879be9c7 done
Copying blob 9ad47fcd2c0c done
…

@tsibley tsibley marked this pull request as ready for review April 26, 2023 23:36
@tsibley tsibley requested a review from a team April 26, 2023 23:36
@tsibley tsibley merged commit a8a159f into master May 3, 2023
@tsibley tsibley deleted the trs/disable-provenance branch May 3, 2023 23:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

nextstrain update singularity throws error about OCI indexes with Singularity 2.6 and 3.5.3
2 participants