Support new OCI image index content type to get docker image digest from registry (content digest) #548
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…rom registry (content digest) - ubuntu:20.04 is now only available with new 'application/vnd.oci.image.index.v1+json' - ubuntu:12.04 is only available with old 'application/vnd.docker.distribution.manifest.v2+json' (and should hopefully stay that way in the future, for tests stability) - asking only the new oci.image.index triggers an implicit fallback to 'application/vnd.docker.distribution.manifest.v1+prettyjws', with a different digest, we don't want that (we want digest stability for base image cache) => ask both content types, priority to new one. Added a quick unit test testing get_image_digest on both old and new content-types (hopefully it should stay stable on docker hub side; if it changes, we should adapt it). Manually tested test failures when removing either of the accepted_content_types.
|
Kudos, SonarCloud Quality Gate passed!
|
|
For reference: Manual tests on old & new images, with various combinations of 'accept' headerold ubuntu:16.04headers = {'Accept': 'application/vnd.docker.distribution.manifest.v2+json'} response.headers = {
"content-length": "1150",
"content-type": "application/vnd.docker.distribution.manifest.v2+json",
"date": "Wed, 01 Feb 2023 10:25:32 GMT",
"docker-content-digest": "sha256:a3785f78ab8547ae2710c89e627783cfa7ee7824d3468cae6835c9f4eae23ff7",
"docker-distribution-api-version": "registry/2.0",
"docker-ratelimit-source": "78ffd34a-5c31-4cdb-9281-74f2526563db",
"etag": "\"sha256:a3785f78ab8547ae2710c89e627783cfa7ee7824d3468cae6835c9f4eae23ff7\"",
"strict-transport-security": "max-age=31536000"
}print(response.text) {
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"config": {
"mediaType": "application/vnd.docker.container.image.v1+json",
"size": 3362,
"digest": "sha256:b6f50765242581c887ff1acc2511fa2d885c52d8fb3ac8c4bba131fd86567f2e"
},
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 46497548,
"digest": "sha256:58690f9b18fca6469a14da4e212c96849469f9b1be6661d2342a4bf01774aa50"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 857,
"digest": "sha256:b51569e7c50720acf6860327847fe342a1afbe148d24c529fb81df105e3eed01"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 528,
"digest": "sha256:da8ef40b9ecabc2679fe2419957220c0272a965c5cf7e0269fa1aeeb8c56f2e1"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 170,
"digest": "sha256:fb15d46c38dcd1ea0b1990006c3366ecd10c79d374f341687eb2cb23a2c8672e"
}
]
}new ubuntu:16.04headers = {'Accept': 'application/vnd.oci.image.index.v1+json'} => it return an even older manifest/schema; strange response2.headers {
"content-length": "5522",
"content-type": "application/vnd.docker.distribution.manifest.v1+prettyjws",
"date": "Wed, 01 Feb 2023 10:29:35 GMT",
"docker-content-digest": "sha256:f1a6359df04fcddd5b99b3643125390a363ae2059a5b95992c6718e31ab8ea4b",
"docker-distribution-api-version": "registry/2.0",
"docker-ratelimit-source": "78ffd34a-5c31-4cdb-9281-74f2526563db",
"etag": "\"sha256:f1a6359df04fcddd5b99b3643125390a363ae2059a5b95992c6718e31ab8ea4b\"",
"strict-transport-security": "max-age=31536000"
}print(response2.text) {
"schemaVersion": 1,
"name": "library/ubuntu",
"tag": "16.04",
"architecture": "amd64",
"fsLayers": [
{
"blobSum": "sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4"
},
{
"blobSum": "sha256:fb15d46c38dcd1ea0b1990006c3366ecd10c79d374f341687eb2cb23a2c8672e"
},
{
"blobSum": "sha256:da8ef40b9ecabc2679fe2419957220c0272a965c5cf7e0269fa1aeeb8c56f2e1"
},
{
"blobSum": "sha256:b51569e7c50720acf6860327847fe342a1afbe148d24c529fb81df105e3eed01"
},
{
"blobSum": "sha256:58690f9b18fca6469a14da4e212c96849469f9b1be6661d2342a4bf01774aa50"
}
],
"history": [
{
"v1Compatibility": "{\"architecture\":\"amd64\",\"config\":{\"Hostname\":\"\",\"Domainname\":\"\",\"User\":\"\",\"AttachStdin\":false,\"AttachStdout\":false,\"AttachStderr\":false,\"Tty\":false,\"OpenStdin\":false,\"StdinOnce\":false,\"Env\":[\"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\"],\"Cmd\":[\"/bin/bash\"],\"Image\":\"sha256:3bdb1b7ec7d0b0427c63c46bfc1b2e7e953771af26648b34d0bec9e558709a64\",\"Volumes\":null,\"WorkingDir\":\"\",\"Entrypoint\":null,\"OnBuild\":null,\"Labels\":null},\"container\":\"02b9813c58908ec4e449545066e8dfcff693ced2765493be69f64749f8b5ec70\",\"container_config\":{\"Hostname\":\"02b9813c5890\",\"Domainname\":\"\",\"User\":\"\",\"AttachStdin\":false,\"AttachStdout\":false,\"AttachStderr\":false,\"Tty\":false,\"OpenStdin\":false,\"StdinOnce\":false,\"Env\":[\"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\"],\"Cmd\":[\"/bin/sh\",\"-c\",\"#(nop) \",\"CMD [\\\"/bin/bash\\\"]\"],\"Image\":\"sha256:3bdb1b7ec7d0b0427c63c46bfc1b2e7e953771af26648b34d0bec9e558709a64\",\"Volumes\":null,\"WorkingDir\":\"\",\"Entrypoint\":null,\"OnBuild\":null,\"Labels\":{}},\"created\":\"2021-08-31T01:21:30.672229355Z\",\"docker_version\":\"20.10.7\",\"id\":\"e04aba1046556bb9cac48d566c49cf5d8ca3e7945c14f46b6485fa9e8c6cff25\",\"os\":\"linux\",\"parent\":\"e53a6501ea332ecf595ba4171f4d8ce414a40d20d6b039811602fa27f5c8df5a\",\"throwaway\":true}"
},
{
"v1Compatibility": "{\"id\":\"e53a6501ea332ecf595ba4171f4d8ce414a40d20d6b039811602fa27f5c8df5a\",\"parent\":\"e9ff3de792b711a0f75c7597b9d7f6ead0e0f9be1e8e5a6a20b58cc084c1a109\",\"created\":\"2021-08-31T01:21:30.454442334Z\",\"container_config\":{\"Cmd\":[\"/bin/sh -c mkdir -p /run/systemd \\u0026\\u0026 echo 'docker' \\u003e /run/systemd/container\"]}}"
},
{
"v1Compatibility": "{\"id\":\"e9ff3de792b711a0f75c7597b9d7f6ead0e0f9be1e8e5a6a20b58cc084c1a109\",\"parent\":\"83ebd2c4bab01f227e1ff9a612bad81a4b7ab69ceeda55f397b96a59fcd0dd51\",\"created\":\"2021-08-31T01:21:29.65334794Z\",\"container_config\":{\"Cmd\":[\"/bin/sh -c rm -rf /var/lib/apt/lists/*\"]}}"
},
{
"v1Compatibility": "{\"id\":\"83ebd2c4bab01f227e1ff9a612bad81a4b7ab69ceeda55f397b96a59fcd0dd51\",\"parent\":\"3eaf18f63ccd49304cdca1cf0c473e49d6d10559008c1a0b9cafda4052734296\",\"created\":\"2021-08-31T01:21:28.751673238Z\",\"container_config\":{\"Cmd\":[\"/bin/sh -c set -xe \\t\\t\\u0026\\u0026 echo '#!/bin/sh' \\u003e /usr/sbin/policy-rc.d \\t\\u0026\\u0026 echo 'exit 101' \\u003e\\u003e /usr/sbin/policy-rc.d \\t\\u0026\\u0026 chmod +x /usr/sbin/policy-rc.d \\t\\t\\u0026\\u0026 dpkg-divert --local --rename --add /sbin/initctl \\t\\u0026\\u0026 cp -a /usr/sbin/policy-rc.d /sbin/initctl \\t\\u0026\\u0026 sed -i 's/^exit.*/exit 0/' /sbin/initctl \\t\\t\\u0026\\u0026 echo 'force-unsafe-io' \\u003e /etc/dpkg/dpkg.cfg.d/docker-apt-speedup \\t\\t\\u0026\\u0026 echo 'DPkg::Post-Invoke { \\\"rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true\\\"; };' \\u003e /etc/apt/apt.conf.d/docker-clean \\t\\u0026\\u0026 echo 'APT::Update::Post-Invoke { \\\"rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true\\\"; };' \\u003e\\u003e /etc/apt/apt.conf.d/docker-clean \\t\\u0026\\u0026 echo 'Dir::Cache::pkgcache \\\"\\\"; Dir::Cache::srcpkgcache \\\"\\\";' \\u003e\\u003e /etc/apt/apt.conf.d/docker-clean \\t\\t\\u0026\\u0026 echo 'Acquire::Languages \\\"none\\\";' \\u003e /etc/apt/apt.conf.d/docker-no-languages \\t\\t\\u0026\\u0026 echo 'Acquire::GzipIndexes \\\"true\\\"; Acquire::CompressionTypes::Order:: \\\"gz\\\";' \\u003e /etc/apt/apt.conf.d/docker-gzip-indexes \\t\\t\\u0026\\u0026 echo 'Apt::AutoRemove::SuggestsImportant \\\"false\\\";' \\u003e /etc/apt/apt.conf.d/docker-autoremove-suggests\"]}}"
},
{
"v1Compatibility": "{\"id\":\"3eaf18f63ccd49304cdca1cf0c473e49d6d10559008c1a0b9cafda4052734296\",\"created\":\"2021-08-31T01:21:27.577690932Z\",\"container_config\":{\"Cmd\":[\"/bin/sh -c #(nop) ADD file:11b425d4c08e81a3e0cb2e0345d27cd5fc844dd83f1096af4cc05f635824ff5d in / \"]}}"
}
],
"signatures": [
{
"header": {
"jwk": {
"crv": "P-256",
"kid": "6Y46:BLVK:IRYO:AU7V:3VPB:IQQQ:HCCC:EGVL:5OOB:CFGB:PL7R:NYVO",
"kty": "EC",
"x": "kGc7GZFr7ZO73eyoknvb4veHx5aNa-wVe5tfeVInim0",
"y": "Kzn_mwG6wQiYBH55mogjXDLEe50BDywvltLb-9Ou3Dc"
},
"alg": "ES256"
},
"signature": "sRZ4xaJGpodf1zQMvT8oFfYTktRUXtzI63TBiLNbG3kY9xpiGjWMni6PP_7AZWPOHj-vqRns7INS4MTHV_x2gg",
"protected": "eyJmb3JtYXRMZW5ndGgiOjQ4NzUsImZvcm1hdFRhaWwiOiJDbjAiLCJ0aW1lIjoiMjAyMy0wMi0wMVQxMDoyOTozNVoifQ"
}
]
}old+new ubuntu:16.04headers = {'Accept': 'application/vnd.docker.distribution.manifest.v2+json, application/vnd.oci.image.index.v1+json'} => returns old manifest, as expected response3.headers {
"content-length": "1150",
"content-type": "application/vnd.docker.distribution.manifest.v2+json",
"date": "Wed, 01 Feb 2023 10:35:35 GMT",
"docker-content-digest": "sha256:a3785f78ab8547ae2710c89e627783cfa7ee7824d3468cae6835c9f4eae23ff7",
"docker-distribution-api-version": "registry/2.0",
"docker-ratelimit-source": "78ffd34a-5c31-4cdb-9281-74f2526563db",
"etag": "\"sha256:a3785f78ab8547ae2710c89e627783cfa7ee7824d3468cae6835c9f4eae23ff7\"",
"strict-transport-security": "max-age=31536000"
}print(response3.text) {
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"config": {
"mediaType": "application/vnd.docker.container.image.v1+json",
"size": 3362,
"digest": "sha256:b6f50765242581c887ff1acc2511fa2d885c52d8fb3ac8c4bba131fd86567f2e"
},
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 46497548,
"digest": "sha256:58690f9b18fca6469a14da4e212c96849469f9b1be6661d2342a4bf01774aa50"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 857,
"digest": "sha256:b51569e7c50720acf6860327847fe342a1afbe148d24c529fb81df105e3eed01"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 528,
"digest": "sha256:da8ef40b9ecabc2679fe2419957220c0272a965c5cf7e0269fa1aeeb8c56f2e1"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 170,
"digest": "sha256:fb15d46c38dcd1ea0b1990006c3366ecd10c79d374f341687eb2cb23a2c8672e"
}
]
}new+old ubuntu:16.04headers = {'Accept': 'application/vnd.oci.image.index.v1+json, application/vnd.docker.distribution.manifest.v2+json'} => returns old manifest, as expected because new manifest is unavailable (and new only probably did a strange/implicit fallback, not here, good) response4.headers {
"content-length": "1150",
"content-type": "application/vnd.docker.distribution.manifest.v2+json",
"date": "Wed, 01 Feb 2023 10:37:02 GMT",
"docker-content-digest": "sha256:a3785f78ab8547ae2710c89e627783cfa7ee7824d3468cae6835c9f4eae23ff7",
"docker-distribution-api-version": "registry/2.0",
"docker-ratelimit-source": "78ffd34a-5c31-4cdb-9281-74f2526563db",
"etag": "\"sha256:a3785f78ab8547ae2710c89e627783cfa7ee7824d3468cae6835c9f4eae23ff7\"",
"strict-transport-security": "max-age=31536000"
}print(response4.text) {
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"config": {
"mediaType": "application/vnd.docker.container.image.v1+json",
"size": 3362,
"digest": "sha256:b6f50765242581c887ff1acc2511fa2d885c52d8fb3ac8c4bba131fd86567f2e"
},
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 46497548,
"digest": "sha256:58690f9b18fca6469a14da4e212c96849469f9b1be6661d2342a4bf01774aa50"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 857,
"digest": "sha256:b51569e7c50720acf6860327847fe342a1afbe148d24c529fb81df105e3eed01"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 528,
"digest": "sha256:da8ef40b9ecabc2679fe2419957220c0272a965c5cf7e0269fa1aeeb8c56f2e1"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 170,
"digest": "sha256:fb15d46c38dcd1ea0b1990006c3366ecd10c79d374f341687eb2cb23a2c8672e"
}
]
}old ubuntu:20.04headers = {'Accept': 'application/vnd.docker.distribution.manifest.v2+json'} => returns error, as expected: old schema unavailable on this tag now response.headers {
"content-length": "117",
"content-type": "application/json",
"date": "Wed, 01 Feb 2023 10:44:22 GMT",
"docker-distribution-api-version": "registry/2.0",
"docker-ratelimit-source": "78ffd34a-5c31-4cdb-9281-74f2526563db",
"strict-transport-security": "max-age=31536000"
}print(response.text) {
"errors": [
{
"code": "MANIFEST_UNKNOWN",
"message": "OCI index found, but accept header does not support OCI indexes"
}
]
}old+new ubuntu:20.04headers = {'Accept': 'application/vnd.docker.distribution.manifest.v2+json, application/vnd.oci.image.index.v1+json'} => returns new, as expected (old is unavailable on this image) response6.headers {
"content-length": "1133",
"content-type": "application/vnd.oci.image.index.v1+json",
"date": "Wed, 01 Feb 2023 10:42:37 GMT",
"docker-content-digest": "sha256:b33325a00c7c27b23ae48cf17d2c654e2c30812b35e7846c006389318f6a71c2",
"docker-distribution-api-version": "registry/2.0",
"docker-ratelimit-source": "78ffd34a-5c31-4cdb-9281-74f2526563db",
"etag": "\"sha256:b33325a00c7c27b23ae48cf17d2c654e2c30812b35e7846c006389318f6a71c2\"",
"strict-transport-security": "max-age=31536000"
}print(response6.text) {
"manifests": [
{
"digest": "sha256:d5d4814ffb155d588f10ec8926d9e1cd09b6d41a3110d2f42959e2fc37f6d0b4",
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"platform": {
"architecture": "amd64",
"os": "linux"
},
"size": 436
},
{
"digest": "sha256:d5d767be339687f755e936fffaa860a965316ebb23035242994ff7e3a8e26472",
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"platform": {
"architecture": "arm",
"os": "linux",
"variant": "v7"
},
"size": 436
},
{
"digest": "sha256:a840dd8130bfe149a60f0ad6ca972b7259f9c55e323b16b659f58af2eb6dabfa",
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"platform": {
"architecture": "arm64",
"os": "linux",
"variant": "v8"
},
"size": 436
},
{
"digest": "sha256:cebffa8cceec4bea7ef66193cc86b9ad216bd45292ccd79e2ea9a6830a568e4c",
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"platform": {
"architecture": "ppc64le",
"os": "linux"
},
"size": 436
},
{
"digest": "sha256:602b58d620499c50ae12f6fc2d7f6caa0fe5e00875dd7b2d291c595199c6fb43",
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"platform": {
"architecture": "s390x",
"os": "linux"
},
"size": 436
}
],
"mediaType": "application/vnd.oci.image.index.v1+json",
"schemaVersion": 2
}new+old ubuntu:20.04headers = {'Accept': 'application/vnd.oci.image.index.v1+json, application/vnd.docker.distribution.manifest.v2+json'} => returns new, as expected (old is unavailable on this image) response5.headers {
"content-length": "1133",
"content-type": "application/vnd.oci.image.index.v1+json",
"date": "Wed, 01 Feb 2023 10:40:07 GMT",
"docker-content-digest": "sha256:b33325a00c7c27b23ae48cf17d2c654e2c30812b35e7846c006389318f6a71c2",
"docker-distribution-api-version": "registry/2.0",
"docker-ratelimit-source": "157.143.146.15",
"etag": "\"sha256:b33325a00c7c27b23ae48cf17d2c654e2c30812b35e7846c006389318f6a71c2\"",
"strict-transport-security": "max-age=31536000"
}print(response5.text) {
"manifests": [
{
"digest": "sha256:d5d4814ffb155d588f10ec8926d9e1cd09b6d41a3110d2f42959e2fc37f6d0b4",
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"platform": {
"architecture": "amd64",
"os": "linux"
},
"size": 436
},
{
"digest": "sha256:d5d767be339687f755e936fffaa860a965316ebb23035242994ff7e3a8e26472",
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"platform": {
"architecture": "arm",
"os": "linux",
"variant": "v7"
},
"size": 436
},
{
"digest": "sha256:a840dd8130bfe149a60f0ad6ca972b7259f9c55e323b16b659f58af2eb6dabfa",
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"platform": {
"architecture": "arm64",
"os": "linux",
"variant": "v8"
},
"size": 436
},
{
"digest": "sha256:cebffa8cceec4bea7ef66193cc86b9ad216bd45292ccd79e2ea9a6830a568e4c",
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"platform": {
"architecture": "ppc64le",
"os": "linux"
},
"size": 436
},
{
"digest": "sha256:602b58d620499c50ae12f6fc2d7f6caa0fe5e00875dd7b2d291c595199c6fb43",
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"platform": {
"architecture": "s390x",
"os": "linux"
},
"size": 436
}
],
"mediaType": "application/vnd.oci.image.index.v1+json",
"schemaVersion": 2
} |
S4ckBr
approved these changes
Feb 1, 2023
thomas-riccardi
deleted the
dev-fix-get-image-digest-for-new-oci-image-index
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Docker-Content-Digestheader is still a valid digest with the new content type=> ask both content types, priority to new one.
Added a quick unit test testing get_image_digest on both old and new content-types (hopefully it should stay stable on docker hub side; if it changes, we should adapt it).
Manually tested test failures when removing either of the accepted_content_types.