The type of manifests may not be correctly determined when syncing content

[lubosmj]

http https://registry-1.docker.io/v2/lubosmj/test/manifests/latest Accept:application/vnd.docker.distribution.manifest.list.v2+json

HTTP/1.1 200 OK
content-length: 669
content-type: application/vnd.oci.image.index.v1+json
date: Fri, 25 Nov 2022 17:25:57 GMT
docker-content-digest: sha256:9a0a39570a12cf465209db9ab02812b02b87320bfcc1cee58978d1129191d0c8
docker-distribution-api-version: registry/2.0
docker-ratelimit-source: 86.49.248.80
etag: "sha256:9a0a39570a12cf465209db9ab02812b02b87320bfcc1cee58978d1129191d0c8"
ratelimit-limit: 100;w=21600
ratelimit-remaining: 91;w=21600
strict-transport-security: max-age=31536000

{
    "manifests": [
        {
            "digest": "sha256:d8fbbbf3fec1857c32c110292a9decf9744f9f97d7247019ae4776c241395221",
            "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
            "platform": {
                "architecture": "amd64",
                "os": "linux"
            },
            "size": 588
        },
        {
            "digest": "sha256:f8634bb68dccf0dc2a3113933a67f91dc10c4ac17dee90988cb6bc4ae55cf802",
            "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
            "platform": {
                "architecture": "amd64",
                "os": "linux"
            },
            "size": 588
        },
        {
            "digest": "sha256:6489ee892f64e59755435ee53f7d10cce5588a7788b4b2ae4a510a8bbc92704d",
            "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
            "platform": {
                "architecture": "amd64",
                "os": "linux"
            },
            "size": 588
        }
    ],
    "schemaVersion": 2
}

Error message:

pulp container repository sync --name test
Started background task /pulp/api/v3/tasks/cacfda02-b134-4fe5-9d7e-8ad23e9acfbc/
..........................................................................................Error: Task /pulp/api/v3/tasks/cacfda02-b134-4fe5-9d7e-8ad23e9acfbc/ failed: '{'errors': [{'code': ErrorDetail(string='MANIFEST_INVALID', code='parse_error'), 'message': ErrorDetail(string=": 'mediaType' is a required property", code='parse_error'), 'detail': {'digest': ErrorDetail(string='9a0a39570a12cf465209db9ab02812b02b87320bfcc1cee58978d1129191d0c8', code='parse_error')}}]}'

We should make the mediaType field optional like we already did for OCI indexes.

The function for determining media types of manifests is not working properly.

[ipanova]

based on the response from docker hub, this is an OCI index ( looking at content-type header)

[lubosmj]

Thanks for pointing that out. I can confirm that the function for determining the correct media type does not work properly. The problem is actually here:

pulp_container/pulp_container/app/utils.py

Line 153 in 0b979a7

elif media_type in (MANIFEST_MEDIA_TYPES.IMAGE or MANIFEST_MEDIA_TYPES.LIST):

Even though the media_type variable is set to application/vnd.oci.image.index.v1+json, the condition is evaluated as false.

[lubosmj]

After some discussion with @ipanova, we have decided that the problem is more complex and the root cause of the issue is not on our side. The specification does not allow listing application/vnd.docker.distribution.manifest.v2+json manifests inside the OCI index (see the section manifests in https://github.com/opencontainers/image-spec/blob/main/image-index.md). The attached snippet contains the OCI index with docker manifests. Docker Hub possibly allows users to upload manifests which do not follow the specification.

Only the elif condition will be addressed.

[ipanova]

https://bugzilla.redhat.com/show_bug.cgi?id=2102078