Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot Resolve descriptor: 404 not found while oras blob fetch does find it #823

Closed
Skarlso opened this issue Sep 3, 2024 · 11 comments
Closed

Comments

@Skarlso
Copy link

Skarlso commented Sep 3, 2024

Basically what the title says.

I'm trying to resolve a layer like this but getting not found:

oras resolve ghcr.io/open-component-model/mpas-bootstrap-component/fluxcd/notification-controller@sha256:2e3521c3642113452b802980672e7c4846c3f1069c5067c569d2bd64c7552764
Error response from registry: failed to resolve digest: ghcr.io/open-component-model/mpas-bootstrap-component/fluxcd/notification-controller@sha256:2e3521c3642113452b802980672e7c4846c3f1069c5067c569d2bd64c7552764: not found

Yet when I run oras blob fetch, it's clearly there:

oras blob fetch --output - ghcr.io/open-component-model/mpas-bootstrap-component/fluxcd/notification-controller@sha256:2e3521c3642113452b802980672e7c4846c3f1069c5067c569d2bd64c7552764
{"architecture":"amd64","config":{"User":"65534:65534","Env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],"Entrypoint":["notification-controller"],"Labels":{"org.opencontainers.image.created":"2023-08-23T14:47:13.967Z","org.opencontainers.image.description":"The GitOps Toolkit event forwarded and notification dispatcher","org.opencontainers.image.licenses":"Apache-2.0","org.opencontainers.image.revision":"7de11b33d94c017dc2ad47e13f93fcf13667c4d4","org.opencontainers.image.source":"https://github.com/fluxcd/notification-controller","org.opencontainers.image.title":"notification-controller","org.opencontainers.image.url":"https://github.com/fluxcd/notification-controller","org.opencontainers.image.version":"v1.1.0"},"OnBuild":null},"created":"2023-08-23T14:56:17.420674442Z","history":[{"created":"2023-08-07T19:20:20.71894984Z","created_by":"/bin/sh -c #(nop) ADD file:32ff5e7a78b890996ee4681cc0a26185d3e9acdb4eb1e2aaccb2411f922fed6b in / "},{"created":"2023-08-07T19:20:20.894140623Z","created_by":"/bin/sh -c #(nop)  CMD [\"/bin/sh\"]","empty_layer":true},{"created":"2023-08-23T14:47:24.837095878Z","created_by":"ARG TARGETPLATFORM","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2023-08-23T14:47:24.837095878Z","created_by":"RUN |1 TARGETPLATFORM=linux/amd64 /bin/sh -c apk --no-cache add ca-certificates   \u0026\u0026 update-ca-certificates # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2023-08-23T14:56:17.420674442Z","created_by":"COPY /workspace/notification-controller /usr/local/bin/ # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2023-08-23T14:56:17.420674442Z","created_by":"USER 65534:65534","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2023-08-23T14:56:17.420674442Z","created_by":"ENTRYPOINT [\"notification-controller\"]","comment":"buildkit.dockerfile.v0","empty_layer":true}],"os":"linux","rootfs":{"type":"layers","diff_ids":["sha256:4693057ce2364720d39e57e85a5b8e0bd9ac3573716237736d6470ec5b7b7230","sha256:6bfe58a2bb4cf0e274f5a96a872acb132e45d159fcf4e887a2f5309eaf9dd650","sha256:a1668b593cbc6f971e61d4efd84455d77d6eccb02d33fcedf597c790900a47a4"]}}%

What am I doing wrong?

This is the manifest layer of that repo:

oras manifest fetch ghcr.io/open-component-model/mpas-bootstrap-component/fluxcd/notification-controller:v1.3.0
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:49efb61aaec2b19664181995a22894102e206165f666c9280adfa7266a696ea4",
      "size": 865,
      "platform": {
        "architecture": "amd64",
        "os": "linux"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:3114fa44cf8aad5b1fdc44cd17aea7a9c09163d85621e833930593e534f2f926",
      "size": 865,
      "platform": {
        "architecture": "arm",
        "os": "linux",
        "variant": "v7"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:f1b641270009b4225363e72b7f5b1d90e1d4cadfa45f4d382cd68336ba73cab5",
      "size": 865,
      "platform": {
        "architecture": "arm64",
        "os": "linux"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:f1a7bb3e6b44f4581bbf093262e3c566128d6cda5d0af78517a210e8ebf3d1a9",
      "size": 840,
      "annotations": {
        "vnd.docker.reference.digest": "sha256:49efb61aaec2b19664181995a22894102e206165f666c9280adfa7266a696ea4",
        "vnd.docker.reference.type": "attestation-manifest"
      },
      "platform": {
        "architecture": "unknown",
        "os": "unknown"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:8a5780e99e75591c21ff6703621cdda59ded82cd89da478d16fdb19db6f0d5f0",
      "size": 840,
      "annotations": {
        "vnd.docker.reference.digest": "sha256:3114fa44cf8aad5b1fdc44cd17aea7a9c09163d85621e833930593e534f2f926",
        "vnd.docker.reference.type": "attestation-manifest"
      },
      "platform": {
        "architecture": "unknown",
        "os": "unknown"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:d11d59c3b22fbceba2df9fc9d47371e6ea36d856fb344cfe720de121561d967f",
      "size": 840,
      "annotations": {
        "vnd.docker.reference.digest": "sha256:f1b641270009b4225363e72b7f5b1d90e1d4cadfa45f4d382cd68336ba73cab5",
        "vnd.docker.reference.type": "attestation-manifest"
      },
      "platform": {
        "architecture": "unknown",
        "os": "unknown"
      }
    }
  ]
}%
@Skarlso Skarlso changed the title Cannot Resolve manifest: 404 not found while oras blob fetch does find it Cannot Resolve descriptor: 404 not found while oras blob fetch does find it Sep 3, 2024
@Skarlso
Copy link
Author

Skarlso commented Sep 3, 2024

Doing it in code yields the same result:

	repo, err := remote.NewRepository("ghcr.io/open-component-model/mpas-bootstrap-component/fluxcd/notification-controller")
	if err != nil {
		fmt.Println(err)
		os.Exit(1)
	}

	desc, err := repo.Resolve(context.Background(), "sha256:2e3521c3642113452b802980672e7c4846c3f1069c5067c569d2bd64c7552764")
	if err != nil {
		fmt.Println("failed to resolve descriptor ", err)
                os.Exit(1)
	}

	fmt.Println(desc.Digest)

Fails too.

@Skarlso
Copy link
Author

Skarlso commented Sep 4, 2024

That's weird, I know just noticed that the SHA is in fact not under the listed layers. But then what the heck is it fetching when using this sha? :D :D :D What's going on here?

@Skarlso
Copy link
Author

Skarlso commented Sep 4, 2024

okay, I think I'm getting the wrong SHA somehow in my code. :/

This works:

func main() {
	repo, err := remote.NewRepository("ghcr.io/open-component-model/mpas-bootstrap-component/fluxcd/notification-controller")
	if err != nil {
		fmt.Println(err)
		os.Exit(1)
	}

	desc, err := repo.Resolve(context.Background(), "v1.3.0")
	if err != nil {
		fmt.Println("failed to resolve descriptor ", err)
	}

	fmt.Println(desc.Digest)
}

Looks like, this will get the right thing.

@Wwwsylvia
Copy link
Member

Hi @Skarlso, oras resolve internally calls Repository.Resolve, which only resolves manifests.

Resolve resolves a reference to a manifest descriptor. See also ManifestMediaTypes.

To resolve a layer, you would need to explicitly specify the blob path by invoking repo.Blobs() like this:

	repo, err := remote.NewRepository("ghcr.io/open-component-model/mpas-bootstrap-component/fluxcd/notification-controller")
	if err != nil {
		fmt.Println(err)
		os.Exit(1)
	}

-	desc, err := repo.Resolve(context.Background(), "sha256:2e3521c3642113452b802980672e7c4846c3f1069c5067c569d2bd64c7552764")
+     	desc, err := repo.Blobs().Resolve(context.Background(), "sha256:2e3521c3642113452b802980672e7c4846c3f1069c5067c569d2bd64c7552764")

	if err != nil {
		fmt.Println("failed to resolve descriptor ", err)
                os.Exit(1)
	}

	fmt.Println(desc.Digest)

@Skarlso
Copy link
Author

Skarlso commented Sep 4, 2024

OMG THAT WORKED. :D Hum, I need to redesign things 🤔 Check out if I do fetch the right thing. But thanks, this really helped me go further. :) Thank you again :DDDD <3

@Skarlso Skarlso closed this as completed Sep 4, 2024
@Wwwsylvia
Copy link
Member

@Skarlso No problem. Good luck! 😀

@Skarlso
Copy link
Author

Skarlso commented Sep 4, 2024

I might need to read up on how OCI layers are structured better, but... could you help me out one last time, or point me to a doc where it makes a bit more sense so I structure my code better please? :)

The question is, when do I use repos Resolve vs Blob.Resolve? :D

@Skarlso
Copy link
Author

Skarlso commented Sep 4, 2024

Ah, so one lists blobs, and the other lists ManifestMediaTypes. So to list tags and such, I can use the repo Resolver to fetch things, but the actually get the blob data, I need to use Blobs()?

@Wwwsylvia
Copy link
Member

@Skarlso The image-spec and the distribution-spec are what ORAS is following and are the source of truth.
You may want to check manifest.md which describes the components that make up a container image.

The question is, when do I use repos Resolve vs Blob.Resolve? :D

Generally the API doc should help (just read it, LOL).

But the pattern is that:

  • In most cases, if a Repository-level function takes a descriptor, it can automatically determine which path (manifests or blobs) to go based on the media type in the descriptor, and you don't have to explicitly specify the path.
  • It other cases, when the Repository-level function cannot determine the target media type, or a blob does not make sense in the context, it goes for manifests by default.

BTW, welcome to join the #oras Slack channel to ask questions and get community news! 🙌

@Wwwsylvia
Copy link
Member

Ah, so one lists blobs, and the other lists ManifestMediaTypes. So to list tags and such, I can use the repo Resolver to fetch things, but the actually get the blob data, I need to use Blobs()?

Not really, Repository.Fetch() can be used to fetch both manifests and blobs. You can take a look at the examples associated with it. 😀

But it does not harm to explicitly specify the path if you know what you want.

image

We would love to hear feedback on oras-go documentation! Please feel free to raise suggestions/complaints under #778 if you have any. 🙂

@Skarlso
Copy link
Author

Skarlso commented Sep 4, 2024

Ah thank you so much for the information!! That is a lot that can help here! :) Thank you! :) ❤️ ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants