-
Notifications
You must be signed in to change notification settings - Fork 382
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support OCI artifacts #1574
Support OCI artifacts #1574
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, nice work
// This is publicly visible as c/image/manifest.NonImageArtifactError (but we don’t provide a public constructor) | ||
type NonImageArtifactError struct { | ||
// Callers should not be blindly calling image-specific operations and only checking MIME types | ||
// on failure; if they care about the artifact type, they should check before using it. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that it's non-trivial to figure out whether an image is an artifact or not. An artifact may have a custom config and/or custom layers.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we can help callers in figuring that out.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
https://github.com/opencontainers/artifacts/blob/main/artifact-authors.md#visualizing-artifacts explicitly says that the config.mediaType
is how they are supposed to be differentiated.
But then both the primary motivations for this work are not compliant: #1408 wants to use images with custom layers, and the https://github.com/sigstore/cosign/blob/main/specs/SIGNATURE_SPEC.md explicitly defines the signature to use an image MIME type.
In the cases where this PR adds this failure mode, we really care about the thing being an image (e.g. we inspect the config or even convert the config’s format). In cases of abuse of that MIME type, it seems … technically correct… to complain about an “invalid image” more than about an “unexpected artifact”.
But you’re completely right, in practice it might be much more practical to detect the major violators and provide good error messages for them. That’s actually a pretty good reason not to expose the mimeType
value of the error (which I was quite on the fence about) until we can provide something useful to callers.
... primarily so that it accepts OCI-formatted images. Proof of concept, as well as other updates necessary to work with that registry version, are in containers/image#1574 . Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... primarily so that it accepts OCI-formatted images. Proof of concept, as well as other updates necessary to work with that registry version, are in containers/image#1574 . See See containers/skopeo#1673 for more context. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... primarily so that it accepts OCI-formatted images. Proof of concept, as well as other updates necessary to work with that registry version, are in containers/image#1574 . See containers/skopeo#1673 for more context. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... primarily so that it accepts OCI-formatted images. Proof of concept, as well as other updates necessary to work with that registry version, are in containers/image#1574 . See containers/skopeo#1673 for more context. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
b9cc5a9
to
a8b2234
Compare
... from containers/image#1574 . > go mod edit -replace github.com/containers/image/v5=github.com/mtrmac/image/v5@oci-artifacts > make vendor Signed-off-by: Miloslav Trmač <mitr@redhat.com>
|
4bead21
to
ed0ab8c
Compare
... from containers/image#1574 . > go mod edit -replace github.com/containers/image/v5=github.com/mtrmac/image/v5@oci-artifacts > make vendor Signed-off-by: Miloslav Trmač <mitr@redhat.com>
OK, I’m done with this. One of the intermediate versions (motivating #1579 ) added a So, this PR does not handle that any more — and now we need only a single new This also updates the CI image to a newer registry that can accept OCI artifacts, although we don’t currently test with any. Then, containers/skopeo#1680 re-enables tests of syncing the repo with a Cosign signature; see that for at least a smoke test of this PR. |
Oh, actually, that requires containers/skopeo#1679 . Dropping that commit again. So I think the merging order should be:
|
One of the is fine, but better placed closer to the reference. The other doesn't relate to that code path at all, and is already documented in types.BlobInfo. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Having each test case streched across four lines doesn't make it any more readable; seeing the test cases on the same screen as the test body is more valuable. Should not change (test) behavior. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Add manifestSchema2FromFixture and manifestOCI1FromFixture instead of open-coding the read+decode steps over and over. The tests also fail immediately if they can't use the fixture, instead of somehow trying to continue (and probably crashing). Signed-off-by: Miloslav Trmač <mitr@redhat.com>
…ly caller ... to make the code a tiny bit easier to follow. Only moves unchanged code, should not change behavior. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... and add tests for the ImageID implementations. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This will allow us to refer to ic.c.cannotModifyManifestReason and image-specific properties; it just replaces the objects without changing the logic. Should not change behavior. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... to eliminate at least one of the many parameters. Should not change behavior. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Should not change behavior. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
We will use it for debug logs. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
We will add another user of the lookup code. Erorr messages now use mimeType instead of mt, which were required required to be equal on that code path, now that mt is not visible. Should not change behavior. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
- Don't compress/decompress layers with unknown MIME types, and layers in OCI artifacts. - Don't even change manifest MIME types in these situations, whatever happens. - Don't substitute compressed/uncompressed variants (via TryReusingBlobWithOptions) for OCI artifacts, if we discover the same variants when copying images that refer to the same blobs. Note that this does _not_ restrict compression to algorithms supported by the SourcedImage, because that would prohibit a single-pass conversion from v2s2 to OCI while compressing to zstd [1], and that's a feature we currently exercise in tests. So, this prevents us from failing to copy OCI artifacts, but users of zstd still need to be careful about choosing OCI manually. [1] We would need to ask the _destination_ format handler about zstd, not the source-format SourcedImage, and we don't currently have that infrastructure. It's also not immediately clear how to combine this with the sequence of alternative manifest formats returned by determineManifestConversion. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Here, c/image needs to switch to the updated CI image here as well, or c/image CI is going to break: #1581 .
|
including containers/image#1574 . > go get github.com/containers/image/v5@main > make vendor Signed-off-by: Miloslav Trmač <mitr@redhat.com>
including containers/image#1574 . > go get github.com/containers/image/v5@main > make vendor Signed-off-by: Miloslav Trmač <mitr@redhat.com>
including containers/image#1574 . > go get github.com/containers/image/v5@main > make vendor Signed-off-by: Miloslav Trmač <mitr@redhat.com>
including containers/image#1574 . > go get github.com/containers/image/v5@main > make vendor Signed-off-by: Miloslav Trmač <mitr@redhat.com>
including containers/image#1574 . > go get github.com/containers/image/v5@main > make vendor Signed-off-by: Miloslav Trmač <mitr@redhat.com>
including containers/image#1574 . > go get github.com/containers/image/v5@main > make vendor Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This broadly follows the plan from #1408 (comment) :
See the individual commits for details.
FIXME: At this point I’m rather unhappy about the
CanChangeLayerCompression
API. It’s good enough for internal use for now, exposing it inc/image/manifest
is not great. OTOH I’d rather not spend time investigating the more general which-image-formats-can-support-which-compression-and-what-to-do-if-they-don’t problem space.At this point absolutely untested apart from unit tests (which don’t cover most of the
copy
code); this will be the first run against Skopeo’s integration test and a real artifact.See individual commit messages for details. Quite a lot of the refactoring could be split into a separate PR… if an existing OCI artifact didn’t break CI right now. Something to consider later.