Handle multiple matching index entries

[sudo-bmitch]

The goal is to handle an index where there are multiple entries that look the same to older clients, but a newer client may see a feature it supports in the annotations. This would allow an SBoM to be packaged as an OCI-artifact and pushed in an index along side the image. It would also give a way to support different layer encodings for the same image without breaking older clients.

I say "SHOULD", but if there are no use cases where this change would break, then I'd be tempted to say "MUST".

Fixes #862, fixes #776

Signed-off-by: Brandon Mitchell git@bmitch.net

[vrothberg]

LGTM

I say "SHOULD", but if there are no use cases where this change would break, then I'd be tempted to say "MUST".

Wouldn't "MUST" break use cases where clients would discriminate multiple matching entries via annotations? Annotations are sometimes used, for instance in HPC, to add information on compiler flags. A client may look for these annotations and select the ones optimized for the local platform. Using MUST would break that use case since annotations are not part of the platform.

[sudo-bmitch]

Wouldn't "MUST" break use cases where clients would discriminate multiple matching entries via annotations? Annotations are sometimes used, for instance in HPC, to add information on compiler flags. A client may look for these annotations and select the ones optimized for the local platform. Using MUST would break that use case since annotations are not part of the platform.

This is certainly a use case I'm thinking this would support. I didn't call out what the "client or runtime's requirements" were in that line, so my assumption is these would also include annotations if a client was created that looked for annotations (or other fields in the descriptor).

Perhaps better phrasing may be "if multiple manifest entries are effectively equivalent to the client" and newer clients that handle features identified in annotations wouldn't see the entries as equivalent. Happy to find the best phrasing that gets across this idea.

[vrothberg]

The current phrasing looks good to me since clients can still decide to not pick the first matching entry if they have good reasons for it.

[SteveLasker]

I agree with the wording in the PR. But, I struggle with the example of an SBoM in an index. This is the forward and reverse indexing topic that's part of the Reference Type working groups.

[sudo-bmitch]

I agree with the wording in the PR. But, I struggle with the example of an SBoM in an index. This is the forward and reverse indexing topic that's part of the Reference Type working groups.

The Reference Type proposals would be a welcome addition and would likely handle more scenarios than injecting the artifact along side the image in an index's manifest list. However it will be some time before that is approved and available in all the registries I work with, so I'm looking at intermediate solutions that allow me to package artifacts with an image today. The other option would be something like the cosign model of a tag that's a digest and short type, but those tags clutter the tag listing in a less than ideal way.

[stevvooe]

LGTM

[sajayantony]

Should we handle this in down conversion in distribution conformance? @jdolitsky

[jdolitsky]

@sajayantony yes, but prior to that we need to be testing indexes in general (larger issue)

[vbatts]

Suggested change

	If multiple manifests match a client or runtime's requirements, the first matching entry SHOULD be used.
	If multiple manifests match a client or runtime's `platform` requirements, the first matching entry SHOULD be used.

[sudo-bmitch]

I was intentionally leaving that out since some clients may have different requirements, like checking annotations to see if a different layer compression format is available.

[cyphar]

LGTM.