Specify and implement on-disk "verification bundle" format

[dlorenc]

Right now we stash a bunch of information for verifying signatures (fixed-key and keyless) in OCI images. This includes fields like:

• the signature itself
• the signed payload
• the certificate (and chain)
• the transparency log index/uuid/inclusion proof
• a signed timestamp

Full info is here: https://github.com/sigstore/cosign/blob/main/specs/SIGNATURE_SPEC.md

This works great for OCI stuff (cosign sign) but doesn't really work for blobs. Right now users that sign blobs have to keep track of all of these things on their own.

We should try to standardize a single "bundle" for all of this stuff on disk. It could be as simple as just the JSON info we stick into OCI, but there might be better options. JSON is a necessary prereq for working with OCI so relying on it there isn't a big deal, but for blobs maybe we could avoid a JSON dependency with something simpler.

The overall flow could be something like:

cosign sign-blob <blob> > bundle.sig

cosign verify-blob <blob> --bundle bundle.sig

Where this would contain enough information for a keyless verification and offline transparency log lookup.

cc @luhring

[priyawadhwa]

This could also be useful for Chains for our non-oci storage backends, we could start storing the entire bundle in gcs instead of having a separate file for each payload/signature (having a consistent bundle would make it easier for clients to verify as well)

Slightly related, but @asraa and I were just chatting about maybe adding support for those backends (like gcs) in cosign directly. Just mentioning it here in case anyone has any thoughts around it.

[dlorenc]

This could also be useful for Chains for our non-oci storage backends, we could start storing the entire bundle in gcs instead of having a separate file for each payload/signature (having a consistent bundle would make it easier for clients to verify as well)

+1!

Slightly related, but @asraa and I were just chatting about maybe adding support for those backends (like gcs) in cosign directly. Just mentioning it here in case anyone has any thoughts around it.

What would that look like? cosign sign-blob --store=gcs|docdb|something?

[asraa]

What would that look like? cosign sign-blob --store=gcs|docdb|something?

At least for GCS is it user-friendly to do something like sign-blob gcs://<bucket>/<object> and infer the bucket to place the bundle? It seems more seamless than an alternative like: cosign sign-blob a.txt --store=gcs://<bucket> (since I'd assume that the blob is already on GCS)

[dlorenc]

At least for GCS is it user-friendly to do something like sign-blob gcs://<bucket>/<object> and infer the bucket to place the bundle? It seems more seamless than an alternative like: cosign sign-blob a.txt --store=gcs://<bucket> (since I'd assume that the blob is already on GCS)

Yeah, but it's technically a bit "racy", similar to tag signing in a container image. GCS doesn't enforce any kind of immutability, and you need to fetch something to calculate its digest :(

Signing locally and then uploading sort of sidesteps that race condition. GCS does store hashes separately in the API but IIRC they're just md5.

[priyawadhwa]

At least for the Chains case signing locally and then uploading would totally work, cosign sign-blob --store=gcs|docdb|something is pretty much what I had in mind.

[dlorenc]

Cool! If anyone wants to start on this one, go for it!

[priyawadhwa]

I'll give this one a go :)