Support for Sigstore signature generation and verification

[mayaCostantini]

Is your feature request related to a problem? Please describe.
Implement Sigstore signature generation and verification for Ansible collections in addition to the existing GPG signature workflow.

Describe the solution you'd like
Sigstore signing and verification for Ansible collections will provide the following advantages:

• Key management: Sigstore solves the issue of private key storage and revocation by providing “keyless” signatures for software artifacts, eliminating the need for the user to store private GPG keys which could eventually become compromised.

• Identity-based signatures: Sigstore generates free, short-lived signing certificates based on identities, delegating the authentication part to an OIDC provider (Google, GitHub, Microsoft…).

• An immutable Transparency Log (Rekor) for signature and provenance attestations that can be deployed as a private instance.

The signature and verification workflow would exist in parallel with the current implementation utilizing GPG key signing via Signing Service objects.
Sigstore provides “keyless” signing for software artifacts, where a signer can connect to an OIDC provider and Sigstore will produce an ephemeral signing certificate and store a signature entry in an immutable transparency log queried during signature verification.

The signing and verification workflow for collections uploaded to pulp_ansible could be as follow:

• Uploading signed collections
  A developer creates a collection, generates a MANIFEST.json file containing a list of collection files along with their sha256 checksums, and signs this file using the pulp ansible, pulp or ansible-sign command lines supporting sigstore signatures. The developer can sign the collection using a key pair or via an OIDC identity ("keyless" signing). It is possible to specify the Fulcio (certificate authority) and Rekor (transparency log) instances to use by passing the instance URLs as command line parameters. The two files necessary for signature verification are included in the resulting compressed artifact for upload.

• Signing collections
  Applies the same signing workflow than above, where content units are signed using the pulp ansible repository sign command line. The signature and certificates are directly stored on the repository side.

• Verifying signatures
  Verifying signatures requires two steps: 1) validating the integrity of the collection by verifyingMANIFEST.json file checksums, 2) checking the signature authenticity using the signature and certificate files. As for the current workflow, verification could be done upon install via the ansible-galaxy collection install command or for already installed collections with ansible-galaxy collection verify.

The signature and verification workflow would exist in parallel with the current implementation utilizing GPG key signing via Signing Services.
A private instance of Sigstore will be deployed alongside customers Ansible infrastructure to support their workflow. This infrastructure consists of the following components:

• Rekor, an immutable, append-only transparency log used to create entries for the signatures produced
• Fulcio, a Certificate Authority delivering ephemeral certificates attesting to the signer’s identity
• An OIDC server (for example, the Dex OIDC session manager or Keycloak) to get an OIDC token from an identity provider and authenticate to Fulcio.

Verification materials needed to attest of a signature’s validity include the artifact signed, the signature file, the ephemeral certificate and the corresponding transparency log entry that can be queried from the deployed Rekor server.

The implementation relies on the sigstore-python library, which should reach its first stable release in the weeks or months to come. The library implements keyless signatures via OIDC only, and maintainers do not currently plan on supporting key pair signing. This functionality can however be implemented outside of the library to allow users to reuse GPG keys stored on the Pulp server to sign artifacts.

Here are some of the aspects of this implementation I would like to discuss about, please feel free to comment / add anything else you would find worth discussing:

OIDC credentials storage

Sigstore supports ambient credentials detection, which is the solution usually used for automatic signing of content (which here would be for example to sign automatically a set of collections or a namespace). Users can specify OIDC tokens from an identity provider (Google, Microsoft, GitHub) as environment variables that are detected at the time of signing.
How could users store (and renew later if needed) OIDC credentials on their Pulp server?
OIDC identities / public keys database

Verifying signatures consists in attesting of its integrity with regards to the content signed, but also of its authenticity. For this, it is necessary for administrators to manage identities that have permissions to sign artifacts for a given registry, and thus to have access to a database of OIDC identities and public key fingerprints authorized to sign content.

Changes to the current data model

The current data model for pulp_ansible includes CollectionVersionSignature objects, with attributes for the public key fingerprint used for signing the collection and the data signed (and its sha256 digest). This proposal introduces a new class for representing Sigstore signatures CollectionVersionSigstoreSignature with the following attributes:

data (models.TextField): An artifact signature.
digest (models.CharField): A signature sha256 digest.
pubkey_fingerprint (models.CharField): The fingerprint of the public key used.
oidc_identity (models.CharField): The OIDC identity present in the signing certificate.
x509_certificate (models.BinaryField): The PEM-encoded X509 certificate associated with the signature.
x509_certificate_digest (models.CharField): The sha256 digest of the certificate.

This model comprises the signed data and its digest, the fingerprint of the public key used for signing and an eventual OIDC identity present in the signing certificate as the Subject Alternative Name (SAN), the certificate itself and its digest. The public key fingerprint and / or OIDC identity used to sign the content could be used to create a unique constraint on the signed collections.

Changes to the Pulp API
On the API side, adding the Sigstore verification materials (signed manifest file, signature file and certificate) in addition to or in place of the GPG key would allow users to add these materials when creating content and to verify signatures when listing content . I am still unsure about all the endpoint that would be affected by this change, but an example here could be with the /pulp/api/v3/repositories/ansible/ansible/ endpoint.

cc @gerrod3