Added support for cosign and well-known types.

closes pulp#1165
closes pulp#1166
closes pulp#1167
closes pulp#1232
closes pulp#1233
closes pulp#464

CHANGES/1165.feature

@@ -0,0 +1 @@
+Added support to serve cosign signatures, SBOMs, and attestations.

CHANGES/1166.feature

@@ -0,0 +1 @@
+Added support to mirror cosign signatures, SBOMs and attestations.

CHANGES/1167.feature

@@ -0,0 +1 @@
+Added suport to push cosign signatures, attestations or SBOMs to Pulp Registry.

CHANGES/1232.feature

@@ -0,0 +1 @@
+Enabled Pulp registry to support by default some well-known OCI types.

CHANGES/1233.feature

@@ -0,0 +1,2 @@
+Added ``ADDITIONAL_OCI_ARTIFACT_TYPES`` setting to make the list of supported OCI artifact types
+configurable.

CHANGES/464.feature

@@ -0,0 +1 @@
+Added OCI artifact support for Helm charts.

docs/workflows/cosign-support.rst

@@ -0,0 +1,102 @@
+.. _cosign-support:
+
+Mirror cosign signatures
+========================
+
+Being an OCI compliant registry, Pulp Container registry can natively mirror cosign signatures
+that are stored as an OCI image::
+
+ {
+    "schemaVersion": 2,
+    "mediaType": "application/vnd.oci.image.manifest.v1+json",
+    "config": {
+        "mediaType": "application/vnd.oci.image.config.v1+json",
+        "digest": "sha256:f35028aa1563f37ccbaa0b32c57777ffbd8e9e3d81d739fec0022995e58a375a",
+        "size": 153
+    },
+    "layers": [
+        {
+            "mediaType": "application/vnd.dev.cosign.simplesigning.v1+json",
+            "digest": "sha256:d3370bd32b32aba43de2b45bb4a2de2fb5c95fd2edbe738acbc3bc595b80c456",
+            "size": 305,
+            "annotations": {
+                "dev.cosignproject.cosign/signature": "MEUCIBWDnTKhbf5x3mSuEHWkv3ixloIFXeDpfXipF9szqrd5AiEA+UU5J84gQ9JnmT6QZAXiPXqSoDVW0CXQYssGh63e9Ro="
+            }
+        }
+    ]
+ }
+
+
+During the sync task, Pulp will automatically mirror cosign signatures or atomic
+signatures (accessible via the signatures extensions API).
+
+
+Sign and push cosign signatures
+===============================
+
+Pulp Container registry can host cosign signatures which can be pushed via cosign or podman clients:
+
+Cosign::
+
+
+  # This command creates an ECDSA-P256 key pair (a private and a public key). 
+  cosign generate-key-pair
+  cosign sign --key cosign.key pulp-registry/ipanova/cosign-test:latest
+
+or via Podman::
+
+  podman push pulp-registry/ipanova/cosign-test:latest --sign-by-sigstore-private-key cosign.key
+
+.. warning::
+   To use this with images hosted on image registries, the relevant registry or repository must have
+   the use-sigstore-attachments option enabled in containers-registries.d(5). This specifies whether
+   sigstore image attachments (signatures, attestations and the like) are going to be read/written
+   along with the image. If disabled, the images are treated as if no attachments exist; attempts to
+   write attachments fail.
+
+As a result of this operation, the ``ipanova/cosign-test:latest`` image is signed and its
+cosign signature is stored in the registry as an OCI image. Cosign uses a fixed naming convention
+to decide the name for a separate image, at which we can store the signature. The tag name resolved
+to a fixed digest of the image/or manifest list which is being signed in the form of ``sha256-12345.sig``
+
+The payload of the signature will be stored as an image layer::
+
+  {
+    "critical": {
+        "identity": {
+            "docker-reference": "pulp-registry/ipanova/cosigned:latest"
+        },
+        "image": {
+            "docker-manifest-digest": "sha256:81cd171c4eda75046c31d6ed26f1241bbfa9326640613430be780ea931b02c24"
+        },
+        "type": "cosign container image signature"
+    },
+    "optional": {
+        "creator": "containers/image 5.23.1",
+        "timestamp": 1673006074
+    }
+  }
+
+
+.. note::
+    Besides cosign signatures, Pulp Container Registry can mirror and host SBOMs and attestations.
+
+
+Cosign signature verification
+=============================
+
+Signature verification can be done via cosign or podman clients::
+
+  cosign verify --key cosign.pub pulp-registry/ipanova/cosign-test:latest
+
+When using podman client the policy.json file should be properly configured per specs.
+A new requirement type ``sigstoreSigned`` has been introduced:
+
+https://github.com/containers/image/blob/main/docs/containers-policy.json.5.md#sigstoresigned
+
+.. warning::
+   To use this with images hosted on image registries, the relevant registry or repository must have
+   the use-sigstore-attachments option enabled in containers-registries.d(5). This specifies whether
+   sigstore image attachments (signatures, attestations and the like) are going to be read/written
+   along with the image. If disabled, the images are treated as if no attachments exist; attempts to
+   write attachments fail.

docs/workflows/helm-support.rst

@@ -0,0 +1,83 @@
+.. _helm-support:
+
+Using Helm charts with Pulp Container
+=====================================
+
+
+Push and Host
+-------------
+
+Use the following **example** to download and push an etherpad chart from the Red Hat community repository.
+
+Add a chart repository::
+
+    $ helm repo add redhat-cop https://redhat-cop.github.io/helm-charts
+
+Update the information of available charts locally from the chart repository::
+
+    $ helm repo update
+
+Download a chart from a repository::
+
+    $ helm pull redhat-cop/etherpad --version=0.0.4 --untar
+
+Package the chart into a chart archive::
+
+    $ helm package ./etherpad
+    Successfully packaged chart and saved it to: /home/vagrant/devel/pulp_container/etherpad-0.0.4.tgz
+
+Log in to your Pulp container registry using helm registry login::
+
+    $ helm registry login pulp3-source-fedora36.puffy.example.com
+
+Push the chart to your Pulp Container registry using the helm push command::
+
+    $ helm push etherpad-0.0.4.tgz oci://pulp3-source-fedora36.puffy.example.com
+    Pushed: pulp3-source-fedora36.puffy.example.com/etherpad:0.0.4
+    Digest: sha256:a6667ff2a0e2bd7aa4813db9ac854b5124ff1c458d170b70c2d2375325f2451b
+
+Ensure that the push worked by deleting the local copy, and then pulling the chart from the repository::
+
+    $ rm -rf etherpad-0.0.4.tgz
+
+    $ helm pull oci://pulp3-source-fedora36.puffy.example.com/etherpad --version 0.0.4
+    Pulled: pulp3-source-fedora36.puffy.example.com/etherpad:0.0.4
+    Digest: sha256:4f627399685880daf30cf77b6026dc129034d68c7676c7e07020b70cf7130902
+
+The chart can then be installed using the helm install command::
+
+    $ helm install etherpad-0.0.4.tgz
+
+Alternatively, charts can be installed directly from the registry without needing to download locally.
+Use the helm install command and reference the registry location::
+
+    $ helm install oci://pulp3-source-fedora36.puffy.example.com/helm/etherpad --version=0.0.4
+
+
+
+Mirror
+------
+
+Being an OCI compliant registry, Pulp Container registry can natively mirror helm charts
+that are stored as an OCI image::
+ 
+ {
+  "schemaVersion": 2,
+  "config": {
+    "mediaType": "application/vnd.cncf.helm.config.v1+json",
+    "digest": "sha256:8ec7c0f2f6860037c19b54c3cfbab48d9b4b21b485a93d87b64690fdb68c2111",
+    "size": 117
+  },
+  "layers": [
+    {
+      "mediaType": "application/vnd.cncf.helm.chart.content.v1.tar+gzip",
+      "digest": "sha256:1b251d38cfe948dfc0a5745b7af5ca574ecb61e52aed10b19039db39af6e1617",
+      "size": 2487
+    },
+    {
+      "mediaType": "application/vnd.cncf.helm.chart.provenance.v1.prov",
+      "digest": "sha256:3e207b409db364b595ba862cdc12be96dcdad8e36c59a03b7b3b61c946a5741a",
+      "size": 643
+    }
+  ]
+ }

docs/workflows/index.rst

@@ -60,11 +60,21 @@ Basic Workflows
    push
    import-export
 
-Managing Signatures
--------------------
+Managing Atomic Signatures
+--------------------------
 
 .. toctree::
    :maxdepth: 2
 
    sign-images
    verify-images
+
+OCI artifact support
+--------------------
+
+.. toctree::
+   :maxdepth: 2
+
+   cosign-support
+   helm-support
+   oci-artifacts

docs/workflows/oci-artifacts.rst

@@ -0,0 +1,54 @@
+Managing additional OCI media types
+===================================
+
+.. _default-oci-types:
+
+By default the following list of media types is enabled in the Container Registry::
+
+ * OCI images
+ * Helm
+ * Cosign, SBOMs, attestations
+ * Source containers
+ * Singularity
+ * Conftest policies
+ * WASM
+
+For any other OCI media type that is not supported by default, you can add them to the
+``ADDITIONAL_OCI_ARTIFACT_TYPES`` setting using the following format::
+
+ ADDITIONAL_OCI_ARTIFACT_TYPES = {
+    "<oci config type 1>": [
+        "<oci layer type A>",
+        "<oci layer type B>",
+    ],
+    "<oci config type 2>": [
+        "<oci layer type C>",
+        "<oci layer type D>",
+    ],
+ }
+
+
+For example, you can add support for custom defined mediatype by adding the following to your
+``ADDITIONAL_OCI_ARTIFACT_TYPES`` setting::
+
+ ADDITIONAL_OCI_ARTIFACT_TYPES = {
+    "<oci config type 1>": [
+        "<oci layer type A>",
+        "<oci layer type B>",
+    ],
+    "<oci config type 2>": [
+        "<oci layer type C>",
+        "<oci layer type D>",
+    ],
+    "application/vnd.guardians.groot.config.v1+json": [
+        "text/plain",
+        "application/vnd.guardians.groot.docs.layer.v1+tar",
+    ],
+ }
+
+ .. note::
+
+When adding OCI media types that are not configured by default, it is necessary then to manually add
+the :ref:`Default oci types<default-oci-types>` to the list.
+The OCI image-spec types are supported by default, they are built-in and cannot be disabled, it is
+not necessary to add them manually to the list.

docs/workflows/sign-images.rst

@@ -12,10 +12,10 @@ The example below demonstrates how a manifest signing service can be created usi
    hardware cryptographic device.
 
 2. Create a signing script that accepts a manifest path as the only argument. The script invokes
-   ``skopeo standalone-sign`` command that generates a container signature for the image manifest,
-   using the key specified via the ``PULP_SIGNING_KEY_FINGERPRINT`` environment variable. The script
-   should then print out a JSON structure with the following format. The path of the created
-   signature is a relative path inside the current working directory::
+   ``skopeo standalone-sign`` command that generates an atomic container signature for the image
+   manifest, using the key specified via the ``PULP_SIGNING_KEY_FINGERPRINT`` environment variable.
+   The script should then print out a JSON structure with the following format. The path of the
+   created signature is a relative path inside the current working directory::
 
        {"signature_path": "signature"}
 
@@ -30,7 +30,7 @@ The example below demonstrates how a manifest signing service can be created usi
         IMAGE_REFERENCE="$REFERENCE"
         SIGNATURE_PATH="$SIG_PATH"
 
-        # Create container signature
+        # Create atomic container signature
         skopeo standalone-sign $MANIFEST_PATH $IMAGE_REFERENCE $FINGEPRINT --output $SIGNATURE_PATH
         # Check the exit status
         STATUS=$?
@@ -80,9 +80,12 @@ The example below demonstrates how a manifest signing service can be created usi
 
 Afterwards, users are able to sign selected content by the provided script.
 
+   .. warning::
+       The underlying singing facility will sign anything that represents an OCI container image.
+
 
 Sign Images Pushed to the Registry
-==================================
+----------------------------------
 
 Given that an image is pushed to the Pulp Registry via ``podman/docker push`` or via the standard
 DockerRegistry v2 push API, a repository is created containing it::
@@ -140,7 +143,7 @@ operation or it can be explictly specified in the following manner::
              "worker": "/pulp/api/v3/workers/eb65c2d9-31b2-47dc-847e-dad0e744c539/"
          }
 
-Upon task complection, a signature is created and added to the repository::
+Upon task completion, an atomic container signature is created and added to the repository::
 
       $ http GET https://pulp.example.com/pulp/api/v3/repositories/container/container-push/3b279a32-b313-44bf-ad41-4359a92cae24/versions/10/
         {
@@ -202,7 +205,7 @@ Upon task complection, a signature is created and added to the repository::
 
 
 Sign Images Mirrored into the Registry
-======================================
+--------------------------------------
 
 It is possible to sign content that was synchronized from remote registries.
 If the content was synced together with signatures, upon signing task completion new signatures will be
@@ -313,8 +316,8 @@ This API exposes an endpoint for reading and writing image signatures. Users sho
 sigstore section in the `registries.d file <https://github.com/containers/image/blob/main/docs/containers-registries.d.5.md>`_
 accordingly to benefit from the API.
 
-Reading Image Signatures
-------------------------
+Reading Signatures
+------------------
 
 To read existing signatures, issue the following GET request::
 
@@ -323,8 +326,8 @@ To read existing signatures, issue the following GET request::
 Signatures are retrieved by container clients automatically if the policy requires so. The policy is
 defined in the file ``/etc/containers/policy.json``.
 
-Writing Image Signatures
-------------------------
+Writing Signatures
+------------------
 
 To add a new signature to an image, execute the following PUT request::
 

docs/workflows/sync.rst

@@ -72,9 +72,9 @@ the ``sigstore`` field when creating a Remote.
 
 .. note::
     Some registries provide docker API extensions for ``atomic container signature`` type only, or
-    have ``cosign`` type signatures that are stored as a separate OCI artifact in a registry.
-    Pulp will automatically sync signatures provided via the docker API extension. At the moment,
-    `cosign` signatures are not supported.
+    have ``cosign`` type signatures that are stored as a separate OCI image in a registry.
+    Pulp will automatically sync signatures provided via the docker API extension or cosign
+    signatures stored as an OCI image.
 
 
 Reference: `Container Remote Usage <../restapi.html#tag/Remotes:-Container>`_

pulp_container/app/__init__.py

@@ -8,3 +8,15 @@ class PulpContainerPluginAppConfig(PulpPluginAppConfig):
     label = "container"
     version = "2.16.0.dev"
     python_package_name = "pulp-container"
+
+    def ready(self):
+        super().ready()
+        self.register_registry_types()
+
+    def register_registry_types(self):
+        # circular import avoidance
+        from pulp_container import constants
+        from django.conf import settings
+
+        for media_type, layer_types in settings.ADDITIONAL_OCI_ARTIFACT_TYPES.items():
+            constants.register_well_known_types(media_type, layer_types)