Investigate how to retrieve metadata from an OCI registry

[ppbaena]

Summary
Investigate how to retrieve all metadata stored as OCI artifacts in an OCI registry to be listed to users at deployment time, so they can evaluate not only the chart metadata, but any other information stored as OCI artifacts in the registry.

Background and rationale
In addition to the work started for Catalog browsing in OCI repository, once Kubeapps is able to specify an OCI registry and list charts from that registry (without having to explicitly list which apps/repositories users are interested in), next step to improve the user experience is to allow users to retrieve all metadata stored in the OCI registry related to each apps/repositories.

Description
Initial investigation and design (technical and UI) to provide this feature in Kubeapps.

Acceptance criteria
A formalized list of conditions that ensures that the feature can be considered finished.

Additional context
Related to Catalog browsing in OCI repository initiative

[absoludity]

Testing the bitnami charts catalog on Dockerhub and querying the referrers API for a digest results in a 404 from docker.io, which means that the referrers API is not supported. Sure enough, in October 2022, when announcing the artifacts support, Docker said:

While the OCI Artifact Specification is considered a release candidate, full Docker Hub support for OCI Reference Types and the accompanying Referrers API is on the horizon. Stay tuned for upcoming enhancements, improved repo organization, and more.

So, although it's not yet supported, it will be at some time. Testing demo.harbor.io, on the other hand, shows that the referrers API is supported (an empty {} is returned, rather than a 404).

Interestingly, the OCI Distribution Spec actually includes backwards compatibility specifications for how clients must behave when attempting to use the referrers API with a registry that does not support it, which includes ensuring the same information (the list of artifacts referring to a subject) can be returned by the client even when the referrers API is not supported. So if Bitnami did choose to add metadata via the referrers API, it would not depend on dockerhub support being available.

[absoludity]

With the above 3 PR's, the kubeapps-apis service now includes a GetAvailablePackageMetadatas RPC which is called when an available package displays. This method infers the AppRepository, and if it is an OCI repo, currently fetches all artifacts that are referring to the chart.

In the last PR, I've added a script which I'm using to populate an OCI repository on demo.goharbor.io (need to do this each day) and attaching metadata to it. Unfortunately I've not yet found out why the attached metadata always has the same (incorrect) artifact type (I need to look into this further).

Anyway, with the last PR, I've also updated the dashboard to query, update state, and display any metadata, which looks as follows (again, currently the incorrect artifact type makes them look the same):

Other things we'd want to do beyond this point:

• Use annotations when attaching artifacts so that we can display something more meaningful when present (name, description)
• Update the backend to enable downloading (proxied by the backend so creds are used)

[absoludity]

I've created the list of conditions for completion of the implementation at #7038 , and so will close this investigation.