Design doc for Import charts from an OCI registry

[absoludity]

Description:

With Helm 3 we have experimental support for OCI registries as chart repositories. For background information, see Upcoming changes to Helm Chart Repositories from early 2019.

One of the advantages listed there which directly impacts Kubeapps is "Another big advantage this brings is the ability to co-locate Helm Charts and Docker images inside the same registry, making it much simpler to establish a chain of trust between a Helm Chart and the Docker images it references in a Deployment template." which will be likely very relevant for fixing #1256 .

As Helm is adopting the OCI spec, chartmuseum will cease with chart repository support in Harbor moving to OCI, Kubeapps will not only need updating to support this change, but can also benefit from the alignment of authn for private images and charts (see #1256).

[andresmgot]

One thing we will need to figure out is how to list charts from a registry in the OCI case

[absoludity]

One thing we will need to figure out is how to list charts from a registry in the OCI case

Is this equivalent to listing the repositories in a registry, which is currently not required but can be implemented: https://github.com/opencontainers/distribution-spec/blob/master/spec.md#listing-repositories

[absoludity]

Also note that docker registry ls is "Docker Enterprise Only": https://docs.docker.com/engine/reference/commandline/registry_ls/

[tvvignesh]

Harbor now has support for OCI registries (https://goharbor.io/blog/harbor-2.0/) It would be great if Kubeapps can support OCI as well and it will be a match made in heaven ❤️

[absoludity]

I had another look at the state of things this morning, given the Harbor 2.0 release. Unfortunately the OCI distribution spec still has repository listing as a future feature (as mentioned above). So while Harbor can nicely list the charts/artifacts in a harbor OCI repository, it can only do this because it maintains a postgresql DB with the contents of the registry.

Unless I'm missing something, Kubeapps (or any other OCI registry client) can only pull (or push) tagged artifacts, or list the tags present in a repository (unless they use a non OCI api such as Harbor's API).

So for the use-case where we created a registry/project https://demo.goharbor.io/bitnami-charts including repositories for each chart and/or image, we cannot currently list the repositories (generically).

Also, what we currently model as an AppRepository (a reference to an index of many charts), does not map to an OCI repository, which would contain multiple tagged versions of the same application chart name (ie. ghost) or image name or other artifact with the same name.

So at the very least, we'd need to use Harbor's API to list the repositories for a project (registry) https://demo.goharbor.io/#/repository/listRepositories . Other options would be to require an index of charts maintained outside the registry, or an index repository with tagged versions of something similar to an index.yaml.

Or moving in the other direction, Kubeapps updates it's model of an AppRepository to map to a repository of an OCI registry - ie a single name for a chart (eg. we add an AppRepository for ghost). Not sure.

Thoughts?

[tvvignesh]

Ahh, you are right. Just noticed this: https://github.com/opencontainers/distribution-spec/blob/master/spec.md#listing-repositories

I clearly understand what you are saying. The only thing I feel is that it is better to limit the choices/options and just stick to what is standard. Since helm, Docker and Harbor have all started leaning towards OCI, I don't see a future scope for repos like chartmuseum, JFrog, etc.

What we can do is have sort of plugins/adapters to kubeapps rather than building everything into this repo. Have an abstraction layer and move chartmuseum, JFrog, OCI integrations into its own separate repo so that it will work in the future for any other specification/library as well without having to change Kubeapps code.

Harbor supports Chartmuseum, OCI, etc. in the same way like you said with a Postgres DB maintained behind the scenes but that would be an overhead for kubeapps. Rather, it can just be an adapters/plugins to maintain standardization. OCI won't be supported till they release list repo feature but when they do, its just a point of plugin change.

Hope this makes sense.

[absoludity]

I clearly understand what you are saying. The only thing I feel is that it is better to limit the choices/options and just stick to what is standard. Since helm, Docker and Harbor have all started leaning towards OCI, I don't see a future scope for repos like chartmuseum, JFrog, etc.

Yeah, I agree, though the transition will take a while.

What we can do is have sort of plugins/adapters to kubeapps rather than building everything into this repo. Have an abstraction layer and move chartmuseum, JFrog, OCI integrations into its own separate repo so that it will work in the future for any other specification/library as well without having to change Kubeapps code.

+100. We've been trying to step back and think about this for other things like the Operator support, or other package support (non helm), but so far haven't made the time in to do it. They don't need to be in a separate repo (at least initially) to be able to demonstrate how to create a plugin.

OCI won't be supported till they release list repo feature but when they do, its just a point of plugin change.

Or another option is that we could support an OCIRegistry (via plugin/adapter if we get to do that), which defines the OCI registry and at first, explicitly requires the chart repositories available in the registry. Later if/when listing repositories within a registry becomes a standard, we could update to that.

[tvvignesh]

Yup. Makes a lot of sense. Infact, thinking about this, even kubeapps can be architected like a plugin. For instance, say you use Octant (https://github.com/vmware-tanzu/octant) you can add kubeapps plugin to octant or drop it anywhere so that it can be integrated with other K8 ops tools. Just my thoughts, may not be a priority.

[absoludity]

Just some additional info: tried using crane today to see how it implements its catalog feature:

crane catalog demo.goharbor.io      
2020/06/10 16:18:44 reading repos for demo.goharbor.io: GET https://demo.goharbor.io/v2/_catalog?n=10000: UNAUTHORIZED: unauthorized to list catalog: unauthorized to list catalog

So it uses the _catalog endpoint which is not required to be implemented currently (future reservation). Though it looks like harbor supports this, the crane tool, as per the spec, ensures that the registry is an authority (restricted to host[:port]), so listing the repos of a project fails on the CLI:

crane catalog demo.goharbor.io/test1
2020/06/10 16:18:47 reading repos for demo.goharbor.io/test1: registries must be valid RFC 3986 URI authorities: demo.goharbor.io/test1

as per the checkRegistry function in the code.

So I think this is also restricted in Harbor to credentials authorized to view the catalog for the entire registry (not limited to specific repository path fragments).

[absoludity]

Related to (or part of) #2078