Support third-party registries

[StephenWeatherford]

Specifically requested (but goal is generic support), in no particular order:

Open:

• goharbor.io (see harbor login #878, connect registries error #3449, container registries can't login to repo1.dso.mil even tho it supports Docker HTTP API V2 #3965)
• ECR (Amazon, see Error connecting to ECR with temporary password #3202, AWS ECR docs #3604)
• GCR (Google) (+3 see Google container registry? #394, Add Google Cloud Products in list to Connect Registry. #2890)
• quay.io (+4 see Add quay.io as registry option #390)
• DTR (Enterprise Docker trusted registry) (see Can't sign in to Docker Enterprise Trusted Registry 2.7.8 #3325)
• Cloudsmith (see Adding Cloudsmith Registry #1887)
• Aliyun (see Not Support Aliyun Docker registry #1964)
• Portus (see Can't sign-in to Generic Docker Registries #2735)
• Digital Ocean (see Support for Digital Ocean registry #2879)
• Generic registries with self-signed certificate (docker register: failed,reason:self signed certificate #3639)
• Artifactory (Unable to connect to jfrog.io-based Artifactory Cloud registry using Docker extension #4011)

Blocked on no/partial Registry V2 support:

• V2 registries that don't use /v2/ endpoint (see The extension corrupts the URL to the docker registry, making it impossible to connect. #3562)
• GitLab self-hosted (+6 see Allow to use self hosted gitlab registry. #1832, Support for self-managed gitlab instance #2457, Can't sign-in to Generic Docker Registries #2735, Support for self-hosted gitlab  #3119, Can't connect to custom Gitlab instance in docker registry #3271, Configuration Option Self-hosted GitLab Instances #3813)
• harbor.korporate.org (see registry conected but eror 405 #1927)
• GitLab Groups (see Add GitLab Groups to list of supported registry providers #1970)
• IBM Container Registry (see Unable to connect to IBM Container Registry #3961)

Done:

• DockerHub
• Azure
• GitLab (+25-ish see add access to gitlab.com registry #162)
• Generic V2 registry (+2)
• GitHub (see Add Github Registry to the supported list #1525, Add support for GitHub pacakges registry #1938, Connect to GitHub #3686)

[gauravlakhani]

while connecting using private registry option, is private Docker DTR (Enterprise Docker trusted registry) supported at all?

[StephenWeatherford]

@gauravlakhani Support for registries beyond Docker Hub is fairly new, we haven't tested on DTR. I've added it to the list.

[shaneholloman]

@here champion!

[imkhairulikhwan]

Hi, do we have an estimated date for this to complete?

[Nhimself]

Following this...

[bwateratmsft]

@BigMorty I think that we should take this one for 1.1...

[BigMorty]

Agree we should try to get it into 1.1 or 1.2

[bwateratmsft]

This could potentially require and/or take advantage of microsoft/vscode#88309

[bwateratmsft]

I've spent a lot of time researching in the past few days. Unfortunately this is going to be extremely difficult, due to two main issues:

1. No / incomplete Registry V2 API support
   • GitHub, Docker Hub lack the _catalog endpoint
   • GitLab has no V2 API support at all, but instead a custom API
   • DTR has no V2 API; or at least I can't find any docs suggesting it does Looks like DTR might: https://success.docker.com/article/how-do-i-use-api-v2-with-dtr
2. OAuth flows totally inconsistent with each other and with the spec
   • Azure and Quay require a web page sign-in
   • Docker Hub has a hilariously broken auth flow--it issues tokens regardless of anything (though the tokens will not work), and does not issue refresh tokens even when asked for them, and does not accept POST's for OAuth tokens like the spec says
   • GCR and ECR use credential helpers--not something we can use easily
   • Quay requires the users to create a client application, and provide its ID to any clients (e.g. this extension). Possible, but onerously difficult.
   • goharbor.io might be the only one with a proper implementation that also permits password auth? At least, their docs sound like it.

Implementing support for each registry with their own custom APIs is a non-starter. Dealing with the OAuth flows is possible, but guaranteed to result in a lot of bad user experiences, due to the complexity and prerequisite setup that most of them demand.

[alexfricker]

+1 for Gitlab self-hosted.

it does look like self hosted uses v2 and _catalog endpoint fwiw

[Mk-arc]

+1 for Gitlab self-hosted.

[OperativeThunny]

+1 for Gitlab self-hosted for me as well. My environment is very constrained so I tried setting up a personal access token in the gitlab self hosted instance, but there is no way using the generic docker option or the gitlab option to connect (tried editing hosts file to point gitlab.com to self hosted instance, but that did not work the url it attempted was a 404)

[mjgardner]

@bwateratmsft wrote:

1. No / incomplete Registry V2 API support

   • GitLab has no V2 API support at all, but instead a custom API

Then @alexfricker wrote:

it does look like self hosted uses v2 and _catalog endpoint fwiw

GitLab’s Container Registry API documentation lists instance-wide Docker V2 API endpoints apart from their custom API. It looks like one just uses HTTP Basic Auth with the username and password (or Personal Access Token if using 2FA) as part of the first request for a Bearer token.

[Hyperclaw79]

Is the Github integration completed? I am able to add the Registry and push to it but I'm seeing an error message when trying to list the packages.

This is the complete error message:
Error: Request with GET/HEAD method cannot have body

Is this related to the discussion regarding GHCR now having _catalog or am I missing something?

[bwateratmsft]

GitHub Container Registry is not yet supported but we are actively working on a significant revamp of the registries functionality, and have included support of GitHub in that work.

[Hyperclaw79]

@bwateratmsft
Great! Thank you for the information. Looking forward to the completion of all the different container registries so that I can manage them all from one place.
Are there any plans for UI implementations for migrating/cloning the images between multiple registries?
I think there is a CLI tool called Crane for such tasks but having a built-in functionality for it in vscode would be a blessing.

[bwateratmsft]

That wasn't in the plan but you could create a new issue to suggest that.

[bwateratmsft]

A fix for this has, at long last, been released. We know this was one of our most popular asks and we wanted to make sure we got it right. Anyone who wants to add support for a registry can now do so. This means nobody has to wait for us to add support for a registry--you can easily do it yourself!

Starting from https://github.com/microsoft/vscode-docker-extensibility/tree/main/packages/vscode-docker-registries (and the associated NPM package), you can either directly implement the RegistryDataProvider interface, or more easily, extend the CommonRegistryDataProvider class, or RegistryV2DataProvider class if your registry implements the V2 spec. The actual implementations for Docker Hub, generic V2 registry, and GitHub are present in the package, and another implementation for Azure Container Registry is present in the Docker extension itself.

One thing we wanted to call attention to--we decided the best place for the GitLab registry data provider to be is in its own extension, which does not yet exist. As a result, support for GitLab Container Registry is removed in Docker extension version 1.27.0. If you need that GitLab support, I would recommend staying on version 1.26.1 of the Docker extension. You can change by running the command "Extensions: Install Specific Version of Extension...", choosing the Docker extension, and choosing version 1.26.1.

[LordMilutin]

I don't know what you did but private registries are no longer visible for connection...
There is no way anymore to connect them. Please bring back this functionality as this was my only way to manage the registry with a nice UI...