-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow image-builder to consume and publish to different image registries with different docker credentials #12060
Comments
Please correct me when I'm wrong, @Pothulapati. That is what I understood: It seems the following components using the registry auth credentials:
The problem is that the components allow using the built-in registry auth OR the external registry auth but not both. But customers that want to pull workspace images from a registry that needs auths (instead of Docker Hub) need to add an additional registry secret. That happens regularly when using air-gap because the workspace images will be mirrored in a local registry that is accessible in the air-gapped network. We cannot merge the built-in registry auth and the user-given registry auth in the installer because the installer does not have access to the user-given registry auth (which is a K8s secret) during the rendering phase (render does not have access to the cluster). Question (for Team Workspace?): Which of the components listed above need to pull the workspace images? Only image builder? Components that need to pull workspace images need to accept two secrets (for the built-in registry as well as the user-given secret). |
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
I want to bump this because it's still an issue. Right now you can't use ECR as a backing store for Gitpod, despite the new ECR flag. The reason for this is, whilst the imagebuilder-mk3 is able to auth and check the image, it does not pass the ECR auth through to the image-builder-bob. It only passes through the docker config. This means that the bob can authenticate and pull images from a private-registry or ECR, but cannot push to ECR. gitpod/components/image-builder-mk3/pkg/orchestrator/orchestrator.go Lines 382 to 388 in 6499416
It would be a quick win to switch this out for an if statement that asks "is the workspace registry ECR?" if yes pass through the ECR auth, and if no pass through the dockerconfig like normal. The inclusion of this would allow deployments to use ECR natively rather than relying on a cluster local docker-registry or some other managed docker registry. |
Problem
The image builder can only reference one secret object, and though that object can list 20 repositories in it and their corresponding credentials, we have no facility to manage or merge / combine multiple user provided registry credentials into a secret object.
This means that the image-builder cannot pull an image from one registry that requires credentials and then push to another different registry that requires credentials (It can if one of these is public). This is an issue for those cases where a customer is trying to get the workspace base image from one registry and them push the built image for a workspace (that adds customisations on top of the base image) to another image registry ready to be used to start workspaces. In practice, this means you cannot use an in-cluster registry to store workspace images and a private (i.e. one that needs credentials) registry to pull base images from. This is an issue especially in airgapped environments that use the in-cluster registry (we recommend AWS users to use the in-cluster registry backed by s3 for storage), which usually always use a private registry for base images.
❓ Why do we recommend using the in-cluster repository and not the other existing one?
We don’t support all registries for pushing to them (we really only test against GCR and Docker Registry, i’ve not got workspace-image push working with artifactory or harbor), they don’t want workspace-images (multi gigabyte images) stored in their registry space optimized for serving small containers very quickly, or where they have to pay per image push for scanning / auditing, etc❓ Why don't we support all registries?
It wouldn't also solve the issue, but to your question for example, we lack ECR support because:Technical details — Self-hosted team
With #10792, We started investing on ways to allow docker credentials to be passed by the user even when
incluster
registry is enabled, to allow for workspace base images to be pulled from different (& usually private) registries.As the above issue describes, The fix initially seemed simple i.e to move
credentials
out ofexternal
so that a registry credentials secret can be passed even when an external registry is not being used.Roadblock
Internally, The current way this works is in a either or approach i.e pass the secret if external registry is used, or pass
builtin-registry-credentials
ifinclusterRegistry
is enabled.Now, We need a way to pass multiple secrets and thats where it gets tricky, as the current components (that need the registry communication) configs seem to take only a single dockerConfig file.
The
installer
's conventions doesn't let us merge these things for the following reasons:Potential solution
Components should instead have the ability to use multiple files, merging them before using them as a registry. This feels right as it confirms to the installer conventions, where the instalelr would just mount these secrets correctly.
Describe alternatives you've considered
As discussed above, the alternatives would break conventions but were considered.
Additional context
The text was updated successfully, but these errors were encountered: