Private registry with a mirror on host

[avishefi]

Question

I have an air-gapped Kubernetes cluster configured so that the registry mirrors are defined on the host and all references to an image go to a private registry. For example: aquasecurity/trivy is automatically fetched from <private-registry>/aquasecurity/trivy without specifying image pull secrets or a registry explicitly.

Trivy can't fetch images defined this way (used through trivy-operator) and attempts to fetch all images from docker.io instead of the private registry.

Is there a way for Trivy to fetch images from the private registry configured this way? I tried both Standalone and ClientServer and couldn't find a configuration for that.

Target

Container Image

Scanner

Vulnerability

Output Format

None

Mode

None

Operating System

OpenShift 4.11, Kubernetes 1.24, RHEL 8

Version

0.44.0

[knqyf263]

@avishefi It is a problem with trivy-operator, right?

@chen-keinan If I remember correctly, you told me about this enhancement. Do you have an issue in GitHub? I couldn't find it.

[avishefi]

I think this is missing in Trivy since there is no option to work with authentication against a private registry in Trivy configured as a default mirroring without specifying the registry explicitly. The default fallback to docker.io seems to be an issue.

[knqyf263]

Private registries are supported.

[avishefi]

Private registries are supported, but in the case of a mirror configured outside of Trivy's knowledge, it attempts to contact the default registry which is docker.io.

Is there a way for Trivy to work like that? Or maybe the trivy-operator?

[chen-keinan]

@knqyf263 this is the related issue in operator

[avishefi]

I think an optional CLI flag for a default registry or mirror with credentials which precedes the default registry or even replaces it can solve this scenario.

[itaysk]

check out the --image-src flag:
https://aquasecurity.github.io/trivy/v0.42/docs/target/container_image/#supported

[avishefi]

The only relevant scenario is container registry as trivy is executed inside a pod, and that still doesn't work due to mirrors defined on the host.

[avishefi]

Related also to #3004

[caleb-devops]

This feature would also help when using containerd registry mirrors.

Example

# /etc/containerd/certs.d/docker.io/hosts.toml

server = "https://docker.io"

[host."https://harbor.example.com/v2/docker.io"]
  capabilities = ["pull", "resolve"]
  override_path = true

By using the above config, a Kubernetes pod that attempts to use docker.io/library/busybox:latest will instead be redirected to harbor.example.com/docker.io/library/busybox:latest

Trivy is not able to make use of this transparent redirection, and will instead attempt to pull the image directly from docker.io/library/busybox:latest.

[rgarcia89]

Have you found a solution in the meanwhile? I am facing the same issue right now

[caleb-devops]

Hey @itaysk, thanks for talking to me about this issue at KubeCon! As we discussed, do you think an option could be added to the Trivy config file that would permit string replaces for registry mirrors?

Example:

registry-mirrors:
  - src: "docker.io"
    dest: "harbor.example.com/docker.io"
  - src: "ghcr.io"
    dest: "harbor.example.com/ghcr.io"

Note: For images that originate from Docker Hub, Trivy would also need to expand the image to its full path (i.e. docker.io/library/alpine instead of alpine) before replacing the string.

[itaysk]

thanks. opened #7966