[Bug]: wrong credentials are used when no default registry configured and credential config contains URL without scheme

[p-jahn]

Testcontainers version

0.30.0

Using the latest Testcontainers version?

Yes

Host OS

macOS

Host arch

ARM

Go version

1.22

Docker version

Client: Docker Engine - Community
 Version:           26.0.0
 API version:       1.41 (downgraded from 1.45)
 Go version:        go1.22.1
 Git commit:        2ae903e86c
 Built:             Wed Mar 20 15:10:03 2024
 OS/Arch:           darwin/arm64
 Context:           default

Server: linux/arm64/fedora-39
 Podman Engine:
  Version:          4.9.0
  APIVersion:       4.9.0
  Arch:             arm64
  BuildTime:        2024-01-24T11:07:09+01:00
  Experimental:     false
  GitCommit:        
  GoVersion:        go1.21.6
  KernelVersion:    6.7.4-200.fc39.aarch64
  MinAPIVersion:    4.0.0
  Os:               linux
 Conmon:
  Version:          conmon version 2.1.8, commit: 
  Package:          conmon-2.1.8-2.fc39.aarch64
 OCI Runtime (crun):
  Version:          crun version 1.14
commit: 667e6ebd4e2442d39512e63215e79d693d0780aa
rundir: /run/crun
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  Package:          crun-1.14-1.fc39.aarch64
 Engine:
  Version:          4.9.0
  API version:      1.41 (minimum version 1.24)
  Go version:       go1.21.6
  Git commit:       
  Built:            Wed Jan 24 11:07:09 2024
  OS/Arch:          linux/arm64
  Experimental:     false

Docker info

Client: Docker Engine - Community
 Version:    26.0.0
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.13.1
    Path:     /Users/pjahn/.docker/cli-plugins/docker-buildx

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 5
 Server Version: 4.9.0
 Storage Driver: overlay
  Backing Filesystem: xfs
  Supports d_type: true
  Native Overlay Diff: false
  Using metacopy: true
  Supports shifting: true
  Supports volatile: true
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge macvlan ipvlan
  Log: k8s-file none passthrough journald
 Swarm: inactive
 Runtimes: runc runj runsc youki crun crun-wasm ocijail kata krun
 Default Runtime: crun
 Init Binary: 
 containerd version: 
 runc version: 
 init version: 
 Security Options:
  seccomp
   Profile: default
  selinux
 Kernel Version: 6.7.4-200.fc39.aarch64
 Operating System: fedora
 OSType: linux
 Architecture: arm64
 CPUs: 6
 Total Memory: 1.904GiB
 Name: localhost.localdomain
 ID: b13c4eda-2e97-4945-9731-3a2d4ebcb6a6
 Docker Root Dir: /var/lib/containers/storage
 Debug Mode: false
 Experimental: true
 Live Restore Enabled: false
 Product License: Apache-2.0

What happened?

Trying to create a Testcontainer with default Ryuk config fails to pull the Ryuk image in Podman, if credentials without URL scheme are present.

Relevant log output

=== RUN   TestSuite
2024/04/16 14:05:24 github.com/testcontainers/testcontainers-go - Connected to docker: 
  Server Version: 4.9.0
  API Version: 1.41
  Operating System: fedora
  Total Memory: 1949 MB
  Resolved Docker Host: unix:///var/run/docker.sock
  Resolved Docker Socket Path: /var/run/docker.sock
  Test SessionID: bab906f8b9bac86f69ccd07d4da6fb3da04615205abbcce115a021fb855ac82a
  Test ProcessID: 14d2b0a5-4c27-42c3-ac54-3945b168c492
2024/04/16 14:05:25 Failed to pull image: Error response from daemon: {"message":"unable to retrieve auth token: invalid username/password: unauthorized: incorrect username or password"}, will retry
2024/04/16 14:05:26 Failed to pull image: Error response from daemon: {"message":"unable to retrieve auth token: invalid username/password: unauthorized: incorrect username or password"}, will retry
[....]

Additional information

Prerequisites:

• Podman is used as the engine
• Ryuk image was not pulled before
• Docker credentials are stored, for a registry that has no URL scheme

What happens when docker_auth.go tries to find registry credentials:

• checks what registry to use
  • sets default registry by calling /info, request is successful but registry is empty "" (see related Podman issue)
  • tries to look up registry from image testcontainers/ryuk:0.7.0 -> no registry
  • falls back to empty "" default value
• fetches auth config
• tries to match config with registry
  • (remember: registry is empty "")
  • finds auth entry without URL scheme, like artifactory.mycompany.com
  • parses the URL successfully, since relative URL's are allowed by url.Parse()
  • keyURL.Host == reg matches, because parsing a scheme-less URL results in an empty host
  • random credentials are used

---

I think handling scheme-less URL's in the host based matching would be the correct thing to do here, and I'm happy to provide a fix if maintainers agree.

[mdelapenya]

Hi @p-jahn thanks for opening this issue. It's unfortunate that Podman works differently here, but the workaround would be to always pass the fully qualified name of the Docker image.

If you consider you have bandwidth for it, please send the update 🙏

[mdelapenya]

In any case, thinking out loud, I guess this behaviour is forcing you to have all the images declared with their FQN, right?

[p-jahn]

Hey @mdelapenya,
yes, that workaround would also be possible but pulling Ryuk would still be broken as it's image name is a constant in the internals. I guess setting hub.image.name.prefix could help but this then also requires that Ryuk is available in the prefix registry.

I would rather upgrade the host matching to not match empty/missing registry values against random hosts in .docker/config.json when the host there does not have an URI scheme. I think this is a general bug that is only more pronounced in Podman because of it default behaviour.
What do you think?

Edit:
misread the first answer, I'll provide a patch 🙏

forcing you to have all the images declared with their FQN

basically, yes