skopeo not inheriting outgoing proxy configuration

[antifob]

Required information

• Distribution: Debian
• Distribution version: bookworm
• The output of "incus info" or if that fails:
  • Kernel version:
  • LXC version:
  • Incus version: 6.8
  • Storage backend in use:

Issue description

skopeo does not inherit the outgoing proxy settings of core.proxy_http and core.proxy_https.
In some configurations, this results in an Image not found error due to skopeo not being able to reach the Internet.

Steps to reproduce

The below demonstrate that skopeo does not contact the configured proxy server.

# listen on a tcp port
nc -lnvp 1080

# point incusd to the "proxy"
incus config set core.proxy_https=http://127.0.0.1:1080

incus remote add docker https://docker.io --protocol=oci
incus launch --console docker:hello-world --ephemeral

# expected: the proxy/netcat is poked by incusd

Information to attach

Underlying error as seen from incus monitor.

metadata:
  context:
    name: hello-world
    stderr: 'Failed to run: skopeo inspect docker://docker.io/hello-world: exit status
      1 (time="2024-12-13T14:23:16Z" level=fatal msg="Error parsing image name \"docker://docker.io/hello-world\":
      pinging container registry registry-1.docker.io: Get \"https://registry-1.docker.io/v2/\":
      dial tcp: lookup registry-1.docker.io on 9.9.9.9:53: dial udp 9.9.9.9:53: connect:
      network is unreachable")'
    stdout: ""
  level: debug
  message: Error getting image alias

Workaround

For the Zabbly-provided package...

cat>>/etc/default/incus<<__EOF__
http_proxy=http://proxyhost:port
https_proxy=http://proxyhost:port
__EOF__
systemctl restart incus.service

[winiciusallan]

Should we override the HTTPS_PROXY environment variable if core.proxy_http or core.proxy_https exists? Something like HTTPS_PROXY skopeo inspect ...

here

incus/client/oci_images.go

Line 309 in ed2def1

stdout, err := subprocess.RunCommand("skopeo", "inspect", fmt.Sprintf("%s/%s", strings.Replace(r.httpHost, "https://", "docker://", -1), name))

I tested it in my environment

$ nc -lnvp 1080

$ HTTPS_PROXY=http://127.0.0.1:1080 skopeo inspect docker://docker.io/hello-world

# netcat received the connection
$ nc -lnvp 1080
Listening on 0.0.0.0 1080
Connection received on 127.0.0.1 47912
CONNECT registry-1.docker.io:443 HTTP/1.1
Host: registry-1.docker.io:443
User-Agent: Go-http-client/1.1

[stgraber]

We already pass proxy information from the server config to the client logic.

So from within oci_images.go, we should be able to get the proxy by calling r.http.Transport.(*http.Transport).Proxy with a fake http.Request roughly matching what we want to access (scheme and host must be correct at least).

[winiciusallan]

Hi @stgraber. Could you assign this issue to me? I was doing some tests, and I made certain progress. Thanks!

[winiciusallan]

Now, I'm able to get the proxy with what you said.

req, err := http.NewRequest("GET", r.httpHost, nil)
proxy, err := r.http.Transport.(*http.Transport).Proxy(req)

So, do I need to pass it to the skopeo command as I suggested in the previous comment or set it to ProtocolOCI.http?

[stgraber]

Right, you'd want to use subprocess.RunCommandSplit instead so you can pass it the environment.

[winiciusallan]

Hi @stgraber, I tested the changes that I made. Now, it is possible to connect to the proxy, but with the example of the issue, the image can not be inspected, even if the netcat connects to the docker registry. I don't know if it is a firewall issue in my VM or another thing. If you could give me some feedback, I'd appreciate it.