Pulling docker images from behind a company VPN

[olliekennedy]

Hey I've been struggling to apply the correct proxy settings to the colima vm so that I can pull docker images. I've tried various methods with no success, mostly from this thread: https://stackoverflow.com/questions/23111631/cannot-download-docker-images-behind-a-proxy

The most common method seems to be creating /etc/systemd/system/docker.service.d/http-proxy.conf and applying HTTP_PROXY, HTTPS_PROXY and NO_PROXY like this:

[Service]
Environment="HTTP_PROXY=http://proxy.example.com:80"
Environment="HTTPS_PROXY=https://proxy.example.com:443"
Environment="NO_PROXY=localhost,127.0.0.1,docker-registry.example.com,.corp"

...but I've had no success with that. I've also tried adding to daemon.json with no success:

{
 "proxies":
 {
   "default":
   {
     "httpProxy": "http://192.168.1.12:3128",
     "httpsProxy": "http://192.168.1.12:3128",
     "noProxy": "*.test.example.com,.example2.com,127.0.0.0/8"
   }
 }
}

I managed to get it working before colima was switched to alpine, but what is the correct method now? Thanks for your help!

[abiosoft]

Can you try the following?

Note: this assumes Colima v0.4.0 or newer.

1. SSH into the VM colima ssh
2. Edit docker init script sudo vi /etc/init.d/docker. You can install other editors e.g. sudo apk install vim nano.
3. Append with the following

   set -o allexport
   if [ -f /etc/environment ]; then source /etc/environment; fi
   set +o allexport

4. Stop Colima colima stop
5. Set the environment variable in the Colima config. colima start --edit or edit $HOME/.colima/default/colima.yaml

   env:
     HTTP_PROXY: http://proxy.example.com:80
     HTTPS_PROXY: http://proxy.example.com:443
     NO_PROXY: localhost,127.0.0.1,docker-registry.example.com,.corp

6. Start Colima colima start

[olliekennedy]

Thank you very much, that is exactly what I needed! Works perfectly now.

One follow on question - after doing the above, then completely deleting colima with colima stop, colima delete, brew uninstall colima and deleting the ~/.colima folder, when I reinstall and create a new colima VM, my proxy environment variables are already there! Do you know how they are getting populated / where they are cached, even beyond a complete uninstall? Thanks again

[abiosoft]

Proxy environment variables (HTTP_PROXY, HTTPS_PROXY, NO_PROXY) are always forwarded to the VM as long as they are available in the shell before you run colima start, irrespective of their presence in the Colima config file.
That is most likely what is happening.

The issue is that Docker's init file does not utilise the environment variables by default. In your case after the reinstall, I would not expect your Docker commands to outrightly work as before, but the environment variables will be present.

[olliekennedy]

Fantastic, thank you so much for your help, you've unblocked us! Appreciate your time :)

[abiosoft]

Now that the issue is known, a proper fix will be worked on so you would not need to manually edit the init script.

Thanks.

[mdavino]

I solved it this way:

sudo vi /etc/systemd/system/docker.service.d/docker.conf

Append:

EnvironmentFile=/etc/environment

[abiosoft]

I solved it this way:

sudo vi /etc/systemd/system/docker.service.d/docker.conf

Append:

EnvironmentFile=/etc/environment

@mdavino this can be handled automatically by Colima. Thanks for the information.

[mdavino]

@abiosoft ubuntu services are started by systemd. Then, the indicated workaround editing /etc/init.d/docker don't work for me.
¹ "The SCRIPT parameter specifies a System V init script, located in /etc/init.d/SCRIPT, or the name of a systemd unit. The existence of a systemd unit of the same name as a script in /etc/init.d will cause the unit to take precedence over the init.d script."

Thanks.

Footnotes

1. https://manpages.ubuntu.com/manpages/lunar/en/man8/service.8.html ↩

[CallJ051]

Can you try the following?

Note: this assumes Colima v0.4.0 or newer.

1. SSH into the VM `colima ssh`

2. Edit docker init script `sudo vi /etc/init.d/docker`.  You can install other editors e.g. `sudo apk install vim nano`.

3. Append with the following
   ```shell
   set -o allexport
   if [ -f /etc/environment ]; then source /etc/environment; fi
   set +o allexport
   ```

4. Stop Colima `colima stop`

5. Set the environment variable in the Colima config. `colima start --edit` or edit `$HOME/.colima/default/colima.yaml`
   ```yaml
   env:
     HTTP_PROXY: http://proxy.example.com:80
     HTTPS_PROXY: http://proxy.example.com:443
     NO_PROXY: localhost,127.0.0.1,docker-registry.example.com,.corp
   ```

6. Start Colima `colima start`

Thanks @abiosoft !
After a lot of browsing and googling around, these steps solved my issue as well, even in June 2024.
However, I had to rely on apt-get instead to install an editor (sudo apt-get update && sudo apt-get install vim).
Step 5 was not required for me, since my proxy environment variables (HTTP_PROXY, HTTPS_PROXY, ALL_PROXY) are already set and present in my shell.

[SarwarSaif]

Even after following the steps above for adding the proxy, I'm getting this error:
Get "https://xxxxxx.lab": dial tcp: lookup xxxxxx.lab on 127.0.0.53:53: no such host

I'm using Mac M1 with Sonoma 14.3.1.

[CallJ051]

Can you try the following?
Note: this assumes Colima v0.4.0 or newer.

1. SSH into the VM `colima ssh`

2. Edit docker init script `sudo vi /etc/init.d/docker`.  You can install other editors e.g. `sudo apk install vim nano`.

3. Append with the following
   ```shell
   set -o allexport
   if [ -f /etc/environment ]; then source /etc/environment; fi
   set +o allexport

4. Stop Colima colima stop

5. Set the environment variable in the Colima config. colima start --edit or edit $HOME/.colima/default/colima.yaml

   env:
     HTTP_PROXY: http://proxy.example.com:80
     HTTPS_PROXY: http://proxy.example.com:443
     NO_PROXY: localhost,127.0.0.1,docker-registry.example.com,.corp

6. Start Colima colima start

Thanks @abiosoft ! After a lot of browsing and googling around, these steps solved my issue as well, even in June 2024. However, I had to rely on apt-get instead to install an editor (sudo apt-get update && sudo apt-get install vim). Step 5 was not required for me, since my proxy environment variables (HTTP_PROXY, HTTPS_PROXY, ALL_PROXY) are already set and present in my shell.

Meanwhile, I'm on Mac Sonoma 14.6.1 and colima 0.7.3. I now have to provide the proxy environment variables as well in $HOME/.colima/default/colima.yaml. Otherwise, the proxies object in the generated daemon.json would not be properly updated.
I'm using preproxy.
My proxy environment variables in my system are the following (note, these are setup by my employer's configuration tools):

➜  ~ printenv | grep _proxy
http_proxy=http://localhost:3128
https_proxy=http://localhost:3128
all_proxy=http://localhost:3128
no_proxy=localhost,127.0.0.1

In the colima.yaml file, I provide the http_proxy and https_proxy variables, while replacing localhostwith the hardcoded loopback address

# Environment variables for the virtual machine.
# If you use `preproxy`, provide your http_proxy and https_proxy env variables, and replace localhost with `192.168.5.2` (the lima loopback address https://lima-vm.io/docs/config/network/#host-ip-19216852),
# to make sure that the `proxies` value in /etc/docker/daemon.json is populated accordingly.
env:
  http_proxy: http://192.168.5.2:3128
  https_proxy: http://192.168.5.2:3128

This ensures that colima start generates the appropriate daemon.json with correct proxies.

➜  ~ colima ssh                                                  
gu77qc@colima:/Users/gu77qc$ sudo cat /etc/docker/daemon.json 
{
  "exec-opts": [
    "native.cgroupdriver=cgroupfs"
  ],
  "features": {
    "buildkit": true
  },
  "proxies": {
    "http-proxy": "http://192.168.5.2:3128",
    "https-proxy": "http://192.168.5.2:3128",
    "no-proxy": "localhost,127.0.0.1"
  }
}

Without these changes (so env: {}), I end up with the same problems.

In my case, removing the proxies in /etc/docker/daemon.json also did the trick, but it gets populated on each colima start. Using the provision steps in colima.yaml didn't help me, as in my understanding, my changes to the daemon.json are overriden after the provision step anyway.
Just sharing what works for me, please correct me if I'm making wrong assumptions.

For completeness sake, below you can find the output without and with my changes:

Without changes -- env: {}

➜  ~ colima start --arch x86_64 --cpu 4 --memory 4               
INFO[0000] starting colima                              
INFO[0000] runtime: docker                              
INFO[0000] starting ...                                  context=vm
INFO[0057] provisioning ...                              context=docker
INFO[0063] starting ...                                  context=docker
INFO[0068] done                                         
➜  ~ docker pull hello-world                      
Using default tag: latest
Error response from daemon: Get "https://registry-1.docker.io/v2/": proxyconnect tcp: dial tcp 127.0.0.1:3128: connect: connection refused
➜  ~ colima ssh                                   
gu77qc@colima:/Users/gu77qc$ sudo cat /etc/docker/daemon.json 
{
  "exec-opts": [
    "native.cgroupdriver=cgroupfs"
  ],
  "features": {
    "buildkit": true
  },
  "proxies": {
    "http-proxy": "http://localhost:3128",
    "https-proxy": "http://localhost:3128",
    "no-proxy": "localhost,127.0.0.1"
  }
gu77qc@colima:/Users/gu77qc$ exit
logout
➜  ~ colima stop                                  
INFO[0000] stopping colima                              
INFO[0000] stopping ...                                  context=docker
INFO[0002] stopping ...                                  context=vm
INFO[0006] done                                         

Update colima.yaml with

env:
  http_proxy: http://192.168.5.2:3128
  https_proxy: http://192.168.5.2:3128

➜  ~ colima start --arch x86_64 --cpu 4 --memory 4
INFO[0000] starting colima                              
INFO[0000] runtime: docker                              
INFO[0002] starting ...                                  context=vm
INFO[0051] provisioning ...                              context=docker
INFO[0058] starting ...                                  context=docker
INFO[0063] done                                         
➜  ~ docker pull hello-world                      
Using default tag: latest
latest: Pulling from library/hello-world
Digest: sha256:53cc4d415d839c98be39331c948609b659ed725170ad2ca8eb36951288f81b75
Status: Image is up to date for hello-world:latest
docker.io/library/hello-world:latest
➜  ~ colima ssh                                   
gu77qc@colima:/Users/gu77qc$ sudo cat /etc/docker/daemon.json 
{
  "exec-opts": [
    "native.cgroupdriver=cgroupfs"
  ],
  "features": {
    "buildkit": true
  },
  "proxies": {
    "http-proxy": "http://192.168.5.2:3128",
    "https-proxy": "http://192.168.5.2:3128",
    "no-proxy": "localhost,127.0.0.1"
  }
}

[mikehearn]

The init script editing worked for me! Strange that init scripts still exist in 2024 :)

It'd be great if this stuff could be synced from the macOS System Settings.