[Bug]: Failing with "protocol not available, failed to create Docker provider" on Windows

[danielorbach]

Testcontainers version

0.20.1

Using the latest Testcontainers version?

Yes

Host OS

Windows

Host arch

x86

Go version

1.20

Docker version

Client:
 Cloud integration: v1.0.31
 Version:           23.0.5
 API version:       1.42
 Go version:        go1.19.8
 Git commit:        bc4487a
 Built:             Wed Apr 26 16:20:14 2023
 OS/Arch:           windows/amd64
 Context:           default

Server: Docker Desktop 4.19.0 (106363)
 Engine:
  Version:          23.0.5
  API version:      1.42 (minimum version 1.12)
  Go version:       go1.19.8
  Git commit:       94d3ad6
  Built:            Wed Apr 26 16:17:45 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.20
  GitCommit:        2806fc1057397dbaeefbea0e4e17bddfbd388f38
 runc:
  Version:          1.1.5
  GitCommit:        v1.1.5-0-gf19387a
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Docker info

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.4
    Path:     C:\Program Files\Docker\cli-plugins\docker-buildx.exe
  compose: Docker Compose (Docker Inc.)
    Version:  v2.17.3
    Path:     C:\Program Files\Docker\cli-plugins\docker-compose.exe
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-dev.exe
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.19
    Path:     C:\Program Files\Docker\cli-plugins\docker-extension.exe
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.4
    Path:     C:\Program Files\Docker\cli-plugins\docker-init.exe
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-sbom.exe
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-scan.exe
  scout: Command line tool for Docker Scout (Docker Inc.)
    Version:  v0.10.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-scout.exe

Server:
 Containers: 1
  Running: 0
  Paused: 0
  Stopped: 1
 Images: 167
 Server Version: 23.0.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2806fc1057397dbaeefbea0e4e17bddfbd388f38
 runc version: v1.1.5-0-gf19387a
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
 Kernel Version: 5.15.90.1-microsoft-standard-WSL2
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 15.48GiB
 Name: docker-desktop
 ID: 833c289d-cefa-4019-a820-b1b496cc06f6
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

What happened?

I upgraded from v0.19.0 using go get -u and got the latest v0.20.1.

To my surprise running the same tests that had just passed now fails consistently with the message:

protocol not available, failed to create Docker provider

Looking deeper into the changelog of this release I see the error comes from sockets.ConfigureTransport(transport, c.proto, c.addr) at github.com/docker/docker@v24.0.2+incompatible/client/options.go:71.

This option function (client.WithHost) is not called by the previous version because of the introduction of ReadConfig().

Previously, this option is only appended by NewDockerClient() only if the user-environment contains a .testcontainers.properties file that explicitly sets the docker host.

Nowadays, this option is always appended by NewDockerClient() because ReadConfig() always returns constant host, even when not explicitly set by the user-environment.

Relevant log output

I've summarized the difference between the two tags (ignoring comments and whitespaces):

1,3c1,2
< func NewDockerClient() (cli *client.Client, host string, tcConfig TestContainersConfig, err error) {
< 	tcConfig = configureTC()
< 	host = tcConfig.Host
---
> func NewDockerClient() (cli *client.Client, err error) {
> 	tcConfig := ReadConfig()
6,7c5,6
< 	if host != "" {
< 		opts = append(opts, client.WithHost(host))
---
> 	if tcConfig.Host != "" {
> 		opts = append(opts, client.WithHost(tcConfig.Host))
17,20d15
< 	} else if dockerHostEnv := os.Getenv("DOCKER_HOST"); dockerHostEnv != "" {
< 		host = dockerHostEnv
< 	} else {
< 		host = "unix:///var/run/docker.sock"

Additional information

Eventually, on Windows without any custom configuration, v0.19.0 used to access the host npipe:////./pipe/docker_engine.
Without the client.WithHost() option, the DefaultDockerHost is used correctly for Windows and never overridden.

[danielorbach]

@mdelapenya I see the breaking change had been introduced with #941.
I am happy to contribute a fix, could you hint me towards the appropriate resolution?

I fear I don't understand the big picture entirely; I see from the commit message that the break in API was intentional:

break: do not return the Docker Host when retrieving a Docker client (e754c24)
fix: default for docker host (560672a)

[mdelapenya]

@danielorbach indeed, we had to break that method signature in order to reach to a cleaner state. In #1161 we are doing great progress towards having a consistent strategy to bootstrap the Docker client, discovering the Docker host and docker socket path in a single and deterministic manner. Hopefully it will resolve inconsistencies regarding docker alternatives and different setups in the environment/configuration.

Could you give it a try? It was merged past Friday, so I think you could use that commit in your go.mod file

[danielorbach]

Thank you for the quick reply!

Could you give it a try? It was merged past Friday, so I think you could use that commit in your go.mod file

Tests still fail with the same error as before 😢

I've taken the commit from the pull request:

$ go get github.com/testcontainers/testcontainers-go@08a56c9c2999d58db13dacd9b7e445acf3272fdc
go: downloading github.com/testcontainers/testcontainers-go v0.20.2-0.20230526123138-08a56c9c2999
go: downloading golang.org/x/exp v0.0.0-20230510235704-dd950f8aeaea
go: upgraded github.com/imdario/mergo v0.3.13 => v0.3.15
go: upgraded github.com/testcontainers/testcontainers-go v0.20.1 => v0.20.2-0.20230526123138-08a56c9c2999
go: upgraded golang.org/x/exp v0.0.0-20230124195608-d38c7dcee874 => v0.0.0-20230510235704-dd950f8aeaea

-       github.com/testcontainers/testcontainers-go v0.19.0
+       github.com/testcontainers/testcontainers-go v0.20.2-0.20230526123138-08a56c9c2999

[sebastianbuechler]

Can reproduce on Windows 11.

If I add tcp://localhost:2375 as a system env for the name DOCKER_HOST the setup works again.

I think the protocol part (unix) of the default value for docker host unix:///var/run/docker.sock is not recognized as valid when the config is read and thus the exception is thrown.

[meysamhadeli]

I still this error protocol not available in version v0.20.1.
Do you have any update for fixing this problem?

[danielorbach]

@mdelapenya I tried the latest main this morning to no avail.
Looking at the latest PRs, I see you guys have explicitly commented to ignore Windows containers and default to the unix socket when no other docker-host has been provided by the environment.

Would you like me to propose a pull-request addressing this?
The latest refactor seems to collate code-paths that need a docker configuration, I think it can be as small as a few lines of code.

[mdelapenya]

Hi @danielorbach I'm back from a week on PTO, so I'm basically catching up today.

Indeed, but IIRC we were ignoring windows containers, not windows usage... although I could probably have committed a mistake on it. I'm lacking a testing windows instance to perform consistent and repeatable tests for Windows at the moment, but this is something we are working on fixing it asap.

Would you like me to propose a pull-request addressing this?

That would be super!

The latest refactor seems to collate code-paths that need a docker configuration, I think it can be as small as a few lines of code.

Indeed, we tried to have the docker socket resolution as simple as possible, so you could probably figure it out in a very easy manner

[sebastianbuechler]

Hi @danielorbach I'm back from a week on PTO, so I'm basically catching up today.

Indeed, but IIRC we were ignoring windows containers, not windows usage... although I could probably have committed a mistake on it. I'm lacking a testing windows instance to perform consistent and repeatable tests for Windows at the moment, but this is something we are working on fixing it asap.

Would you like me to propose a pull-request addressing this?

That would be super!

The latest refactor seems to collate code-paths that need a docker configuration, I think it can be as small as a few lines of code.

Indeed, we tried to have the docker socket resolution as simple as possible, so you could probably figure it out in a very easy manner

Wouldn't it make sense in general to expand the GitHub Actions to be run on more than just Ubuntu? For example adding something like this to the test workflows?

jobs:
  test-postgres:
    strategy:
      matrix:
        go-version: [1.19.x, 1.x]
        os: [ubuntu-latest, windows-latest, macos-latest]
    runs-on: ${{ matrix.os }}

This would have detected this bug I think: https://github.com/sebastianbuechler/testcontainers-go/actions/runs/5270951017/jobs/9531389308

Btw: I think the pipeline is using the wrong path for the TestSummary action:

      - name: Test Summary
        uses: test-summary/action@4ee9ece4bca777a38f05c8fc578ac2007fe266f7
        with:
          paths: "**/TEST-postgres*.xml"
        if: always()

Where in the gotestsum this is logged out: --junitfile TEST-unit.xml So the TestSummary says it has not found any tests.

[mdelapenya]

@sebastianbuechler

This would have detected this bug I think: https://github.com/sebastianbuechler/testcontainers-go/actions/runs/5270951017/jobs/9531389308

Indeed, a Windows worker would have caught it, although GH Windows workers does not support container operations: actions/runner#904 (comment)

We are exploring Azure for that, and we will have a working Windows pipeline hopefully soon

Btw: I think the pipeline is using the wrong path for the TestSummary action:

Would you mind opening a separate ticket, and if able send a PR with the fix 🙏 ? I'd appreciate that!

[sebastianbuechler]

@sebastianbuechler

This would have detected this bug I think: https://github.com/sebastianbuechler/testcontainers-go/actions/runs/5270951017/jobs/9531389308

Indeed, a Windows worker would have caught it, although GH Windows workers does not support container operations: actions/runner#904 (comment)

We are exploring Azure for that, and we will have a working Windows pipeline hopefully soon

Btw: I think the pipeline is using the wrong path for the TestSummary action:

Would you mind opening a separate ticket, and if able send a PR with the fix 🙏 ? I'd appreciate that!

Thanks for the information. Hopefully it will work with the Azure approach 👍

I opened another bug report #1288 and also a PR #1287.