Flaky TestRedistest

[AlexanderYastrebov]

2023/09/25 11:41:03 github.com/testcontainers/testcontainers-go - Connected to docker: 
  Server Version: 20.10.23
  API Version: 1.41
  Operating System: Ubuntu 20.04.6 LTS
  Total Memory: 31402 MB
  Resolved Docker Host: unix:///var/run/docker.sock
  Resolved Docker Socket Path: /var/run/docker.sock
  Test SessionID: 0bdf3f1121aaca992853190b856d6f0dea6a87d4314bfa596220d34c60b5353f
  Test ProcessID: e714cc5c-639d-423c-be15-256c609eb5f8
--- FAIL: TestRedistest (26.57s)
    redistest.go:39: Failed to start redis server: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.41/containers/a27df84a5ca5910655b167bceb730bcc3800d01be6005fe4d43a04c7111de54b/json": context deadline exceeded: creating reaper failed: failed to create container

[MustafaSaber]

On Mac we always face creating reaper failed: failed to create container, so we disable it via export TESTCONTAINERS_RYUK_DISABLED=true but this won't be good on CI environment I think.

[AlexanderYastrebov]

--- FAIL: TestRedistest (27.04s)
    redistest.go:39: Failed to start redis server: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.41/containers/3f1705e8160379b38a506c25ff2a9ca773f1673a369040c20dfb40e5342ef067/json": context deadline exceeded: creating reaper failed: failed to create container

[AlexanderYastrebov]

--- FAIL: TestRedistest (27.30s)
    redistest.go:39: Failed to start redis server: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.41/containers/02a8b90845b6d23098051cbaa2e45261b90e20901ad882254a54cac3eff807e7/json": context deadline exceeded: creating reaper failed: failed to create container

[szuecs]

--- FAIL: TestRedistest (27.68s)
    redistest.go:39: Failed to start redis server: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.41/containers/9aef1c39ddc3efebaf4f754756323a5fdf702c5aa42b756a7c653a997a057fd8/json": context deadline exceeded: creating reaper failed: failed to create container

[RomanZavodskikh]

Moved to #2556 (comment)

[szuecs]

--- FAIL: TestRedistest (30.99s)
    redistest.go:39: Failed to start redis server: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.41/containers/8e2c7db32998c56635b2d020fc9dbad82b1e005798894fd0fc9c746de0c01570/json": context deadline exceeded: creating reaper failed: failed to create container
FAIL

[szuecs]

PASS
noleak: 2 active

goroutine 19 [IO wait]:
internal/poll.runtime_pollWait(0x7fd63a27bdb8, 0x72)
	/opt/hostedtoolcache/go/1.21.3/x64/src/runtime/netpoll.go:343 +0x85
internal/poll.(*pollDesc).wait(0xc0003e0180?, 0xc0003a3000?, 0x0)
	/opt/hostedtoolcache/go/1.21.3/x64/src/internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	/opt/hostedtoolcache/go/1.21.3/x64/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0003e0180, {0xc0003a3000, 0x1000, 0x1000})
	/opt/hostedtoolcache/go/1.21.3/x64/src/internal/poll/fd_unix.go:164 +0x27a
net.(*netFD).Read(0xc0003e0180, {0xc0003a3000?, 0x7fcfc0?, 0xc000215e00?})
	/opt/hostedtoolcache/go/1.21.3/x64/src/net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc0003a4570, {0xc0003a3000?, 0x0?, 0xc0002fa4e0?})
	/opt/hostedtoolcache/go/1.21.3/x64/src/net/net.go:179 +0x45
net/http.(*persistConn).Read(0xc0003a7560, {0xc0003a3000?, 0xc0003ec420?, 0xc000065d38?})
	/opt/hostedtoolcache/go/1.21.3/x64/src/net/http/transport.go:1954 +0x4a
bufio.(*Reader).fill(0xc0003982a0)
	/opt/hostedtoolcache/go/1.21.3/x64/src/bufio/bufio.go:113 +0x103
bufio.(*Reader).Peek(0xc0003982a0, 0x1)
	/opt/hostedtoolcache/go/1.21.3/x64/src/bufio/bufio.go:151 +0x53
net/http.(*persistConn).readLoop(0xc0003a7560)
	/opt/hostedtoolcache/go/1.21.3/x64/src/net/http/transport.go:2118 +0x1b9
created by net/http.(*Transport).dialConn in goroutine 18
	/opt/hostedtoolcache/go/1.21.3/x64/src/net/http/transport.go:1776 +0x169f

goroutine 20 [select]:
net/http.(*persistConn).writeLoop(0xc0003a7560)
	/opt/hostedtoolcache/go/1.21.3/x64/src/net/http/transport.go:2421 +0xe5
created by net/http.(*Transport).dialConn in goroutine 18
	/opt/hostedtoolcache/go/1.21.3/x64/src/net/http/transport.go:1777 +0x16f1

Try setting GODEBUG=tracebackancestors=10 (or more), see https://pkg.go.dev/runtime#hdr-Environment_Variables
FAIL	github.com/zalando/skipper/net/redistest	6.765s

[szuecs]

CDP error

--- FAIL: TestRedistest (10.54s)
    redistest.go:39: Failed to start redis server: dial tcp 172.31.8.1:32768: i/o timeout: Connecting to Ryuk on 172.31.8.1:32768 failed: connecting to reaper failed: failed to create container

[AlexanderYastrebov]

but this won't be good on CI environment I think.

I think we actually do not needed reaper in CI, I've created #2930 to disable.

[mdelapenya]

Hi all, latest releases create Ryuk the reaper in a more consistent manner. I develop on Mac and I do not see issues with it. I'd love to check with you if the most recent version fixes this issue for you 🙏

[MustafaSaber]

Hi all, latest releases create Ryuk the reaper in a more consistent manner. I develop on Mac and I do not see issues with it. I'd love to check with you if the most recent version fixes this issue for you 🙏

Hi, thanks for reaching out 🙏! Currently we can't test this. However, it would be nice to see this working with no problems so when we can pick it up we will do.