Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GitHub Actions e2e tests #3320

Merged
merged 1 commit into from
Nov 17, 2021
Merged

GitHub Actions e2e tests #3320

merged 1 commit into from
Nov 17, 2021

Conversation

crazy-max
Copy link
Member

@crazy-max crazy-max commented Oct 1, 2021

Follow-up #3234

Switch To GitHub Actions for e2e tests:

  • Jenkinsfile not needed anymore
  • Mutualize Dockerfile to build plugins and e2e docker image
  • Mutualize variables for binary and plugins scripts
  • Fix issue with GO_STRIP var
  • Add bake target for plugins and e2e-image
  • plugins-windows and plugins-osx merged to plugins script
  • Add plugins job in our GHA workflow
  • Add matrix to test against alpine and buster base variants

Also use 20.10-dind instead of stable-dind because it has not been updated for 10 months (19.03.15).

Signed-off-by: CrazyMax crazy-max@users.noreply.github.com

@crazy-max crazy-max force-pushed the gha-e2e branch 2 times, most recently from 1540ef7 to f36c075 Compare October 1, 2021 13:08
@codecov-commenter
Copy link

codecov-commenter commented Oct 1, 2021

Codecov Report

Merging #3320 (06f6a2d) into master (b0343d9) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3320   +/-   ##
=======================================
  Coverage   58.02%   58.02%           
=======================================
  Files         302      302           
  Lines       21761    21761           
=======================================
  Hits        12626    12626           
  Misses       8214     8214           
  Partials      921      921           

@crazy-max crazy-max force-pushed the gha-e2e branch 4 times, most recently from b7135b4 to cb6ba55 Compare October 1, 2021 15:19
@crazy-max
Copy link
Member Author

crazy-max commented Oct 1, 2021

There seems to be some issues with the tests while using the 20.10-dind image: https://github.com/docker/cli/runs/3768855942?check_suite_focus=true


✓  e2e/cli-plugins (2.854s)
∅  e2e/cli-plugins/plugins/badmeta
∅  e2e/cli-plugins/plugins/nopersistentprerun
✓  e2e/context (169ms)
✓  e2e/global (95ms)
✓  e2e/container (9.263s)
✖  e2e/image (14.005s)
∅  e2e/internal/fixtures
✖  e2e/plugin (7.974s)
∅  e2e/plugin/basic
✖  e2e/system (44.249s)
✖  e2e/stack (1m5.859s)
✖  e2e/trust (36.601s)

=== Skipped
=== SKIP: e2e/cli-plugins TestCLIPluginDialStdio (0.00s)
    dial_test.go:16: skipping plugin dial-stdio test since DOCKER_CLI_PLUGIN_USE_DIAL_STDIO is not set

=== SKIP: e2e/container TestRunWithCgroupNamespace (0.07s)
    run_test.go:141: !cgroupNsFound: running against a daemon that doesn't support cgroup namespaces (security options: [name=apparmor name=seccomp,profile=default]
        )

=== SKIP: e2e/image TestBuildIidFileSquash (0.06s)
    build_test.go:112: !experimentalBuild: running against a non-experimental daemon

=== SKIP: e2e/stack TestDeployWithNamedResources/Kubernetes (0.00s)
    deploy_test.go:22: FIXME(chris-crone): currently does not work with compose for kubernetes.
    --- SKIP: TestDeployWithNamedResources/Kubernetes (0.00s)

=== SKIP: e2e/stack TestRemove/Kubernetes (0.00s)
    remove_test.go:22: !environment.KubernetesEnabled()
    --- SKIP: TestRemove/Kubernetes (0.00s)

=== Failed
=== FAIL: e2e/image TestPullWithContentTrust (0.75s)
    pull_test.go:34: assertion failed: 
        --- expected
        +++ actual
        @@ -1,4 +1,4 @@
         Pull (1 of 1): registry:5000/trust-pull:latest@sha256:641b95ddb2ea9dc2af1a0113b6b348ebc20872ba615204fbe12148e98fd6f23d
        -sha256:641b95ddb2ea9dc2af1a0113b6b348ebc20872ba615204fbe12148e98fd6f23d: Pulling from trust-pull
        +registry:5000/trust-pull@sha256:641b95ddb2ea9dc2af1a0113b6b348ebc20872ba615204fbe12148e98fd6f23d: Pulling from trust-pull
         Digest: sha256:641b95ddb2ea9dc2af1a0113b6b348ebc20872ba615204fbe12148e98fd6f23d
         Status: Downloaded newer image for registry:5000/trust-pull@sha256:641b95ddb2ea9dc2af1a0113b6b348ebc20872ba615204fbe12148e98fd6f23d
        
        
        You can run 'go test . -test.update-golden' to automatically update testdata/pull-with-content-trust.golden to the new expected value.'
        

=== FAIL: e2e/plugin TestInstallWithContentTrust (7.82s)
    trust_test.go:52: assertion failed: 
        Command:  docker plugin install --grant-all-permissions registry:5000/plugin-content-trust
        ExitCode: 1
        Error:    exit status 1
        Stdout:   
        Stderr:   Cannot connect to the Docker daemon at tcp://172.18.0.5:2375. Is the docker daemon running?
        
        
        Failures:
        ExitCode was 1 expected 0
        Expected stdout to contain "Status: Downloaded newer image for registry:5000/plugin-content-trust@sha"
        Expected no error

=== FAIL: e2e/stack TestDeployWithNamedResources/Swarm (53.56s)
    deploy_test.go:38: assertion failed: 
        Command:  docker stack deploy -c testdata/stack-with-named-resources.yml test-stack-deploy-with-names-swarm --orchestrator swarm
        ExitCode: 1
        Error:    exit status 1
        Stdout:   
        Stderr:   error during connect: Get "http://172.18.0.5:2375/v1.24/info": dial tcp 172.18.0.5:2375: connect: no route to host
        
        
        Failures:
        ExitCode was 1 expected 0
        Expected no error
    --- FAIL: TestDeployWithNamedResources/Swarm (53.56s)

=== FAIL: e2e/stack TestDeployWithNamedResources (53.56s)

=== FAIL: e2e/stack TestRemove/Swarm (8.23s)
    remove_test.go:43: assertion failed: 
        Command:  docker stack deploy --compose-file=./testdata/full-stack.yml test-stack-remove-swarm --orchestrator swarm
        ExitCode: 1
        Error:    exit status 1
        Stdout:   
        Stderr:   error during connect: Get "http://172.18.0.5:2375/v1.24/info": dial tcp 172.18.0.5:2375: connect: no route to host
        
        
        Failures:
        ExitCode was 1 expected 0
        Expected no error
    --- FAIL: TestRemove/Swarm (8.23s)

=== FAIL: e2e/stack TestRemove (8.23s)

=== FAIL: e2e/system TestInspectInvalidReference (44.25s)
    inspect_test.go:13: assertion failed: 
        Command:  docker inspect FooBar
        ExitCode: 1
        Error:    exit status 1
        Stdout:   []
        
        Stderr:   error during connect: Get "http://172.18.0.5:2375/v1.24/containers/FooBar/json": dial tcp 172.18.0.5:2375: connect: no route to host
        
        
        Failures:
        Expected stderr to contain "Error: No such object: FooBar"

=== FAIL: e2e/trust TestRevokeImage (8.95s)
    revoke_test.go:50: assertion failed: 
        Command:  docker pull registry:5000/alpine:3.6
        ExitCode: 1
        Error:    exit status 1
        Stdout:   
        Stderr:   error during connect: Post "http://172.18.0.5:2375/v1.24/images/create?fromImage=registry%3A5000%2Falpine&tag=3.6": dial tcp 172.18.0.5:2375: connect: no route to host
        
        
        Failures:
        ExitCode was 1 expected 0
        Expected no error

=== FAIL: e2e/trust TestRevokeRepo (9.22s)
    revoke_test.go:59: assertion failed: 
        Command:  docker pull registry:5000/alpine:3.6
        ExitCode: 1
        Error:    exit status 1
        Stdout:   
        Stderr:   error during connect: Post "http://172.18.0.5:2375/v1.24/images/create?fromImage=registry%3A5000%2Falpine&tag=3.6": dial tcp 172.18.0.5:2375: connect: no route to host
        
        
        Failures:
        ExitCode was 1 expected 0
        Expected no error

=== FAIL: e2e/trust TestSignLocalImage (9.22s)
    sign_test.go:26: assertion failed: 
        Command:  docker pull registry:5000/alpine:3.6
        ExitCode: 1
        Error:    exit status 1
        Stdout:   
        Stderr:   error during connect: Post "http://172.18.0.5:2375/v1.24/images/create?fromImage=registry%3A5000%2Falpine&tag=3.6": dial tcp 172.18.0.5:2375: connect: no route to host
        
        
        Failures:
        ExitCode was 1 expected 0
        Expected no error

=== FAIL: e2e/trust TestSignWithLocalFlag (9.22s)
    sign_test.go:51: assertion failed: 
        Command:  docker pull registry:5000/alpine:3.6
        ExitCode: 1
        Error:    exit status 1
        Stdout:   
        Stderr:   error during connect: Post "http://172.18.0.5:2375/v1.24/images/create?fromImage=registry%3A5000%2Falpine&tag=3.6": dial tcp 172.18.0.5:2375: connect: no route to host
        
        
        Failures:
        ExitCode was 1 expected 0
        Expected no error

Will try a replay to see if it's something on the GitHub Runner. Otherwise will revert to use stable-dind again but we should fix that in a follow-up (cc @thaJeztah)

Edit: Revert to stable-dind. Tests are broken using the 20.10-dind image.

@crazy-max crazy-max force-pushed the gha-e2e branch 6 times, most recently from 96f76f8 to ea7ff41 Compare October 1, 2021 16:02
@crazy-max
Copy link
Member Author

Not related to these changes but cli/command/container TestRemoveForce/without_force looks flawky: https://app.circleci.com/pipelines/github/docker/cli/1687/workflows/2c09f02d-cc28-4602-aa1f-0ad4baa14591/jobs/37128

image

@crazy-max crazy-max marked this pull request as ready for review October 1, 2021 16:08
@crazy-max
Copy link
Member Author

PTAL @thaJeztah @silvin-lubecki

@crazy-max crazy-max force-pushed the gha-e2e branch 9 times, most recently from d989020 to f4f9644 Compare October 4, 2021 12:18
@crazy-max crazy-max force-pushed the gha-e2e branch 4 times, most recently from 77afbca to cd1fd87 Compare October 12, 2021 14:13
Dockerfile Outdated Show resolved Hide resolved
Dockerfile Outdated Show resolved Hide resolved
Comment on lines +85 to +86
RUN curl -fsSL https://github.com/docker/compose/releases/download/${COMPOSE_VERSION}/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose && \
chmod +x /usr/local/bin/docker-compose
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a pity we need the uname -m here, but we can probably get rid of that if (in a follow-up) we switch to compose v2

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep looking forward for that :)

Comment on lines -37 to +28
TARGET="${TARGET}.exe"
TARGET="${TARGET}.exe"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like #2993 messed up formatting of out scripts and changed tabs to spaces for indentation 😞

We should probably fix that again (perhaps add a shfmt check somewhere https://github.com/moby/moby/blob/d9e3d383f56119b0ba64ed63c6d5a265fbf35f7d/Dockerfile#L204-L209

https://github.com/moby/moby/blob/a77317882d010b884a9101c6ad0b2d7db141082f/hack/validate/shfmt#L4-L13

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes indeed, we should add shfmt in a follow-up, good idea.

scripts/build/plugins Outdated Show resolved Hide resolved
scripts/build/plugins Outdated Show resolved Hide resolved
scripts/build/.variables Outdated Show resolved Hide resolved
Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

left one minor nit (but let me know if you prefer doing in a follow-up)

Dockerfile Outdated Show resolved Hide resolved
@crazy-max crazy-max force-pushed the gha-e2e branch 2 times, most recently from 06f6a2d to 2e59cdb Compare November 17, 2021 16:16
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@thaJeztah thaJeztah added this to the 21.xx milestone Nov 17, 2021
@thaJeztah thaJeztah merged commit 9bc104e into docker:master Nov 17, 2021
@crazy-max crazy-max deleted the gha-e2e branch November 17, 2021 16:56
@crazy-max
Copy link
Member Author

crazy-max commented Jan 19, 2022

@thaJeztah Was trying to solve issues with the 20.10-dind image and it's worse than I thought. The dind container exits unexpectedly:

time="2022-01-19T19:27:41.161203900Z" level=info msg="Starting up"
time="2022-01-19T19:27:41.163008100Z" level=warning msg="could not change group /var/run/docker.sock to docker: group docker not found"
time="2022-01-19T19:27:41.163199400Z" level=warning msg="Binding to IP address without --tlsverify is insecure and gives root access on this machine to everyone who has access to your network." host="tcp://0.0.0.0:2375"
time="2022-01-19T19:27:41.163238900Z" level=warning msg="Binding to an IP address, even on localhost, can also give access to scripts run in a browser. Be safe out there!" host="tcp://0.0.0.0:2375"
time="2022-01-19T19:27:42.163402700Z" level=warning msg="Binding to an IP address without --tlsverify is deprecated. Startup is intentionally being slowed down to show this message" host="tcp://0.0.0.0:2375"
time="2022-01-19T19:27:42.163468000Z" level=warning msg="Please consider generating tls certificates with client validation to prevent exposing unauthenticated root access to your network" host="tcp://0.0.0.0:2375"   
time="2022-01-19T19:27:42.163478700Z" level=warning msg="You can override this by explicitly specifying '--tls=false' or '--tlsverify=false'" host="tcp://0.0.0.0:2375"
time="2022-01-19T19:27:42.163484700Z" level=warning msg="Support for listening on TCP without authentication or explicit intent to run without authentication will be removed in the next release" host="tcp://0.0.0.0:2375"
time="2022-01-19T19:27:57.173677900Z" level=info msg="libcontainerd: started new containerd process" pid=37
time="2022-01-19T19:27:57.174300100Z" level=info msg="parsed scheme: \"unix\"" module=grpc
time="2022-01-19T19:27:57.174345100Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
time="2022-01-19T19:27:57.174373500Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}" module=grpc
time="2022-01-19T19:27:57.174386000Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
time="2022-01-19T19:27:57.259185100Z" level=info msg="starting containerd" revision=7b11cfaabd73bb80907dd23182b9347b4245eb5d version=v1.4.12
time="2022-01-19T19:27:57.281041300Z" level=info msg="loading plugin \"io.containerd.content.v1.content\"..." type=io.containerd.content.v1
time="2022-01-19T19:27:57.281373500Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.aufs\"..." type=io.containerd.snapshotter.v1
time="2022-01-19T19:27:57.285670600Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.aufs\"..." error="aufs is not supported (modprobe aufs failed: exit status 1 \"ip: can't find device 'aufs'\\nmodprobe: can't change directory to '/lib/modules': No such file or directory\\n\"): skip plugin" type=io.containerd.snapshotter.v1
time="2022-01-19T19:27:57.285712200Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.btrfs\"..." type=io.containerd.snapshotter.v1
time="2022-01-19T19:27:57.286047500Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.btrfs\"..." error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs (ext4) must be a btrfs filesystem to be used with the btrfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
time="2022-01-19T19:27:57.286089100Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.devmapper\"..." type=io.containerd.snapshotter.v1
time="2022-01-19T19:27:57.286503900Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.devmapper" error="devmapper not configured"
time="2022-01-19T19:27:57.286534800Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.native\"..." type=io.containerd.snapshotter.v1
time="2022-01-19T19:27:57.286704900Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.overlayfs\"..." type=io.containerd.snapshotter.v1
time="2022-01-19T19:27:57.287148000Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.zfs\"..." type=io.containerd.snapshotter.v1
time="2022-01-19T19:27:57.287729700Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.zfs\"..." error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
time="2022-01-19T19:27:57.287761200Z" level=info msg="loading plugin \"io.containerd.metadata.v1.bolt\"..." type=io.containerd.metadata.v1
time="2022-01-19T19:27:57.287846200Z" level=warning msg="could not use snapshotter devmapper in metadata plugin" error="devmapper not configured"
time="2022-01-19T19:27:57.287885400Z" level=info msg="metadata content store policy set" policy=shared
time="2022-01-19T19:27:57.301396500Z" level=info msg="loading plugin \"io.containerd.differ.v1.walking\"..." type=io.containerd.differ.v1
time="2022-01-19T19:27:57.301447400Z" level=info msg="loading plugin \"io.containerd.gc.v1.scheduler\"..." type=io.containerd.gc.v1
time="2022-01-19T19:27:57.301738900Z" level=info msg="loading plugin \"io.containerd.service.v1.introspection-service\"..." type=io.containerd.service.v1
time="2022-01-19T19:27:57.301798100Z" level=info msg="loading plugin \"io.containerd.service.v1.containers-service\"..." type=io.containerd.service.v1
time="2022-01-19T19:27:57.301811700Z" level=info msg="loading plugin \"io.containerd.service.v1.content-service\"..." type=io.containerd.service.v1
time="2022-01-19T19:27:57.301825200Z" level=info msg="loading plugin \"io.containerd.service.v1.diff-service\"..." type=io.containerd.service.v1
time="2022-01-19T19:27:57.301840900Z" level=info msg="loading plugin \"io.containerd.service.v1.images-service\"..." type=io.containerd.service.v1
time="2022-01-19T19:27:57.301875400Z" level=info msg="loading plugin \"io.containerd.service.v1.leases-service\"..." type=io.containerd.service.v1
time="2022-01-19T19:27:57.301896900Z" level=info msg="loading plugin \"io.containerd.service.v1.namespaces-service\"..." type=io.containerd.service.v1
time="2022-01-19T19:27:57.301913600Z" level=info msg="loading plugin \"io.containerd.service.v1.snapshots-service\"..." type=io.containerd.service.v1
time="2022-01-19T19:27:57.301928700Z" level=info msg="loading plugin \"io.containerd.runtime.v1.linux\"..." type=io.containerd.runtime.v1
time="2022-01-19T19:27:57.302190000Z" level=info msg="loading plugin \"io.containerd.runtime.v2.task\"..." type=io.containerd.runtime.v2
time="2022-01-19T19:27:57.302405900Z" level=info msg="loading plugin \"io.containerd.monitor.v1.cgroups\"..." type=io.containerd.monitor.v1
time="2022-01-19T19:27:57.303221700Z" level=info msg="loading plugin \"io.containerd.service.v1.tasks-service\"..." type=io.containerd.service.v1
time="2022-01-19T19:27:57.303285000Z" level=info msg="loading plugin \"io.containerd.internal.v1.restart\"..." type=io.containerd.internal.v1
time="2022-01-19T19:27:57.303607000Z" level=info msg="loading plugin \"io.containerd.grpc.v1.containers\"..." type=io.containerd.grpc.v1
time="2022-01-19T19:27:57.303639700Z" level=info msg="loading plugin \"io.containerd.grpc.v1.content\"..." type=io.containerd.grpc.v1
time="2022-01-19T19:27:57.303653600Z" level=info msg="loading plugin \"io.containerd.grpc.v1.diff\"..." type=io.containerd.grpc.v1
time="2022-01-19T19:27:57.303668400Z" level=info msg="loading plugin \"io.containerd.grpc.v1.events\"..." type=io.containerd.grpc.v1
time="2022-01-19T19:27:57.303688900Z" level=info msg="loading plugin \"io.containerd.grpc.v1.healthcheck\"..." type=io.containerd.grpc.v1
time="2022-01-19T19:27:57.303722700Z" level=info msg="loading plugin \"io.containerd.grpc.v1.images\"..." type=io.containerd.grpc.v1
time="2022-01-19T19:27:57.303763900Z" level=info msg="loading plugin \"io.containerd.grpc.v1.leases\"..." type=io.containerd.grpc.v1
time="2022-01-19T19:27:57.303781600Z" level=info msg="loading plugin \"io.containerd.grpc.v1.namespaces\"..." type=io.containerd.grpc.v1
time="2022-01-19T19:27:57.303820000Z" level=info msg="loading plugin \"io.containerd.internal.v1.opt\"..." type=io.containerd.internal.v1
time="2022-01-19T19:27:57.304181400Z" level=info msg="loading plugin \"io.containerd.grpc.v1.snapshots\"..." type=io.containerd.grpc.v1
time="2022-01-19T19:27:57.304224700Z" level=info msg="loading plugin \"io.containerd.grpc.v1.tasks\"..." type=io.containerd.grpc.v1
time="2022-01-19T19:27:57.304239700Z" level=info msg="loading plugin \"io.containerd.grpc.v1.version\"..." type=io.containerd.grpc.v1
time="2022-01-19T19:27:57.304248900Z" level=info msg="loading plugin \"io.containerd.grpc.v1.introspection\"..." type=io.containerd.grpc.v1
time="2022-01-19T19:27:57.304480500Z" level=info msg=serving... address=/var/run/docker/containerd/containerd-debug.sock
time="2022-01-19T19:27:57.304566600Z" level=info msg=serving... address=/var/run/docker/containerd/containerd.sock.ttrpc
time="2022-01-19T19:27:57.304660200Z" level=info msg=serving... address=/var/run/docker/containerd/containerd.sock
time="2022-01-19T19:27:57.304696000Z" level=info msg="containerd successfully booted in 0.046710s"
time="2022-01-19T19:27:57.316810300Z" level=info msg="parsed scheme: \"unix\"" module=grpc
time="2022-01-19T19:27:57.316865200Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
time="2022-01-19T19:27:57.316889100Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}" module=grpc
time="2022-01-19T19:27:57.316900400Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
time="2022-01-19T19:27:57.318650400Z" level=info msg="parsed scheme: \"unix\"" module=grpc
time="2022-01-19T19:27:57.318693100Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
time="2022-01-19T19:27:57.318735500Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}" module=grpc
time="2022-01-19T19:27:57.318748600Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
time="2022-01-19T19:27:57.370154600Z" level=warning msg="Your kernel does not support cgroup blkio weight"
time="2022-01-19T19:27:57.370204300Z" level=warning msg="Your kernel does not support cgroup blkio weight_device"
time="2022-01-19T19:27:57.370210500Z" level=warning msg="Your kernel does not support cgroup blkio throttle.read_bps_device"
time="2022-01-19T19:27:57.370214800Z" level=warning msg="Your kernel does not support cgroup blkio throttle.write_bps_device"
time="2022-01-19T19:27:57.370219000Z" level=warning msg="Your kernel does not support cgroup blkio throttle.read_iops_device"
time="2022-01-19T19:27:57.370223100Z" level=warning msg="Your kernel does not support cgroup blkio throttle.write_iops_device"
time="2022-01-19T19:27:57.370828700Z" level=info msg="Loading containers: start."
time="2022-01-19T19:27:57.456401200Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address"
time="2022-01-19T19:27:57.503150000Z" level=info msg="Loading containers: done."
time="2022-01-19T19:27:57.555714600Z" level=info msg="Docker daemon" commit=459d0df graphdriver(s)=overlay2 version=20.10.12
time="2022-01-19T19:27:57.556311700Z" level=info msg="Daemon has completed initialization"
time="2022-01-19T19:27:57.617397200Z" level=info msg="API listen on /var/run/docker.sock"
time="2022-01-19T19:27:57.620852400Z" level=info msg="API listen on [::]:2375"
time="2022-01-19T19:27:58.204323700Z" level=warning msg="reference for unknown type: " digest="sha256:f006ecbb824d87947d0b51ab8488634bf69fe4094959d935c0c103f4820a417d" remote="docker.io/library/alpine@sha256:f006ecbb824d87947d0b51ab8488634bf69fe4094959d935c0c103f4820a417d"
time="2022-01-19T19:28:00.466009700Z" level=info msg="Attempting next endpoint for push after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:01.288249100Z" level=warning msg="reference for unknown type: " digest="sha256:3e8fa85ddfef1af9ca85a5cfb714148956984e02f00bec3f7f49d3925a91e0e7" remote="docker.io/library/busybox@sha256:3e8fa85ddfef1af9ca85a5cfb714148956984e02f00bec3f7f49d3925a91e0e7"
time="2022-01-19T19:28:03.445871000Z" level=info msg="Attempting next endpoint for push after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:03.718483600Z" level=info msg="parsed scheme: \"\"" module=grpc
time="2022-01-19T19:28:03.718520500Z" level=info msg="scheme \"\" not registered, fallback to default scheme" module=grpc
time="2022-01-19T19:28:03.718565500Z" level=info msg="ccResolverWrapper: sending update to cc: {[{/var/run/docker/swarm/control.sock  <nil> 0 <nil>}] <nil> <nil>}" module=grpc
time="2022-01-19T19:28:03.718573200Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
time="2022-01-19T19:28:03.721531500Z" level=info msg="Listening for connections" addr="[::]:2377" module=node node.id=3t6u9oxb2s4e2x2zwknxhcifd proto=tcp
time="2022-01-19T19:28:03.721615900Z" level=info msg="Listening for local connections" addr=/var/run/docker/swarm/control.sock module=node node.id=3t6u9oxb2s4e2x2zwknxhcifd proto=unix
time="2022-01-19T19:28:03.731287900Z" level=info msg="2936efb334e3f9a9 became follower at term 0" module=raft node.id=3t6u9oxb2s4e2x2zwknxhcifd
time="2022-01-19T19:28:03.731335200Z" level=info msg="newRaft 2936efb334e3f9a9 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]" module=raft node.id=3t6u9oxb2s4e2x2zwknxhcifd
time="2022-01-19T19:28:03.731346600Z" level=info msg="2936efb334e3f9a9 became follower at term 1" module=raft node.id=3t6u9oxb2s4e2x2zwknxhcifd
time="2022-01-19T19:28:03.734713600Z" level=info msg="2936efb334e3f9a9 is starting a new election at term 1" module=raft node.id=3t6u9oxb2s4e2x2zwknxhcifd
time="2022-01-19T19:28:03.734800100Z" level=info msg="2936efb334e3f9a9 became candidate at term 2" module=raft node.id=3t6u9oxb2s4e2x2zwknxhcifd
time="2022-01-19T19:28:03.734812900Z" level=info msg="2936efb334e3f9a9 received MsgVoteResp from 2936efb334e3f9a9 at term 2" module=raft node.id=3t6u9oxb2s4e2x2zwknxhcifd
time="2022-01-19T19:28:03.734822500Z" level=info msg="2936efb334e3f9a9 became leader at term 2" module=raft node.id=3t6u9oxb2s4e2x2zwknxhcifd
time="2022-01-19T19:28:03.734828200Z" level=info msg="raft.node: 2936efb334e3f9a9 elected leader 2936efb334e3f9a9 at term 2" module=raft node.id=3t6u9oxb2s4e2x2zwknxhcifd
time="2022-01-19T19:28:03.736981900Z" level=info msg="Creating default ingress network" module=node node.id=3t6u9oxb2s4e2x2zwknxhcifd
time="2022-01-19T19:28:03.740439300Z" level=info msg="leadership changed from not yet part of a raft cluster to 3t6u9oxb2s4e2x2zwknxhcifd" module=node node.id=3t6u9oxb2s4e2x2zwknxhcifd
time="2022-01-19T19:28:03.740590800Z" level=info msg="dispatcher starting" module=dispatcher node.id=3t6u9oxb2s4e2x2zwknxhcifd
time="2022-01-19T19:28:04.234797100Z" level=info msg="manager selected by agent for new session: { }" module=node/agent node.id=3t6u9oxb2s4e2x2zwknxhcifd
time="2022-01-19T19:28:04.235091800Z" level=info msg="waiting 0s before registering session" module=node/agent node.id=3t6u9oxb2s4e2x2zwknxhcifd
time="2022-01-19T19:28:04.254991000Z" level=info msg="worker 3t6u9oxb2s4e2x2zwknxhcifd was successfully registered" method="(*Dispatcher).register"
time="2022-01-19T19:28:04.259540600Z" level=info msg="initialized VXLAN UDP port to 4789 "
time="2022-01-19T19:28:04.259557600Z" level=info msg="Initializing Libnetwork Agent Listen-Addr=0.0.0.0 Local-addr=172.19.0.5 Adv-addr=172.19.0.5 Data-addr= Remote-addr-list=[] MTU=1500"
time="2022-01-19T19:28:04.259686900Z" level=info msg="New memberlist node - Node:3d41d38f3a17 will use memberlist nodeID:281ae7b58355 with config:&{NodeID:281ae7b58355 Hostname:3d41d38f3a17 BindAddr:0.0.0.0 AdvertiseAddr:172.19.0.5 BindPort:0 Keys:[[168 182 16 114 219 250 123 222 125 206 222 121 190 220 64 94] [21 71 103 78 178 2 158 123 20 48 37 92 224 226 101 92] [156 141 78 19 139 165 134 155 154 115 165 63 135 133 8 63]] PacketBufferSize:1400 reapEntryInterval:1800000000000 reapNetworkInterval:1825000000000 StatsPrintPeriod:5m0s HealthPrintPeriod:1m0s}"
time="2022-01-19T19:28:04.261505200Z" level=info msg="Node 281ae7b58355/172.19.0.5, joined gossip cluster"
time="2022-01-19T19:28:04.262754800Z" level=info msg="Node 281ae7b58355/172.19.0.5, added to nodes list"
time="2022-01-19T19:28:04.264458000Z" level=error msg="error reading the kernel parameter net.ipv4.neigh.default.gc_thresh1" error="open /proc/sys/net/ipv4/neigh/default/gc_thresh1: no such file or directory"
time="2022-01-19T19:28:04.264548000Z" level=error msg="error reading the kernel parameter net.ipv4.neigh.default.gc_thresh2" error="open /proc/sys/net/ipv4/neigh/default/gc_thresh2: no such file or directory"
time="2022-01-19T19:28:04.264567300Z" level=error msg="error reading the kernel parameter net.ipv4.neigh.default.gc_thresh3" error="open /proc/sys/net/ipv4/neigh/default/gc_thresh3: no such file or directory"
time="2022-01-19T19:28:04.270959900Z" level=info msg="No non-localhost DNS nameservers are left in resolv.conf. Using default external servers: [nameserver 8.8.8.8 nameserver 8.8.4.4]"
time="2022-01-19T19:28:04.271007300Z" level=info msg="IPv6 enabled; Adding default IPv6 external servers: [nameserver 2001:4860:4860::8888 nameserver 2001:4860:4860::8844]"
time="2022-01-19T19:28:17.252427300Z" level=info msg="No non-localhost DNS nameservers are left in resolv.conf. Using default external servers: [nameserver 8.8.8.8 nameserver 8.8.4.4]"
time="2022-01-19T19:28:17.252460600Z" level=info msg="IPv6 enabled; Adding default IPv6 external servers: [nameserver 2001:4860:4860::8888 nameserver 2001:4860:4860::8844]"
time="2022-01-19T19:28:17.450175000Z" level=error msg="Error getting service FooBar: service FooBar not found"
time="2022-01-19T19:28:17.451925200Z" level=error msg="Error getting task FooBar: task FooBar not found"
time="2022-01-19T19:28:17.452323800Z" level=warning msg="Error getting v2 registry: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:17.452626700Z" level=info msg="Attempting next endpoint for pull after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:17.454292600Z" level=error msg="Error getting node FooBar: node FooBar not found"
time="2022-01-19T19:28:17.474737300Z" level=info msg="starting signal loop" namespace=moby path=/run/docker/containerd/daemon/io.containerd.runtime.v2.task/moby/48a80af74d0fa5f903358ea4b712e05ae7340e7e2b31ef62e029c6ad984e7f92 pid=389
time="2022-01-19T19:28:17.505786400Z" level=info msg="initialized VXLAN UDP port to 4789 "
time="2022-01-19T19:28:17.539276100Z" level=error msg="Failed to allocate network resources for node 3t6u9oxb2s4e2x2zwknxhcifd" error="could not find network allocator state for network gm9zjxzyh9r6wotjppdfdbqwn" module=node node.id=3t6u9oxb2s4e2x2zwknxhcifd
time="2022-01-19T19:28:17.543821500Z" level=info msg="initialized VXLAN UDP port to 4789 "
time="2022-01-19T19:28:17.552099600Z" level=error msg="Failed to allocate network resources for node 3t6u9oxb2s4e2x2zwknxhcifd" error="could not find network allocator state for network walp8d5cqxx2hunt3d25w9eer" module=node node.id=3t6u9oxb2s4e2x2zwknxhcifd
time="2022-01-19T19:28:17.552134300Z" level=error msg="Failed to allocate network resources for node 3t6u9oxb2s4e2x2zwknxhcifd" error="could not find network allocator state for network gm9zjxzyh9r6wotjppdfdbqwn" module=node node.id=3t6u9oxb2s4e2x2zwknxhcifd
time="2022-01-19T19:28:17.552799000Z" level=info msg="initialized VXLAN UDP port to 4789 "
time="2022-01-19T19:28:17.570609300Z" level=info msg="No non-localhost DNS nameservers are left in resolv.conf. Using default external servers: [nameserver 8.8.8.8 nameserver 8.8.4.4]"
time="2022-01-19T19:28:17.570658500Z" level=info msg="IPv6 enabled; Adding default IPv6 external servers: [nameserver 2001:4860:4860::8888 nameserver 2001:4860:4860::8844]"
time="2022-01-19T19:28:17.597696100Z" level=error msg="controller resolution failed" error="invalid mount target, must be an absolute path: volume1" module=node/agent node.id=3t6u9oxb2s4e2x2zwknxhcifd service.id=d6dfg3ga2t2zvm4ngqg9xjf9m task.id=mkdvus6i37a01iwb9kg2q8t9y
time="2022-01-19T19:28:17.597752900Z" level=error msg="failed to start taskManager" error="invalid mount target, must be an absolute path: volume1" module=node/agent node.id=3t6u9oxb2s4e2x2zwknxhcifd
time="2022-01-19T19:28:17.695447600Z" level=info msg="starting signal loop" namespace=moby path=/run/docker/containerd/daemon/io.containerd.runtime.v2.task/moby/f7e4c3e021458a3c3ed025ed45bd9474421d5c6f3a06b9c675478abe0c85bb5a pid=441
time="2022-01-19T19:28:18.007280000Z" level=info msg="initialized VXLAN UDP port to 4789 "
time="2022-01-19T19:28:18.217923000Z" level=error msg="Failed to allocate network resources for node 3t6u9oxb2s4e2x2zwknxhcifd" error="could not find network allocator state for network ri5mccebtjxc6hwqcocmvopgh" module=node node.id=3t6u9oxb2s4e2x2zwknxhcifd
time="2022-01-19T19:28:18.219036500Z" level=info msg="initialized VXLAN UDP port to 4789 "
time="2022-01-19T19:28:18.223356900Z" level=info msg="No non-localhost DNS nameservers are left in resolv.conf. Using default external servers: [nameserver 8.8.8.8 nameserver 8.8.4.4]"
time="2022-01-19T19:28:18.223427200Z" level=info msg="IPv6 enabled; Adding default IPv6 external servers: [nameserver 2001:4860:4860::8888 nameserver 2001:4860:4860::8844]"
time="2022-01-19T19:28:18.289064800Z" level=info msg="ignoring event" container=48a80af74d0fa5f903358ea4b712e05ae7340e7e2b31ef62e029c6ad984e7f92 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
time="2022-01-19T19:28:18.292145100Z" level=info msg="shim disconnected" id=48a80af74d0fa5f903358ea4b712e05ae7340e7e2b31ef62e029c6ad984e7f92
time="2022-01-19T19:28:18.292867100Z" level=error msg="copy shim log" error="read /proc/self/fd/14: file already closed"
time="2022-01-19T19:28:18.296170500Z" level=error msg="Failed to allocate network resources for node 3t6u9oxb2s4e2x2zwknxhcifd" error="could not find network allocator state for network ri5mccebtjxc6hwqcocmvopgh" module=node node.id=3t6u9oxb2s4e2x2zwknxhcifd
time="2022-01-19T19:28:18.296883700Z" level=error msg="Failed to allocate network resources for node 3t6u9oxb2s4e2x2zwknxhcifd" error="could not find network allocator state for network ri5mccebtjxc6hwqcocmvopgh" module=node node.id=3t6u9oxb2s4e2x2zwknxhcifd
time="2022-01-19T19:28:18.392884300Z" level=error msg="Handler for DELETE /v1.41/networks/ri5mccebtjxc6hwqcocmvopgh returned error: network ri5mccebtjxc6hwqcocmvopgh not found"
time="2022-01-19T19:28:18.397311000Z" level=info msg="Attempting next endpoint for push after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:18.557187100Z" level=info msg="shim disconnected" id=f7e4c3e021458a3c3ed025ed45bd9474421d5c6f3a06b9c675478abe0c85bb5a
time="2022-01-19T19:28:18.557773700Z" level=error msg="copy shim log" error="read /proc/self/fd/16: file already closed"
time="2022-01-19T19:28:18.560500700Z" level=info msg="ignoring event" container=f7e4c3e021458a3c3ed025ed45bd9474421d5c6f3a06b9c675478abe0c85bb5a module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
time="2022-01-19T19:28:19.049929100Z" level=warning msg="Error getting v2 registry: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:19.049979700Z" level=info msg="Attempting next endpoint for pull after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:19.196887900Z" level=warning msg="Error getting v2 registry: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:19.196964100Z" level=info msg="Attempting next endpoint for pull after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:19.235613800Z" level=info msg="Layer sha256:219cda422ba5473cd05ade2198ce77b05004a239f3e2998dfcf97800678a6a0e cleaned up"
time="2022-01-19T19:28:19.242049900Z" level=info msg="Attempting next endpoint for push after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:19.586178900Z" level=info msg="Attempting next endpoint for push after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:19.598971000Z" level=warning msg="Error getting v2 registry: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:19.599020500Z" level=info msg="Attempting next endpoint for pull after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:19.718464200Z" level=error msg="Failed to allocate network resources for node 3t6u9oxb2s4e2x2zwknxhcifd" error="could not find network allocator state for network ri5mccebtjxc6hwqcocmvopgh" module=node node.id=3t6u9oxb2s4e2x2zwknxhcifd
time="2022-01-19T19:28:19.718569800Z" level=error msg="Failed to allocate network resources for node 3t6u9oxb2s4e2x2zwknxhcifd" error="could not find network allocator state for network ri5mccebtjxc6hwqcocmvopgh" module=node node.id=3t6u9oxb2s4e2x2zwknxhcifd
time="2022-01-19T19:28:19.753020100Z" level=warning msg="Error getting v2 registry: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:19.753372500Z" level=info msg="Attempting next endpoint for pull after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:19.817758000Z" level=info msg="Attempting next endpoint for push after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:20.003128400Z" level=info msg="Attempting next endpoint for push after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:20.065811500Z" level=warning msg="Error getting v2 registry: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:20.065900500Z" level=info msg="Attempting next endpoint for pull after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:20.258587900Z" level=warning msg="Error getting v2 registry: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:20.258620700Z" level=info msg="Attempting next endpoint for pull after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:20.314041900Z" level=info msg="Attempting next endpoint for push after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:20.371563600Z" level=info msg="Attempting next endpoint for push after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:20.384497400Z" level=warning msg="Error getting v2 registry: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:20.384585800Z" level=info msg="Attempting next endpoint for pull after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:20.615823100Z" level=warning msg="Error getting v2 registry: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:20.615871200Z" level=info msg="Attempting next endpoint for pull after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:20.638261100Z" level=warning msg="Error getting v2 registry: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:20.638332200Z" level=info msg="Attempting next endpoint for pull after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:20.683401900Z" level=warning msg="failed to deactivate service binding for container test-stack-remove-swarm_two.1.e0e6guak18rn2un7zj1mab25g" error="No such container: test-stack-remove-swarm_two.1.e0e6guak18rn2un7zj1mab25g" module=node/agent/taskmanager node.id=3t6u9oxb2s4e2x2zwknxhcifd service.id=ss5oyvyc5motv43nrsv3drgv2 task.id=e0e6guak18rn2un7zj1mab25g
time="2022-01-19T19:28:20.688821900Z" level=warning msg="failed to deactivate service binding for container test-stack-remove-swarm_one.1.ffoloftoadi095qbmvb1l66b3" error="No such container: test-stack-remove-swarm_one.1.ffoloftoadi095qbmvb1l66b3" module=node/agent/taskmanager node.id=3t6u9oxb2s4e2x2zwknxhcifd service.id=z9v8wvjn5sgftoax6urr9vtsm task.id=ffoloftoadi095qbmvb1l66b3
time="2022-01-19T19:28:20.740380200Z" level=info msg="Attempting next endpoint for push after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:20.793254700Z" level=info msg="Attempting next endpoint for push after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:20.894150400Z" level=warning msg="Error getting v2 registry: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:20.894212100Z" level=info msg="Attempting next endpoint for pull after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:21.112810000Z" level=warning msg="Error getting v2 registry: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:21.112850900Z" level=info msg="Attempting next endpoint for pull after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:21.310453300Z" level=info msg="Attempting next endpoint for push after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:21.311103200Z" level=warning msg="Error getting v2 registry: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:21.311162100Z" level=info msg="Attempting next endpoint for pull after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:21.339005400Z" level=info msg="Attempting next endpoint for push after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:21.463890500Z" level=info msg="Attempting next endpoint for push after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:21.645577200Z" level=warning msg="Error getting v2 registry: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:21.645635300Z" level=info msg="Attempting next endpoint for pull after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:21.677984300Z" level=info msg="Attempting next endpoint for push after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:21.908201500Z" level=warning msg="Error getting v2 registry: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:21.908237300Z" level=info msg="Attempting next endpoint for pull after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:21.990443000Z" level=info msg="Attempting next endpoint for push after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:22.054451300Z" level=info msg="No non-localhost DNS nameservers are left in resolv.conf. Using default external servers: [nameserver 8.8.8.8 nameserver 8.8.4.4]"
time="2022-01-19T19:28:22.054497200Z" level=info msg="IPv6 enabled; Adding default IPv6 external servers: [nameserver 2001:4860:4860::8888 nameserver 2001:4860:4860::8844]"
time="2022-01-19T19:28:22.139224800Z" level=warning msg="Error getting v2 registry: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:22.139289100Z" level=info msg="Attempting next endpoint for pull after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:22.140123300Z" level=info msg="starting signal loop" namespace=moby path=/run/docker/containerd/daemon/io.containerd.runtime.v2.task/moby/390abf88b728db2251040fb6658f0b18def670f9b9931b960903981a4b7f477c pid=667
time="2022-01-19T19:28:22.278317400Z" level=warning msg="Error getting v2 registry: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:22.278375600Z" level=info msg="Attempting next endpoint for pull after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:22.424666900Z" level=info msg="Attempting next endpoint for push after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:22.509548900Z" level=info msg="Attempting next endpoint for push after error: Get \"https://registry:5000/v2/\": http: server gave HTTP response to HTTPS client"
time="2022-01-19T19:28:22.579220600Z" level=info msg="ignoring event" container=390abf88b728db2251040fb6658f0b18def670f9b9931b960903981a4b7f477c module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
time="2022-01-19T19:28:22.580198900Z" level=info msg="shim disconnected" id=390abf88b728db2251040fb6658f0b18def670f9b9931b960903981a4b7f477c
time="2022-01-19T19:28:22.580697400Z" level=error msg="copy shim log" error="read /proc/self/fd/14: file already closed"
time="2022-01-19T19:28:22.968206000Z" level=warning msg="reference for unknown type: application/vnd.docker.plugin.v1+json"
time="2022-01-19T19:28:22.968278200Z" level=warning msg="reference for unknown type: application/vnd.docker.plugin.v1+json" digest="sha256:159f82359582dfed8e869ee8f9621397ce3d0b189622e64519c061d7f09d4e55" mediatype=application/vnd.docker.plugin.v1+json size=527
time="2022-01-19T19:28:22.970282200Z" level=warning msg="reference for unknown type: application/vnd.docker.plugin.v1+json"
time="2022-01-19T19:28:22.970368000Z" level=warning msg="reference for unknown type: application/vnd.docker.plugin.v1+json" digest="sha256:159f82359582dfed8e869ee8f9621397ce3d0b189622e64519c061d7f09d4e55" mediatype=application/vnd.docker.plugin.v1+json size=527
panic: interface conversion: reference.canonicalReference is not reference.Tagged: missing method Tag

goroutine 3508 [running]:
github.com/docker/docker/plugin.withFetchProgress.func1(0x29be650, 0xc000f1f200, 0xc00206b240, 0x34, 0xc00090f5ab, 0x47, 0x1a2, 0x0, 0x0, 0x0, ...)
        /go/src/github.com/docker/docker/plugin/fetch_linux.go:206 +0x50b
github.com/docker/docker/vendor/github.com/containerd/containerd/images.HandlerFunc.Handle(0xc000f1f140, 0x29be650, 0xc000f1f200, 0xc00206b240, 0x34, 0xc00090f5ab, 0x47, 0x1a2, 0x0, 0x0, ...)
        /go/src/github.com/docker/docker/vendor/github.com/containerd/containerd/images/handlers.go:55 +0x77
github.com/docker/docker/vendor/github.com/containerd/containerd/images.Handlers.func1(0x29be650, 0xc000f1f200, 0xc00206b240, 0x34, 0xc00090f5ab, 0x47, 0x1a2, 0x0, 0x0, 0x0, ...)
        /go/src/github.com/docker/docker/vendor/github.com/containerd/containerd/images/handlers.go:65 +0xf8
github.com/docker/docker/vendor/github.com/containerd/containerd/images.HandlerFunc.Handle(0xc001631c20, 0x29be650, 0xc000f1f200, 0xc00206b240, 0x34, 0xc00090f5ab, 0x47, 0x1a2, 0x0, 0x0, ...)
        /go/src/github.com/docker/docker/vendor/github.com/containerd/containerd/images/handlers.go:55 +0x77
github.com/docker/docker/vendor/github.com/containerd/containerd/images.Dispatch.func1(0x2969a01, 0xc0021dceb0)
        /go/src/github.com/docker/docker/vendor/github.com/containerd/containerd/images/handlers.go:134 +0xd5
github.com/docker/docker/vendor/golang.org/x/sync/errgroup.(*Group).Go.func1(0xc001b316b0, 0xc002657d00)
        /go/src/github.com/docker/docker/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x5b
created by github.com/docker/docker/vendor/golang.org/x/sync/errgroup.(*Group).Go
        /go/src/github.com/docker/docker/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x68

Seems to occur in https://github.com/moby/moby/blob/459d0dfbbb51fb2423a43655e6c62368ec0f36c9/plugin/fetch_linux.go#L206. Any idea @cpuguy83?

Linked test

icmd.RunCommand("docker", "plugin", "create", pluginName, pluginDir.Path()).Assert(t, icmd.Success)

@thaJeztah
Copy link
Member

Reading from my phone, but I recall the "unknown type" error from moby/moby#42889 (haven't found the time to look into that one).

Looks like we also need to add the local registry as "insecure registry" (?)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants