Use cri-dockerd instead of dockershim where needed #12102

afbjorklund · 2021-08-03T11:21:39Z

Starting with Kubernetes version 1.24, the CRI socket
for Docker is handled externally instead of internally.

See https://www.mirantis.com/blog/the-future-of-dockershim-is-cri-dockerd/

I added a feature to start the socket/service, transparently to the casual user...

For the power user giving their own CRI socket, they get to maintain it as well.

Closes #12099

Note: needs cri-dockerd installed, for testing (see #12103).

Also tweak the version check, when to not use dockershim:

KubernetesVersion.GTE(semver.Version{Major: 1, Minor: 24})

i.e. If you want to try it out, before Kubernetes 1.24 is available.

EDIT: The dockershim removal was delayed again, from 1.23 to 1.24

Starting with Kubernetes version 1.23, the CRI socket for Docker is handled externally instead of internally.

The socket-activated service for cri-dockerd will start on demand, similar to how the docker service works (for /var/run/docker.sock)

It is cri-docker.sock, even for cri-dockerd deb

That is, reverse order from when starting up.

medyagh · 2021-08-05T23:53:33Z

/ok-to-test

medyagh · 2021-08-06T00:16:32Z

I am excited about this PR

afbjorklund · 2021-08-06T06:52:49Z

I tested with the alpha, and it doesn't work because minikube passes "0.0.0" for the kubernetes version.

        r, err := cruntime.New(cruntime.Config{Type: cfg.KubernetesConfig.ContainerRuntime, Runner: k.c})

But it still starts, because apparently Kubernetes v1.23.0-alpha.0 "forgot" to remove the dockershim...

Also found dockershim hard-coded in a lot of other places, but using the kubernetes constant instead...

afbjorklund · 2022-01-05T09:15:20Z

Need to change the k8s version (1.24.0-alpha.1), and need to change the cri socket path (cri-dockerd.sock).

/hold

afbjorklund · 2022-01-05T09:25:14Z

Related PR, for kubeadm:

kubeadm: adapt CRI detection and defaults after the dockershim removal kubernetes#107317

minikube-pr-bot · 2022-01-05T23:06:41Z

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 12102) |
+----------------+----------+---------------------+
| minikube start | 46.6s    | 46.1s               |
| enable ingress | 30.4s    | 30.4s               |
+----------------+----------+---------------------+

Times for minikube start: 49.8s 45.8s 45.0s 46.1s 46.5s
Times for minikube (PR 12102) start: 45.1s 47.7s 45.9s 46.2s 45.6s

Times for minikube ingress: 30.2s 29.7s 30.8s 30.2s 31.3s
Times for minikube (PR 12102) ingress: 30.2s 31.2s 30.3s 30.3s 29.8s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 12102) |
+----------------+----------+---------------------+
| minikube start | 22.1s    | 22.5s               |
| enable ingress | 27.6s    | 27.7s               |
+----------------+----------+---------------------+

Times for minikube start: 21.7s 21.6s 21.8s 23.0s 22.4s
Times for minikube (PR 12102) start: 21.6s 22.9s 21.5s 22.8s 23.4s

Times for minikube ingress: 32.9s 26.4s 25.9s 25.9s 26.9s
Times for minikube (PR 12102) ingress: 28.5s 26.4s 26.4s 30.4s 26.9s

docker driver with containerd runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 12102) |
+----------------+----------+---------------------+
| minikube start | 41.3s    | 41.7s               |
| enable ingress | 22.5s    | 32.5s               |
+----------------+----------+---------------------+

Times for minikube (PR 12102) start: 41.9s 41.8s 42.8s 42.0s 40.3s
Times for minikube start: 43.3s 42.6s 37.0s 41.9s 41.7s

Times for minikube ingress: 28.4s 18.4s 24.3s 18.9s 22.4s
Times for minikube (PR 12102) ingress: 53.4s 22.4s 32.4s 32.4s 21.9s

minikube-pr-bot · 2022-01-06T01:08:06Z

These are the flake rates of all failed tests.

Environment	Failed Tests	Flake Rate (%)
Hyper-V_Windows	TestNetworkPlugins/group/calico/Start (gopogh)	2.76 (chart)
Hyper-V_Windows	TestNetworkPlugins/group/custom-weave/Start (gopogh)	2.76 (chart)
Docker_macOS	TestJSONOutput/pause/Command (gopogh)	4.73 (chart)
Docker_Windows	TestForceSystemdEnv (gopogh)	9.76 (chart)
KVM_Linux	TestNetworkPlugins/group/flannel/Start (gopogh)	10.24 (chart)
Docker_macOS	TestNetworkPlugins/group/kubenet/DNS (gopogh)	39.02 (chart)
Docker_Windows	TestStartStop/group/embed-certs/serial/Pause (gopogh)	48.36 (chart)
Docker_macOS	TestNetworkPlugins/group/calico/Start (gopogh)	61.11 (chart)
Hyper-V_Windows	TestNoKubernetes/serial/StartWithK8s (gopogh)	61.38 (chart)
Docker_macOS	TestNetworkPlugins/group/kindnet/Start (gopogh)	66.67 (chart)
Docker_Windows	TestNoKubernetes/serial/StartWithStopK8s (gopogh)	74.80 (chart)
Docker_macOS	TestNetworkPlugins/group/bridge/DNS (gopogh)	75.25 (chart)
Docker_macOS	TestNetworkPlugins/group/enable-default-cni/DNS (gopogh)	79.51 (chart)
Docker_Cloud_Shell	TestFunctional/parallel/TunnelCmd/serial/AccessDirect (gopogh)	100.00 (chart)
Docker_Cloud_Shell	TestFunctional/parallel/TunnelCmd/serial/WaitService/IngressIP (gopogh)	100.00 (chart)
Docker_macOS	TestDownloadOnly/v1.16.0/preload-exists (gopogh)	100.00 (chart)
Docker_Windows	TestFunctional/parallel/DryRun (gopogh)	100.00 (chart)
Docker_Windows	TestFunctional/parallel/InternationalLanguage (gopogh)	100.00 (chart)
Docker_Windows	TestFunctional/parallel/TunnelCmd/serial/WaitService/IngressIP (gopogh)	100.00 (chart)
Docker_Windows	TestNetworkPlugins/group/cilium/Start (gopogh)	100.00 (chart)
Hyperkit_macOS	TestAddons/Setup (gopogh)	100.00 (chart)
Hyperkit_macOS	TestErrorSpam/setup (gopogh)	100.00 (chart)
Hyperkit_macOS	TestFunctional/serial/AuditLog (gopogh)	100.00 (chart)
Hyperkit_macOS	TestFunctional/serial/StartWithProxy (gopogh)	100.00 (chart)
Hyperkit_macOS	TestIngressAddonLegacy/serial/ValidateIngressAddonActivation (gopogh)	100.00 (chart)
Hyperkit_macOS	TestIngressAddonLegacy/serial/ValidateIngressAddons (gopogh)	100.00 (chart)
Hyperkit_macOS	TestIngressAddonLegacy/StartLegacyK8sCluster (gopogh)	100.00 (chart)
Hyperkit_macOS	TestJSONOutput/pause/Audit (gopogh)	100.00 (chart)
Hyperkit_macOS	TestJSONOutput/pause/Command (gopogh)	100.00 (chart)
Hyperkit_macOS	TestJSONOutput/start/Audit (gopogh)	100.00 (chart)
More tests...	Continued...

Too many tests failed - See test logs for more details.

To see the flake rates of all tests by environment, click here.

medyagh · 2022-01-11T22:03:21Z

thank you for this PR :)

k8s-ci-robot · 2022-01-11T22:03:49Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: afbjorklund, medyagh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [afbjorklund,medyagh]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Use cri-dockerd instead of dockershim where needed

f84ae2a

Starting with Kubernetes version 1.23, the CRI socket for Docker is handled externally instead of internally.

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 3, 2021

k8s-ci-robot requested review from prasadkatti and prezha August 3, 2021 11:22

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 3, 2021

afbjorklund mentioned this pull request Aug 3, 2021

Add cri-dockerd (and crictl) to installation #12103

Closed

Use the socket systemd unit instead of the service

52dcf3d

The socket-activated service for cri-dockerd will start on demand, similar to how the docker service works (for /var/run/docker.sock)

afbjorklund requested a review from sharifelgamal August 3, 2021 16:52

afbjorklund added 2 commits August 4, 2021 19:58

The socket has a different name than the package

51b3f1c

It is cri-docker.sock, even for cri-dockerd deb

Stop CRIDocker socket before Docker socket

865e5be

That is, reverse order from when starting up.

k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Aug 5, 2021

This comment has been minimized.

Sign in to view

afbjorklund changed the title ~~Use cri-dockerd instead of dockershim where needed~~ WIP: Use cri-dockerd instead of dockershim where needed Aug 6, 2021

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 6, 2021

afbjorklund added 2 commits August 6, 2021 09:17

Fix version comparison for pre-release versions

9957016

Make sure to pass k8s version for socket path

c738c78

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 6, 2021

This comment has been minimized.

Sign in to view

afbjorklund added 2 commits December 4, 2021 10:00

Dockershim removal was postponed until 1.24

f874533

Merge branch 'master' into cri-dockerd

f810fd2

afbjorklund changed the title ~~WIP: Use cri-dockerd instead of dockershim where needed~~ Use cri-dockerd instead of dockershim where needed Dec 4, 2021

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 4, 2021

This comment has been minimized.

Sign in to view

This was referenced Dec 10, 2021

k8s v1.24.0-alpha.1: failed to run Kubelet: using dockershim is not supported #13134

Closed

Dockershim removal feedback & issues kubernetes/kubernetes#106917

Closed

This was referenced Dec 17, 2021

ISO: Add packaging for cri-dockerd #13191

Merged

KIC: Add building of cri-dockerd #13227

Merged

Merge branch 'master' into cri-dockerd

2a26cdc

This was referenced Dec 28, 2021

Make default container runtime dynamic #13250

Closed

interested in integrating with minikube Mirantis/cri-dockerd#1

Closed

afbjorklund mentioned this pull request Jan 5, 2022

default socket path on Linux/Windows Mirantis/cri-dockerd#24

Closed

afbjorklund added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 5, 2022

afbjorklund added 2 commits January 5, 2022 23:01

Merge branch 'master' into cri-dockerd

bd09b12

The cri-dockerd socket path changed in 0.2.0

87249ab

afbjorklund mentioned this pull request Jan 6, 2022

chore: rename systemd unit file Mirantis/cri-dockerd#31

Closed

afbjorklund removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 7, 2022

medyagh approved these changes Jan 11, 2022

View reviewed changes

medyagh merged commit 254af36 into kubernetes:master Jan 11, 2022

afbjorklund mentioned this pull request Feb 2, 2022

Support docker container runtime in Kubernetes 1.24 #13545

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use cri-dockerd instead of dockershim where needed #12102

Use cri-dockerd instead of dockershim where needed #12102

afbjorklund commented Aug 3, 2021 •

edited

Loading

medyagh commented Aug 5, 2021

medyagh commented Aug 6, 2021

This comment has been minimized.

This comment has been minimized.

afbjorklund commented Aug 6, 2021 •

edited

Loading

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

afbjorklund commented Jan 5, 2022 •

edited

Loading

afbjorklund commented Jan 5, 2022

minikube-pr-bot commented Jan 5, 2022

minikube-pr-bot commented Jan 6, 2022

medyagh commented Jan 11, 2022

k8s-ci-robot commented Jan 11, 2022

Use cri-dockerd instead of dockershim where needed #12102

Use cri-dockerd instead of dockershim where needed #12102

Conversation

afbjorklund commented Aug 3, 2021 • edited Loading

medyagh commented Aug 5, 2021

medyagh commented Aug 6, 2021

This comment has been minimized.

This comment has been minimized.

afbjorklund commented Aug 6, 2021 • edited Loading

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

afbjorklund commented Jan 5, 2022 • edited Loading

afbjorklund commented Jan 5, 2022

minikube-pr-bot commented Jan 5, 2022

minikube-pr-bot commented Jan 6, 2022

medyagh commented Jan 11, 2022

k8s-ci-robot commented Jan 11, 2022

afbjorklund commented Aug 3, 2021 •

edited

Loading

afbjorklund commented Aug 6, 2021 •

edited

Loading

afbjorklund commented Jan 5, 2022 •

edited

Loading