Skip to content

Commit

Permalink
kubeadm: adapt docs for 1.24 and dockershim removal
Browse files Browse the repository at this point in the history 
Touch the following files:
- Implementation details: remove docker specifics, which is changing
in 1.24
- Create cluster: small language cleanup, remove note about 1.24
- Install kubeadm: Include two up-to-date tables for Linux / Windows
with known endpoints. Include cri-dockerd.
- Kubelet integration: (side cleanup) use "container runtime" instead of
"CRI runtime" (which is incorrect). Mention that only updating
"--container-runtime-endpoint=.." is required if the user wishes
to override the CR on a certain host. Dockershim->CR-foo migration
guides would make the "--container-runtime=remote" flag explicit
and we want to remove it at some point.
- Troubleshooting kubeadm: Remove some instances of Docker troubleshooting
that imply docker as default CR, or talk about old Docker versions.
Be more generic about container runtimes.
- Adding Windows nodes: move the containerd tab before the Docker
tab, as containerd is now the default. Remove note about being explicit
about --cri-socket. Add note that crictl is required for both
Docker and containerd. Add note that cri-dockerd is required if
the user wants to use Docker EE on Windows.
  • Loading branch information
@neolit123
neolit123 committed Jan 20, 2022
1 parent c8c474d commit 4be8f85
Show file tree
Hide file tree
Showing 6 changed files with 101 additions and 101 deletions.
12 changes: 3 additions & 9 deletions content/en/docs/reference/setup-tools/kubeadm/implementation-details.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -81,13 +81,8 @@ The user can skip specific preflight checks or all of them with the `--ignore-pr
- Kubernetes system requirements:
- if running on linux:
- [error] if Kernel is older than the minimum required version
- [error] if required cgroups subsystem aren't in set up
- if using docker:
- [warning/error] if Docker service does not exist, if it is disabled, if it is not active.
- [error] if Docker endpoint does not exist or does not work
- [warning] if docker version is not in the list of validated docker versions
- If using other cri engine:
- [error] if crictl socket does not answer
- [error] if required cgroups subsystem aren't set up
- [error] if the CRI endpoint does not answer
- [error] if user is not root
- [error] if the machine hostname is not a valid DNS subdomain
- [warning] if the host name cannot be reached via network lookup
Expand Down Expand Up @@ -434,8 +429,7 @@ cluster startup problems.
Please note that:

1. `kubeadm join` preflight checks are basically a subset `kubeadm init` preflight checks
1. Starting from 1.9, kubeadm provides better support for CRI-generic functionality; in that case, docker specific controls
are skipped or replaced by similar controls for crictl.
1. Starting from 1.24, kubeadm uses crictl to communicate to all known CRI endpoints.
1. Starting from 1.9, kubeadm provides support for joining nodes running on Windows; in that case, linux specific controls are skipped.
1. In any case the user can skip specific preflight checks (or eventually all preflight checks) with the `--ignore-preflight-errors` option.

Expand Down
11 changes: 5 additions & 6 deletions ...nt/en/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -111,10 +111,9 @@ for all control-plane nodes. Such an endpoint can be either a DNS name or an IP
be passed to `kubeadm init`. Depending on which
third-party provider you choose, you might need to set the `--pod-network-cidr` to
a provider-specific value. See [Installing a Pod network add-on](#pod-network).
1. (Optional) Since version 1.14, `kubeadm` tries to detect the container runtime on Linux
by using a list of well known domain socket paths. To use different container runtime or
if there are more than one installed on the provisioned node, specify the `--cri-socket`
argument to `kubeadm init`. See
1. (Optional) `kubeadm` tries to detect the container runtime by using a list of well
known endpoints. To use different container runtime or if there are more than one installed
on the provisioned node, specify the `--cri-socket` argument to `kubeadm`. See
[Installing a runtime](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#installing-runtime).
1. (Optional) Unless otherwise specified, `kubeadm` uses the network interface associated
with the default gateway to set the advertise address for this particular control-plane node's API server.
Expand Down Expand Up @@ -384,8 +383,8 @@ A few seconds later, you should notice this node in the output from `kubectl get
nodes` when run on the control-plane node.
{{< note >}}
As the cluster nodes are usually initialized sequentially, the CoreDNS Pods are likely to all run
on the first control-plane node. To provide higher availability, please rebalance the CoreDNS Pods
As the cluster nodes are usually initialized sequentially, the CoreDNS Pods are likely to all run
on the first control-plane node. To provide higher availability, please rebalance the CoreDNS Pods
with `kubectl -n kube-system rollout restart deployment coredns` after at least one new node is joined.
{{< /note >}}
Expand Down
62 changes: 35 additions & 27 deletions content/en/docs/setup/production-environment/tools/kubeadm/install-kubeadm.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -79,51 +79,59 @@ The pod network plugin you use (see below) may also require certain ports to be
open. Since this differs with each pod network plugin, please see the
documentation for the plugins about what port(s) those need.

## Installing runtime {#installing-runtime}
## Installing a container runtime {#installing-runtime}

To run containers in Pods, Kubernetes uses a
{{< glossary_tooltip term_id="container-runtime" text="container runtime" >}}.

{{< tabs name="container_runtime" >}}
{{% tab name="Linux nodes" %}}

By default, Kubernetes uses the
{{< glossary_tooltip term_id="cri" text="Container Runtime Interface">}} (CRI)
to interface with your chosen container runtime.

If you don't specify a runtime, kubeadm automatically tries to detect an installed
container runtime by scanning through a list of well known Unix domain sockets.
The following table lists container runtimes and their associated socket paths:

{{< table caption = "Container runtimes and their socket paths" >}}
| Runtime | Path to Unix domain socket |
|------------|-----------------------------------|
| Docker | `/var/run/dockershim.sock` |
| containerd | `/run/containerd/containerd.sock` |
| CRI-O | `/var/run/crio/crio.sock` |
{{< /table >}}
container runtime by scanning through a list of known endpoints.

<br />
If both Docker and containerd are detected, Docker takes precedence. This is
needed because Docker 18.09 ships with containerd and both are detectable even if you only
installed Docker.
If any other two or more runtimes are detected, kubeadm exits with an error.

The kubelet integrates with Docker through the built-in `dockershim` CRI implementation.
If multiple or no container runtimes are detected kubeadm will throw an error
and will request that you specify which one you want to use.

See [container runtimes](/docs/setup/production-environment/container-runtimes/)
for more information.

{{< note >}}
Docker Engine does not implement the [CRI](/docs/concepts/architecture/cri/)
which is a requirement for a container runtime to work with Kubernetes.
For that reason, an additional service [cri-dockerd](https://github.com/Mirantis/cri-dockerd)
has to be installed. cri-dockerd is a new project based on the legacy built-in
Docker Engine support that was [removed](/dockershim) from the kubelet.
{{< /note >}}

The tables below include the known endpoints for supported operating systems:

{{< tabs name="container_runtime" >}}
{{% tab name="Linux" %}}

{{< table >}}
| Runtime | Path to Unix domain socket |
|------------------------------------|----------------------------------------------|
| containerd | `unix:///var/run/containerd/containerd.sock` |
| CRI-O | `unix:///var/run/crio/crio.sock` |
| Docker Engine (using cri-dockerd) | `unix:///var/run/cri-dockerd.sock` |
{{< /table >}}

{{% /tab %}}
{{% tab name="other operating systems" %}}
By default, kubeadm uses {{< glossary_tooltip term_id="docker" >}} as the container runtime.
The kubelet integrates with Docker through the built-in `dockershim` CRI implementation.

See [container runtimes](/docs/setup/production-environment/container-runtimes/)
for more information.
{{% tab name="Windows" %}}

{{< table >}}
| Runtime | Path to Windows named pipe |
|------------------------------------|----------------------------------------------|
| containerd | `npipe:////./pipe/containerd-containerd` |
| Docker Engine (using cri-dockerd) | `npipe:////./pipe/cri-dockerd` |
{{< /table >}}

{{% /tab %}}
{{< /tabs >}}


## Installing kubeadm, kubelet and kubectl

You will install these packages on all of your machines:
Expand Down
16 changes: 7 additions & 9 deletions content/en/docs/setup/production-environment/tools/kubeadm/kubelet-integration.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ using kubeadm, rather than managing the kubelet configuration for each Node manu
### Propagating cluster-level configuration to each kubelet

You can provide the kubelet with default values to be used by `kubeadm init` and `kubeadm join`
commands. Interesting examples include using a different CRI runtime or setting the default subnet
commands. Interesting examples include using a different container runtime or setting the default subnet
used by services.

If you want your services to use the subnet `10.96.0.0/12` as the default for services, you can pass
Expand All @@ -51,7 +51,7 @@ by the kubelet, using the `--cluster-dns` flag. This setting needs to be the sam
on every manager and Node in the cluster. The kubelet provides a versioned, structured API object
that can configure most parameters in the kubelet and push out this configuration to each running
kubelet in the cluster. This object is called
[`KubeletConfiguration`](/docs/reference/config-api/kubelet-config.v1beta1/).
[`KubeletConfiguration`](/docs/reference/config-api/kubelet-config.v1beta1/).
The `KubeletConfiguration` allows the user to specify flags such as the cluster DNS IP addresses expressed as
a list of values to a camelCased key, illustrated by the following example:

Expand All @@ -78,14 +78,12 @@ networking, or other host-specific parameters. The following list provides a few
unless you are using a cloud provider. You can use the `--hostname-override` flag to override the
default behavior if you need to specify a Node name different from the machine's hostname.

- Currently, the kubelet cannot automatically detect the cgroup driver used by the CRI runtime,
but the value of `--cgroup-driver` must match the cgroup driver used by the CRI runtime to ensure
- Currently, the kubelet cannot automatically detect the cgroup driver used by the container runtime,
but the value of `--cgroup-driver` must match the cgroup driver used by the container runtime to ensure
the health of the kubelet.

- Depending on the CRI runtime your cluster uses, you may need to specify different flags to the kubelet.
For instance, when using Docker, you need to specify flags such as `--network-plugin=cni`, but if you
are using an external runtime, you need to specify `--container-runtime=remote` and specify the CRI
endpoint using the `--container-runtime-endpoint=<path>`.
- To specify the container runtime you must set its endpoint with the
`--container-runtime-endpoint=<path>` flag.

You can specify these flags by configuring an individual kubelet's configuration in your service manager,
such as systemd.
Expand Down Expand Up @@ -123,7 +121,7 @@ KUBELET_KUBEADM_ARGS="--flag1=value1 --flag2=value2 ..."
```

In addition to the flags used when starting the kubelet, the file also contains dynamic
parameters such as the cgroup driver and whether to use a different CRI runtime socket
parameters such as the cgroup driver and whether to use a different container runtime socket
(`--cri-socket`).

After marshalling these two files to disk, kubeadm attempts to run the following two
Expand Down
26 changes: 7 additions & 19 deletions ...t/en/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,8 @@ and investigating each container by running `docker logs`. For other container r

## kubeadm blocks when removing managed containers

The following could happen if Docker halts and does not remove any Kubernetes-managed containers:
The following could happen if the container runtime halts and does not remove
any Kubernetes-managed containers:

```shell
sudo kubeadm reset
Expand All @@ -111,35 +112,22 @@ sudo kubeadm reset
(block)
```

A possible solution is to restart the Docker service and then re-run `kubeadm reset`:

```shell
sudo systemctl restart docker.service
sudo kubeadm reset
```

Inspecting the logs for docker may also be useful:

```shell
journalctl -u docker
```
A possible solution is to restart the container runtime and then re-run `kubeadm reset`.
You can also use `crictl` to debug the state of the container runtime. See
[Debugging Kubernetes nodes with crictl](/docs/tasks/debug-application-cluster/crictl/).

## Pods in `RunContainerError`, `CrashLoopBackOff` or `Error` state

Right after `kubeadm init` there should not be any pods in these states.

- If there are pods in one of these states _right after_ `kubeadm init`, please open an
issue in the kubeadm repo. `coredns` (or `kube-dns`) should be in the `Pending` state
issue in the kubeadm repo. `coredns` should be in the `Pending` state
until you have deployed the network add-on.
- If you see Pods in the `RunContainerError`, `CrashLoopBackOff` or `Error` state
after deploying the network add-on and nothing happens to `coredns` (or `kube-dns`),
after deploying the network add-on and nothing happens to `coredns`,
it's very likely that the Pod Network add-on that you installed is somehow broken.
You might have to grant it more RBAC privileges or use a newer version. Please file
an issue in the Pod Network providers' issue tracker and get the issue triaged there.
- If you install a version of Docker older than 1.12.1, remove the `MountFlags=slave` option
when booting `dockerd` with `systemd` and restart `docker`. You can see the MountFlags in `/usr/lib/systemd/system/docker.service`.
MountFlags can interfere with volumes mounted by Kubernetes, and put the Pods in `CrashLoopBackOff` state.
The error happens when Kubernetes does not find `var/run/secrets/kubernetes.io/serviceaccount` files.

## `coredns` is stuck in the `Pending` state

Expand Down
75 changes: 44 additions & 31 deletions content/en/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ in the `flannel-host-gw.yml` or `flannel-overlay.yml` file and specify your inte
curl -L https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/flannel-overlay.yml | sed 's/Ethernet/Ethernet0 2/g' | kubectl apply -f -
```
{{< /note >}}



### Joining a Windows worker node
Expand All @@ -147,33 +147,7 @@ with elevated permissions (Administrator) on the Windows worker node.
{{< /note >}}

{{< tabs name="tab-windows-kubeadm-runtime-installation" >}}
{{% tab name="Docker EE" %}}

#### Install Docker EE

Install the `Containers` feature

```powershell
Install-WindowsFeature -Name containers
```

Install Docker
Instructions to do so are available at [Install Docker Engine - Enterprise on Windows Servers](https://docs.microsoft.com/en-us/virtualization/windowscontainers/quick-start/set-up-environment?tabs=Windows-Server#install-docker).

#### Install wins, kubelet, and kubeadm

```PowerShell
curl.exe -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/kubeadm/scripts/PrepareNode.ps1
.\PrepareNode.ps1 -KubernetesVersion {{< param "fullversion" >}}
```

#### Run `kubeadm` to join the node

Use the command that was given to you when you ran `kubeadm init` on a control plane host.
If you no longer have this command, or the token has expired, you can run `kubeadm token create --print-join-command`
(on a control plane host) to generate a new token and join command.

{{% /tab %}}
{{% tab name="CRI-containerD" %}}

#### Install containerD
Expand All @@ -191,9 +165,6 @@ To install a specific version of containerD specify the version with -ContainerD
.\Install-Containerd.ps1 -ContainerDVersion 1.4.1
```

{{< /note >}}

{{< note >}}
If you're using a different interface rather than Ethernet (i.e. "Ethernet0 2") on the Windows nodes, specify the name with `-netAdapterName`.
```powershell
Expand All @@ -210,17 +181,59 @@ curl.exe -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools
.\PrepareNode.ps1 -KubernetesVersion {{< param "fullversion" >}} -ContainerRuntime containerD
```
[Install `crictl` from the cri-tools package](https://github.com/kubernetes-sigs/cri-tools)
which is required so that kubeadm can talk to the CRI endpoint.
#### Run `kubeadm` to join the node
Use the command that was given to you when you ran `kubeadm init` on a control plane host.
If you no longer have this command, or the token has expired, you can run `kubeadm token create --print-join-command`
(on a control plane host) to generate a new token and join command.
{{% /tab %}}
{{% tab name="Docker EE" %}}
#### Install Docker EE
Install the `Containers` feature
```powershell
Install-WindowsFeature -Name containers
```
Install Docker
Instructions to do so are available at [Install Docker Engine - Enterprise on Windows Servers](https://docs.microsoft.com/en-us/virtualization/windowscontainers/quick-start/set-up-environment?tabs=Windows-Server#install-docker).
[Install cri-dockerd](https://github.com/Mirantis/cri-dockerd) which is required so that the kubelet
can communicate with Docker on a CRI compatible endpoint.
{{< note >}}
If using **CRI-containerD** add `--cri-socket "npipe:////./pipe/containerd-containerd"` to the kubeadm call
Docker Engine does not implement the [CRI](/docs/concepts/architecture/cri/)
which is a requirement for a container runtime to work with Kubernetes.
For that reason, an additional service [cri-dockerd](https://github.com/Mirantis/cri-dockerd)
has to be installed. cri-dockerd is a new project based on the legacy built-in
Docker Engine support that was [removed](/dockershim) from the kubelet.
{{< /note >}}
Install `crictl` from the [cri-tools project](https://github.com/kubernetes-sigs/cri-tools)
which is required so that kubeadm can talk to the CRI endpoint.
#### Install wins, kubelet, and kubeadm
```PowerShell
curl.exe -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/kubeadm/scripts/PrepareNode.ps1
.\PrepareNode.ps1 -KubernetesVersion {{< param "fullversion" >}}
```
#### Run `kubeadm` to join the node
Use the command that was given to you when you ran `kubeadm init` on a control plane host.
If you no longer have this command, or the token has expired, you can run `kubeadm token create --print-join-command`
(on a control plane host) to generate a new token and join command.
{{% /tab %}}
{{< /tabs >}}
### Verifying your installation
Expand Down

0 comments on commit 4be8f85

Please sign in to comment.