Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revise “Container runtimes” page in light of dockershim removal #30882

Merged
merged 1 commit into from
Apr 20, 2022
Merged

Revise “Container runtimes” page in light of dockershim removal #30882

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 28 additions & 13 deletions content/en/docs/setup/production-environment/container-runtimes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ what is involved and describes related tasks for setting up nodes.
<!-- body -->

Kubernetes {{< skew currentVersion >}} requires that you use a runtime that
conforms with the
conforms with the
{{< glossary_tooltip term_id="cri" text="Container Runtime Interface">}} (CRI).

See [CRI version support](#cri-versions) for more information.
Expand All @@ -29,6 +29,19 @@ Kubernetes, on Linux:
- [Docker Engine](#docker)
- [Mirantis Container Runtime](#mcr)

{{< note >}}
Dockershim, the portion of code in Kubernetes that provided direct
integration with Docker in prior releases, was removed from Kubernetes
version 1.24. This removal was announced as a [deprecation in Kubernetes v 1.20](
/blog/2020/12/08/kubernetes-1-20-release-announcement/#dockershim-deprecation)
You can check out this [documentation](
/docs/tasks/administer-cluster/migrating-from-dockershim/check-if-dockershim-deprecation-affects-you/)
to understand how this deprecation might affect you. To migrate from
dockershim you can follow [this migration guide](
/docs/tasks/administer-cluster/migrating-from-dockershim/)
to migrate from dockershim.
Comment on lines +33 to +42
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Dockershim, the portion of code in Kubernetes that provided direct
integration with Docker in prior releases, was removed from Kubernetes
version 1.24. This removal was announced as a [deprecation in Kubernetes v 1.20](
/blog/2020/12/08/kubernetes-1-20-release-announcement/#dockershim-deprecation)
You can check out this [documentation](
/docs/tasks/administer-cluster/migrating-from-dockershim/check-if-dockershim-deprecation-affects-you/)
to understand how this deprecation might affect you. To migrate from
dockershim you can follow [this migration guide](
/docs/tasks/administer-cluster/migrating-from-dockershim/)
to migrate from dockershim.
Kubernetes releases before v1.24 included a direct integration with Docker Engine,
using a component named _dockershim_. That special direct integration is no longer
part of Kubernetes (this removal was
[announced](/blog/2020/12/08/kubernetes-1-20-release-announcement/#dockershim-deprecation)
as part of the v1.20 release).
You can read
[Check whether Dockershim deprecation affects you](/docs/tasks/administer-cluster/migrating-from-dockershim/check-if-dockershim-deprecation-affects-you/) to understand how this removal might
affect you. To learn about migrating from using dockershim, see
[Migrating from dockershim](/docs/tasks/administer-cluster/migrating-from-dockershim/).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM

{{< /note >}}

{{< note >}}
For other operating systems, look for documentation specific to your platform.
{{< /note >}}
Expand Down Expand Up @@ -151,10 +164,11 @@ Install containerd:
{{< tabs name="tab-cri-containerd-installation" >}}
{{% tab name="Linux" %}}

1. Install the `containerd.io` package from the official Docker repositories.
Instructions for setting up the Docker repository for your respective Linux distribution and
installing the `containerd.io` package can be found at
[Install Docker Engine](https://docs.docker.com/engine/install/#server).
1. Install the `containerd.io` package from the [official containerd website](
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are no "containerd.io" packages on the containerd.io website, there it just called "containerd" (and no packages)

The alternative (to "containerd") would be "cri-containerd-cni" which also includes the CRI and CNI configuration:

/etc/crictl.yaml

etc/cni/net.d/10-containerd-net.conflist

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where does the package name cri-containerd-cni come from?

Copy link
Contributor

@bart0sh bart0sh Feb 28, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sftim cri-containerd-cni is a tarball, recommended as a way to install containerd by the containerd documentation: https://containerd.io/docs/getting-started/#starting-containerd

+1 for not mentioning containerd.io package.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense to link to https://containerd.io/downloads/ and https://containerd.io/docs/getting-started/#starting-containerd

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@afbjorklund installing CNI can cause hard to debug issues. I'd recommend to install containerd from cri-containerd tarball instead.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this guide was referencing container runtimes previously IIRC. So, I always followed first guide and it worked for me.

Copy link
Member

@mikebrow mikebrow Mar 2, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a somewhat complex topic...

  1. As the container runtimes move up to more recent versions of CNI (library code containernetworking/cni tag) + CNI config (has it's own versioning) + CNI default plugins (container networking/plugins tag)... the cni plugins and config being used for certain networking providers (calico, flannel, weave, ...) may also need to move up depending compatibility/incompatibility issues with older configs and older plugins.

  2. As mentioned earlier, container runtimes can be configured to point to different config and plugin directories. And some container runtimes are designed to support n-different instances of the container runtime daemon, each with a different set of configs. And yes as stated there can only be one instance of a container runtime connection to kubelet at a time.

  3. The default configs for the container runtimes, depending on how you install, may include somewhat functional bridge & loopback network setups, but these should probably not be considered production ready. Good for having something to start with before you pick your production network.

  4. There is an RPM for containerd.io and runc.io these are currently? managed by docker/moby. These RPMs presently? do not install a default containerd config. Rather, the config setup for these RPMs is very limited. This status is in flux. The distros have been looking into shipping with their own setup, GitHub actions has containerd installed, ... and so on. I point this out to note there is a bit of investigation required (systemd services, which paths, check configs, etc.) to discover the current setup on a particular node before taking action to update/upgrade a current install and to be aware of possible events that could cause harm to an install.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor packaging detail: runc is bundled with "containerd.io" and with "cri-containerd", there is no "runc.io" package.

There is a regular system "runc" package, that is a dependency of the regular system "containerd" package though.

The docker packages are basically internal to docker, they could change (and have changed before) at any time.

Haven't looked at the rpm packaging (this was deb), but assume there is a similar vendor vs. system duplication ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For SIG Docs: we want to be as generic as we can be about third-party solutions. Even if someone uses containerd, we don't know what OCI runtime they prefer. runc is common but there are others that are no less valid.

Feedback on this PR should be aimed at helping the author, @PranshuSrivastava, get this ready for merge. For any other discussion, use issues or Slack.

Also:

$ dpkg-query --show runc
runc	1.0.1-0ubuntu2~20.04.1

(that's a .deb)

@PranshuSrivastava the feedback I'd highlight for you here is: #30882 (comment)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sftim nod.. and for the CNI/networking discussion, rather than trying to go into detail here.., it makes sense to link to /docs/concepts/cluster-administration/networking/

https://containerd.io/downloads/).Instructions for setting up the Docker
Copy link
Contributor

@afbjorklund afbjorklund Feb 13, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need to set up the Docker package repository, if installing from containerd.io.

So either you install the "containerd.io" package from Docker, or you install the "containerd"
package from your distribution (apt install containerd) or download it directly from containerd.io

If you want to keep both options (docker.com and containerd.io), they need to be clearly separated.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should remove the docker repository from the containerd steps and leave the user two options:

Copy link
Contributor

@afbjorklund afbjorklund Mar 1, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to supply deb packages for containerd and runc, the same way as for "cri-tools" and "kubernetes-cni" ?

From apt.kubernetes.io, that is.

They would still need to be configured, but at least wouldn't have to juggle tarballs (and version and arch)

Or are the system ones OK ?

Copy link
Member

@neolit123 neolit123 Mar 1, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The package management in k8s needs to be community owned. Currently Google still holds the keys. But also some reorganization of the package repositories might be required for better version flexibility. These efforts are WIP.

The containerd maintainers can open a discussion about that with the k8s SIG Release. In the meantime we should document the best existing options, IMO.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to supply deb packages for containerd and runc, the same way as for "cri-tools" and "kubernetes-cni" ?

That's a decision / discussion to happen outside of tech docs. In k/website, we document existing behavior most of all and rarely look to document or decide on future changes. It's fine to plan to document the next release before it comes out, of course.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PranshuSrivastava I recommend following the approach outlined in #30882 (comment)

@afbjorklund, I'm keen to keep the focus of this PR on just the set of changes we'd make to this page so that the v1.24 docs make sense after the dockershim component is removed. Once this merges, we can consider follow-up PRs that make further improvements.

As the proverb doesn't quite say, the perfect is the enemy of the merged.

Copy link
Contributor

@afbjorklund afbjorklund Mar 1, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think changing from docker.com to containerd.io would be enough.

It also saves the extra step of having to wipe the containerd config (that disables the cri plugin which is required)

containerd config default | sudo tee /etc/containerd/config.toml

This is done to remove the disabled_plugins = ["cri"] line, that would otherwise interfere with Kubernetes.

When installing containerd from upstream, then this Docker-specific configuration file is not included anymore...

Copy link
Contributor

@bart0sh bart0sh Mar 2, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we keep containerd installation instructions as they are as they're not actually related to the purpose of this PR? We can update them in a separate PR I believe.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After we remove the dockershim, the current instructions are too likely to confuse people. I think fixing that does align with the intent of this PR.

repository for your respective Linux distribution and
installing the `containerd.io` package can be found at
[Install Docker Engine](https://docs.docker.com/engine/install/#server).

2. Configure containerd:

Expand All @@ -172,7 +186,7 @@ installing the `containerd.io` package can be found at
{{% /tab %}}
{{% tab name="Windows (PowerShell)" %}}

Start a Powershell session, set `$Version` to the desired version (ex: `$Version="1.4.3"`),
Start a Powershell session, set `$Version` to the desired version (ex: `$Version=1.4.3`),
and then run the following commands:

1. Download containerd:
Expand Down Expand Up @@ -299,7 +313,7 @@ sudo apt-get install cri-o cri-o-runc

{{% tab name="Ubuntu" %}}

To install on the following operating systems, set the environment variable `OS`
To install on the following operating systems, set the environment variable `OS`
to the appropriate field in the following table:

| Operating system | `$OS` |
Expand Down Expand Up @@ -335,7 +349,7 @@ sudo apt-get install cri-o cri-o-runc

{{% tab name="CentOS" %}}

To install on the following operating systems, set the environment variable `OS`
To install on the following operating systems, set the environment variable `OS`
to the appropriate field in the following table:

| Operating system | `$OS` |
Expand Down Expand Up @@ -416,10 +430,8 @@ in sync.

### Docker Engine {#docker}

Docker Engine is the container runtime that started it all. Formerly known just as Docker,
this container runtime is available in various forms.
[Install Docker Engine](https://docs.docker.com/engine/install/) explains your options
for installing this runtime.
On each of your nodes, install Docker for your Linux distribution as per
[Install Docker Engine](https://docs.docker.com/engine/install/#server).

Docker Engine is directly compatible with Kubernetes {{< skew currentVersion >}}, using the deprecated `dockershim` component. For more information
and context, see the [Dockershim deprecation FAQ](/dockershim).
Expand All @@ -428,7 +440,10 @@ You can also find third-party adapters that let you use Docker Engine with Kuber
through the supported {{< glossary_tooltip term_id="cri" text="Container Runtime Interface">}}
(CRI).

The following CRI adaptors are designed to work with Docker Engine:
{{< note >}}
`overlay2` is the preferred storage driver for systems running Linux kernel version 4.0 or higher,
or RHEL or CentOS using version 3.10.0-514 and above.
{{< /note >}}

- [`cri-dockerd`](https://github.com/Mirantis/cri-dockerd) from Mirantis
Comment on lines -431 to 448
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The following CRI adaptors are designed to work with Docker Engine:
{{< note >}}
`overlay2` is the preferred storage driver for systems running Linux kernel version 4.0 or higher,
or RHEL or CentOS using version 3.10.0-514 and above.
{{< /note >}}
- [`cri-dockerd`](https://github.com/Mirantis/cri-dockerd) from Mirantis
You can also find third-party adapters that let you use Docker Engine with Kubernetes
through the supported {{< glossary_tooltip term_id="cri" text="Container Runtime Interface">}}
(CRI).
The following CRI adapters are designed to integrate Docker Engine with CRI clients (such
as Kubernetes v{{< skew currentVersion >}}):
- [`cri-dockerd`](https://github.com/Mirantis/cri-dockerd) from Mirantis
{{< note >}}
If you use Docker Engine on a node that is running Linux kernel version 4.0 or higher, or that
runs RHEL or CentOS and uses the Linux kernel version 3.10.0-514 and above,
the Kubernetes project suggests using the `overlay2` storage driver.
{{< /note >}}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM


Expand Down