sysext: add containerd recipe #97

tormath1 · 2024-11-07T14:02:01Z

this build containerd + runc sysext

This can be useful for users interested to test containerd 2.0 with user namespace support for Kubernetes

Tested on Kubernetes Flatcar cluster with usernamespace feature:

$ kubectl exec -ti userns-01 -- cat /proc/self/uid_map
         0  391577600      65536
$ ssh to-k8s-node -- cat /proc/self/uid_map
         0          0 4294967295

rata

@tormath1 Thanks! Left some comments, but all nits. Feel free to ignore them

rata · 2024-11-07T15:13:55Z

create_containerd_sysext.sh

+
+# Download containerd
+rm -f "containerd-${VERSION}.tgz"
+curl -o "containerd-${VERSION}.tgz" -fsSL "https://github.com/containerd/containerd/releases/download/v${VERSION}/containerd-static-${VERSION}-linux-${ARCH}.tar.gz"


checking the sha256 that is also available in the github repo would be great

rata · 2024-11-07T15:14:43Z

create_containerd_sysext.sh

+[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
+SystemdCgroup = true
+EOF
+sed 's/SystemdCgroup = true/SystemdCgroup = false/g' "${SYSEXTNAME}/usr/share/containerd/config.toml" > "${SYSEXTNAME}/usr/share/containerd/config-cgroupfs.toml"


systemd-cgroup is the default almost everywhere, any reason to change this? Most/All k8s e2e test run with systemd-cgroup too IIRC.

We actually create a new configuration: config-cgroupfs.toml for folks still running on cgroups v1. Even if it's not common, this is still a sceneario supported and tested on Flatcar.
By default containerd will use the config.toml which uses cgroups v2.

Ohhh, makes sense!

rata · 2024-11-07T15:16:37Z

create_containerd_sysext.sh

+{ echo "[Unit]"; echo "Upholds=containerd.service"; } > "${SYSEXTNAME}/usr/lib/systemd/system/multi-user.target.d/10-containerd-service.conf"
+mkdir -p "${SYSEXTNAME}/usr/share/containerd"
+cat > "${SYSEXTNAME}/usr/share/containerd/config.toml" <<-'EOF'
+version = 2


This config works fine, containerd will migrate it to the version 3 format on startup. But we can update it to the version 3 format at some point too.

Ok, I was wondering too if the configuration was still accurate. I can migrate it to version 3 directly - fresh start!

rata · 2024-11-07T15:18:01Z

create_containerd_sysext.sh

+LimitNOFILE=1048576
+LimitNPROC=infinity
+LimitCORE=infinity
+TasksMax=infinity


There were some notes on the release about these settings and some change. I don't remember now, but it might be relevant to check. IIRC, though, it was that it wasn't included by default anymore or something like that.

Correct:

Remove LimitNOFILE from containerd.service (containerd/containerd#8924)

tormath1 · 2024-11-08T09:08:13Z

@tormath1 Thanks! Left some comments, but all nits. Feel free to ignore them

Thanks @rata for the review - your comments have been addressed here: https://github.com/flatcar/sysext-bakery/compare/e183d5ea1b3b524e07555b87e5da7a7f03028fe7..dc704fe9f1bfc2bd7adfab9e6a64421e7aed8a17

rata

LGTM. Just nits, again, feel free to ignore them :)

rata · 2024-11-08T10:57:10Z

create_containerd_sysext.sh

+rmdir "${SYSEXTNAME}"/bin
+runc_version=$(curl -fsSL "https://raw.githubusercontent.com/containerd/containerd/refs/tags/v${VERSION}/script/setup/runc-version")
+echo "Downloading associated runc version: ${runc_version}"
+curl -o "${SYSEXTNAME}/usr/bin/runc" -fsSL "https://github.com/opencontainers/runc/releases/download/${runc_version}/runc.${ARCH}"


Sorry I didn't say this before, but maybe use the sha here too?

runc has a sha file with the sha for all the binaries, so something like this should work:

$ sha256sum --ignore-missing -c runc.sha256sum runc.amd64: OK

Ah yes forgot about runc - I was not fully awake :D

rata · 2024-11-08T11:03:14Z

create_containerd_sysext.sh

+# set containerd's OOM score
+oom_score = -999
+[plugins."io.containerd.cri.v1.runtime".containerd.runtimes.runc]
+  # setting runc.options unsets parent settings


Not sure what you mean with this, but maybe this has changed now? containerd/containerd#9982 Config changes are merged instead of override?

This is the current configuration (shipped in Flatcar) - I am not sure what it means but anyway we can drop this whole section as it's a no-op.

rata · 2024-11-08T11:05:36Z

create_containerd_sysext.sh

+oom_score = -999
+[plugins."io.containerd.cri.v1.runtime".containerd.runtimes.runc]
+  # setting runc.options unsets parent settings
+  runtime_type = "io.containerd.runc.v2"


This is a no-op now, as the v1 have been removed: https://github.com/containerd/containerd/pull/8262/files

But maybe you want to keep it, both are fine :)

rata

LGTM

Nice idea to use the containerd repo on this tag to check the runc version to use!

this build containerd + runc sysext Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>

rata reviewed Nov 7, 2024

View reviewed changes

tormath1 mentioned this pull request Nov 8, 2024

Upgrade Containerd in main from 1.7.23 to 2.0.0 flatcar/scripts#2433

Closed

tormath1 force-pushed the tormath1/containerd branch from e183d5e to dc704fe Compare November 8, 2024 09:07

tormath1 requested a review from a team November 8, 2024 09:44

rata approved these changes Nov 8, 2024

View reviewed changes

tormath1 force-pushed the tormath1/containerd branch from dc704fe to 331dbbc Compare November 8, 2024 12:13

rata approved these changes Nov 8, 2024

View reviewed changes

sysext: add containerd recipe

be0c6b7

this build containerd + runc sysext Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>

tormath1 force-pushed the tormath1/containerd branch from 331dbbc to be0c6b7 Compare November 12, 2024 08:28

tormath1 merged commit 9274cc4 into main Nov 12, 2024

tormath1 deleted the tormath1/containerd branch November 12, 2024 08:28

github-actions bot mentioned this pull request Nov 22, 2024

Monthly contributions report 2024-10-22 - 2024-11-21 flatcar/Flatcar#1585

Closed

tormath1 mentioned this pull request Dec 20, 2024

Upgrade Containerd in main from 1.7.23 to 2.0.1 flatcar/scripts#2548

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

sysext: add containerd recipe #97

sysext: add containerd recipe #97

tormath1 commented Nov 7, 2024 •

edited

Loading

rata left a comment

rata Nov 7, 2024

rata Nov 7, 2024

tormath1 Nov 8, 2024

rata Nov 8, 2024

rata Nov 7, 2024

tormath1 Nov 8, 2024

rata Nov 8, 2024

rata Nov 7, 2024

tormath1 Nov 8, 2024

rata Nov 8, 2024

tormath1 commented Nov 8, 2024

rata left a comment

rata Nov 8, 2024 •

edited

Loading

tormath1 Nov 8, 2024

rata Nov 8, 2024

tormath1 Nov 8, 2024

rata Nov 8, 2024

rata left a comment

sysext: add containerd recipe #97

sysext: add containerd recipe #97

Conversation

tormath1 commented Nov 7, 2024 • edited Loading

rata left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tormath1 commented Nov 8, 2024

rata left a comment

Choose a reason for hiding this comment

rata Nov 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rata left a comment

Choose a reason for hiding this comment

tormath1 commented Nov 7, 2024 •

edited

Loading

rata Nov 8, 2024 •

edited

Loading