Kubeadm Pre-flight check fails for cgroupsv2 (CGROUPS_CPUSET: missing)

[alexcpn]

Not sure if this is an OS thing or Kubeadm related.

Versions

kubeadm version (use kubeadm version):

• kubeadm version: &version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.2",

Environment:

• Kubernetes version (use kubectl version):
  Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.2",
• Cloud provider or hardware configuration:
• Google Cloud VM
• OS (e.g. from /etc/os-release):
• Debian Buster
• Kernel (e.g. uname -a):
• Linux instance-1 4.19.0-17-cloud-amd64 kubeadm join on slave node fails preflight checks #1 SMP Debian 4.19.194-3 (2021-07-18) x86_64 GNU/Linux
• Others:

What happened?

Tried to do Kubeadm init

config.yaml

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: containerd
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
networking:
  podSubnet: "10.244.0.0/16"

root@instance-1:/home/alexcpn# mount | grep '^cgroup' | awk '{print $1}' | uniq
cgroup2

root@instance-1:/home/alexcpn# kubeadm init --config config.yaml
[init] Using Kubernetes version: v1.22.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
KERNEL_VERSION: 4.19.0-17-cloud-amd64
CONFIG_NAMESPACES: enabled
CONFIG_NET_NS: enabled
CONFIG_PID_NS: enabled
CONFIG_IPC_NS: enabled
CONFIG_UTS_NS: enabled
CONFIG_CGROUPS: enabled
CONFIG_CGROUP_CPUACCT: enabled
CONFIG_CGROUP_DEVICE: enabled
CONFIG_CGROUP_FREEZER: enabled
CONFIG_CGROUP_PIDS: enabled
CONFIG_CGROUP_SCHED: enabled
CONFIG_CPUSETS: enabled
CONFIG_MEMCG: enabled
CONFIG_INET: enabled
CONFIG_EXT4_FS: enabled (as module)
CONFIG_PROC_FS: enabled
CONFIG_NETFILTER_XT_TARGET_REDIRECT: enabled (as module)
CONFIG_NETFILTER_XT_MATCH_COMMENT: enabled (as module)
CONFIG_FAIR_GROUP_SCHED: enabled
CONFIG_OVERLAY_FS: enabled (as module)
CONFIG_AUFS_FS: not set - Required for aufs.
CONFIG_BLK_DEV_DM: enabled (as module)
CONFIG_CFS_BANDWIDTH: enabled
CONFIG_CGROUP_HUGETLB: not set - Required for hugetlb cgroup.
CONFIG_SECCOMP: enabled
CONFIG_SECCOMP_FILTER: enabled
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUSET: missing
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
CGROUPS_PIDS: enabled
CGROUPS_HUGETLB: missing
        [WARNING SystemVerification]: missing optional cgroups: hugetlb
error execution phase preflight: [preflight] Some fatal errors occurred:
        [ERROR SystemVerification]: missing required cgroups: cpuset
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher

System is running CGroupsv2

root@instance-1:/home/alexcpn# ls /sys/fs/cgroup/
cgroup.controllers  cgroup.max.descendants  cgroup.stat             cgroup.threads  kubepods      user.slice
cgroup.max.depth    cgroup.procs            cgroup.subtree_control  init.scope      system.slice
root@instance-1:/home/alexcpn# 

What you expected to happen?

How to reproduce it (as minimally and precisely as possible)?

Anything else we need to know?

[neolit123]

Hi, kubeadm uses the kubernetes/system-validators library to perform this validation. It opens /sys/fs/cgroup and looks for required and optional specification https://github.com/kubernetes/system-validators/blob/master/validators/types_unix.go Functionality like cpusets is required on the os level, and the kubelet or container runtime will prorobably fail without them. It is worth understanding why it is disabled in your case - e.g. try contacting the distro maintainers or see how you can enable it. Alternatively use a different distro. /kind support

[alexcpn]

It worked with Ubuntu 21.04, with Cgroupsv2 enabled.

[suravarapusuryakiran]

Hi Guys,

i'm facing exactly similar issue on debian NUC.! but in my case i can't switch to any other other distro.! how can i solve it in debian 4.19 or do i need any kernal update. all are passing except Cgroup.cpusets

[neolit123]

Cpuset is a required cgroup by the kubelet. You can try ignoring the error on the side of kubeadm but you might see unexpected errors in the kubelet. You can enable the cpuset config option and build a new kernel, but also you can ask the distro maintainers why it is not enabled.

…

On Mar 22, 2022 16:13, "surya kiran suravarapu" ***@***.***> wrote: Hi Guys, i'm facing exactly similar issue on debian NUC.! but in my case i can't switch to any other other distro.! how can i solve it in debian 4.19 or do i need any kernal update. all are passing except Cgroup.cpusets — Reply to this email directly, view it on GitHub <#2571 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACRATEVAS7RPGKL6SPPSA3VBHIPRANCNFSM5ELVC3GA> . You are receiving this because you commented.Message ID: ***@***.***>

[suravarapusuryakiran]

i'm getting error while joining the node into cluster. on the otherhand i can see cgroup is enabled .
similar to above output all are enabled in cgroups .! but it still says CPUset is the one only missing.

i'm clueless

[neolit123]

Try getting help on the k8s support channels. More info in the readme in this repo. Do you have proof that cpuset is enabled? If yes and if you find a bug in our validation https://github.com/kubernetes/system-validators/blob/master/validators/cgroup_validator_linux.go open a new issue in that repository please and provide full details.