Enable cgroupv2 disable cgroupv1.

[cyrus-mc]

I see in the latest bottlerocket build that both v1 and v2 are enabled and mounted:

cgroup2 on /sys/fs/cgroup/unified type cgroup2 (rw,nosuid,nodev,noexec,relatime,seclabel,nsdelegate)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,xattr,name=systemd)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,pids)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,perf_event)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,net_cls,net_prio)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,devices)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,cpu,cpuacct)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,freezer)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,hugetlb)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,memory)

With this setup the full compliment of control files (e.g: memory.current, etc) are not present in the cgroup2 unified mount. Assuming this might have something to do with the nsdelegate option but not sure.

What is the reason for having cgroup v1 still enabled?

[cyrus-mc]

Further understanding is that we are running in hybrid mode. Why not just run a pure cgroup v2 system?

[bcressey]

@etungsten looked at enabling cgroup v2 last year and ended up reverting it in #1533 because of an incompatibility with runc at the time. That's been fixed for a while but we've also learned about more incompatibilities. For example, containers that run older versions of systemd from the CentOS 7 or Amazon Linux 2 era don't work properly, which could break solutions like dind until they're based on a newer distro.

My current thought on how to unblock this is to implement #2155 and make it possible to opt-in to "unified" mode, and then ask for feedback on how this works in practice. Once there's a bit more data in hand, future variants like aws-k8s-1.24 or aws-k8s-1.25 can be "unified" by default with "hybrid" as the fallback. Ideally this change in defaults would happen by end of year but it'll depend on how disruptive it is.