[Release-1.31] - Allow configuration of Rootlesskit's CopyUpDirs through an environment variable

[brandond]

Backport fix for Allow configuration of Rootlesskit's CopyUpDirs through an environment variable

• Allow configuration of Rootlesskit's CopyUpDirs through an environment variable #10385

[mdrahman-suse]

Validated on release-1.31 branch with commit 5159131

Environment

Single server, Ubuntu 22.04

• basic_pod.yaml

apiVersion: "v1"
kind: "Pod"
metadata:
  name: "basic"
  labels:
    name: "basic"
spec:
  nodeSelector:
    kubernetes.io/os: linux
  containers:
    - name: "basic"
      image: nginx:1.14.2
      ports:
        - containerPort: 80
          name: "basic"
      volumeMounts:
        - mountPath: /my_share/
          name: "container-share"
  volumes:
    - hostPath:
        path: /home/ubuntu/containerShare
        type: ""
      name: "container-share"

Testing steps

• Setup k3s-rootless

$ curl https://get.k3s.io --output install.sh
$ sudo chmod +x install.sh
$ wget https://raw.githubusercontent.com/k3s-io/k3s/${version/commit}/k3s-rootless.service
$ mkdir -p /home/ubuntu/.config/systemd/user/
$ sudo vi k3s-rootless.service
...
[Service]
Environment=K3S_ROOTLESS_CIDR="10.41.0.0/16"
Environment=K3S_ROOTLESS_PORT_DRIVER=slirp4netns
Environment=K3S_ROOTLESS_DISABLE_HOST_LOOPBACK=true
Environment=K3S_ROOTLESS_MTU=1500
Environment=K3S_ROOTLESS_COPYUPDIRS=/home/ubuntu/containerShare
...
$ cp k3s-rootless.service /home/ubuntu/.config/systemd/user/k3s-rootless.service
$ sudo vi /etc/sysctl.conf
...
net.ipv4.ip_forward = 1
net.ipv6.conf.all.forwarding = 1
...
$ sudo sysctl -w net.ipv4.ip_forward=1
$ sudo sysctl -w net.ipv6.conf.all.forwarding=1
$ sudo apt update
$ sudo apt install uidmap
$  sudo vi /etc/default/grub
...
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
GRUB_CMDLINE_LINUX="systemd.unified_cgroup_hierarchy=1"
...
$ sudo update-grub
$ sudo mkdir -p /etc/systemd/system/user@.service.d
$ cat <<EOF | sudo tee /etc/systemd/system/user@.service.d/delegate.conf
[Service]
Delegate=cpu cpuset io memory pids
EOF
$ sudo systemctl daemon-reload
$ mkdir containerShare
$ touch containerShare/pretest.txt
$ sudo INSTALL_K3S_VERSION=v1.31.1+k3s1 INSTALL_K3S_SKIP_ENABLE=true ./install.sh
$ sudo reboot
$ grep cgroup /proc/mounts
...
cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0
...
$ stat -c %T -f /sys/fs/cgroup
...
cgroup2fs
...
$ sudo sysctl -p
...
net.ipv4.ip_forward = 1
net.ipv6.conf.all.forwarding = 1
...
$ systemctl --user enable --now k3s-rootless

• Deploy basic_pod.yaml
• Create file in my_share
• Expect the file created in pod is not visible in local
• Expect the file created in local is visible on pod

Replication

k3s -v
k3s version v1.31.1+k3s1 (452dbbc1)
go version go1.22.6

• When file is created in pod, its visible in local

$ kubectl exec -it pod/basic -- ls my_share
pretest.txt
$ kubectl exec -it pod/basic -- touch my_share/hellop2l.txt
$ ls containerShare/
hellop2l.txt  pretest.txt
$ kubectl exec -it pod/basic -- ls my_share
hellop2l.txt  pretest.txt

Validation

$ k3s -v
k3s version v1.31.1+k3s-51591315 (51591315)
go version go1.22.6

• When file is created in pod, it is not visible in local

$ kubectl exec -it pod/basic -- ls my_share
pretest.txt
$ kubectl exec -it pod/basic -- touch my_share/hellop2l.txt
$ kubectl exec -it pod/basic -- ls my_share
hellop2l.txt  pretest.txt
$ ls containerShare/
pretest.txt

• Additionally observed this symlink

$ kubectl exec -it pod/basic -- ls -la my_share
total 8
drwxrwxrwt  3 root root  100 Oct 16 22:00 .
drwxr-xr-x 22 root root 4096 Oct 16 21:55 ..
drwxrwxr-x  2 root root 4096 Oct 16 22:00 .ro1100902788
-rw-rw-r--  1 root root    0 Oct 16 22:00 hellop2l.txt
lrwxrwxrwx  1 root root   25 Oct 16 21:42 pretest.txt -> .ro1100902788/pretest.txt