k3s-killall.sh script does not work for user defined data-dir

[jakefhyde]

Environmental Info:
K3s Version: v1.29.6+k3s1

Node(s) CPU architecture, OS, and Version: N/A

Cluster Configuration: N/A

Describe the bug:

The rke2-killall.sh script hardcodes /var/lib/rancher/rke2 when removing pod manifests and unmounting & removing the directory:

• https://github.com/k3s-io/k3s/blob/master/install.sh#L763
• https://github.com/k3s-io/k3s/blob/master/install.sh#L837
  This can cause Rancher provisioned RKE2 clusters with a different data-dir to fail during an etcd resotre, as it runs the killall script.

Steps To Reproduce:

• Provision K3s cluster with custom data-dir through Rancher
• Perform etcd restore

Expected behavior:

data-dir assets are properly removed.

Actual behavior:

Attempts to remove non-existent assets in /var/lib/rancher/k3s

Additional context / logs:

[ShylajaDevadiga]

Validated using k3s version v1.30.3-rc1+k3s1 (086d9ca)

Environment Details

Infrastructure
Cloud EC2 instance

Node(s) CPU architecture, OS, and Version:
Ubuntu 22.04
Cluster Configuration:
Single node

Config.yaml:

cat /etc/rancher/k3s/config,yaml
write-kubeconfig-mode: "0644"
data-dir: /opt/mydir

Steps to reproduce the issue and validate the fix

1. Copy config.yaml passing data-dir
2. Install k3s
3. Confirm data-dir is being used and mounted
4. run killall and uninstall
5. Confirm data-dir is unmounted using kill script and removed using uninstall script

ubuntu@ip-172-31-3-71:~$ k3s -v
k3s version v1.30.3-rc1+k3s1 (086d9ca0)
go version go1.22.5
ubuntu@ip-172-31-3-71:~$ ls /opt/mydir/
agent  data  server

$ mount |grep opt
overlay on /run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/881f863111d823ab02eb9f8c6de58970f5616907aa687cc2ef5d7b55bd40d142/rootfs type overlay (rw,relatime,lowerdir=/opt/mydir/agent/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1/fs,upperdir=/opt/mydir/agent/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/2/fs,workdir=/opt/mydir/agent/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/2/work)
overlay on /run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/f3663712c080a7110a7bdf2cb439527959a1e511cd4f8321fd5881a2f32132dc/rootfs type overlay (rw,relatime,lowerdir=/opt/mydir/agent/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1/fs,upperdir=/opt/mydir


$ sudo K3S_DATA_DIR=/opt/mydir k3s-killall.sh
+ [ -s /etc/systemd/system/k3s.service ]
+ basename /etc/systemd/system/k3s.service
+ systemctl stop k3s.service
+ [ -x /etc/init.d/k3s* ]
...
+ do_unmount_and_remove /opt/mydir
+ set +x

$ mount |grep opt
ubuntu@ip-172-31-3-71:~$ 

$ ls /opt/mydir/
agent  data  server

$ sudo K3S_DATA_DIR=/opt/mydir k3s-uninstall.sh
+ id -u
+ [ 0 -eq 0 ]
+ K3S_DATA_DIR=/opt/mydir
+ /usr/local/bin/k3s-killall.sh
+ [ -s /etc/systemd/system/k3s.service ]
+ basename /etc/systemd/system/k3s.service
+ systemctl stop k3s.service
+ [ -x /etc/init.d/k3s* ]
+ killtree
+ kill -9
+ do_unmount_and_remove /run/k3s
+ set +x
+ do_unmount_and_remove /opt/mydir

$ ls /opt/mydir/
ls: cannot access '/opt/mydir/': No such file or directory

$ ls /var/lib/rancher/k3s
ls: cannot access '/var/lib/rancher/k3s': No such file or directory

Closing this issue as validated on standalone k3s install.