Set defaultVolumeType for storageclass

[xeor]

Is your feature request related to a problem? Please describe.
local-path-provisioner has an option to use local instead of the default hostPath, which is not possible to set during k3s installation.

Reason for setting local is to support solutions like Velero to backup the volum. Also, there is a big warning about using hostPath at https://kubernetes.io/docs/concepts/storage/volumes/#hostpath.

The reason hostPath is default seams to be because RKE had issues (rancher/local-path-provisioner#85 (comment)) when it was set to the default some time ago, hence, it was made configurable instead.

Describe the solution you'd like
Being able to set local as volume type instead of hostpath during rke installation.

Describe alternatives you've considered
Running kubectl patch storageclass local-path -p '{"metadata":{"annotations":{"defaultVolumeType":"local"}}}' after installation

Additional context

https://github.com/k3s-io/k3s/blob/master/manifests/local-storage.yaml#L90-L98 - no config to change the annotation to set defaultVolumeType

[brandond]

A PR to change the default in the the local-storage manifest would be welcome!

[VestigeJ]

##Environment Details
Reproduced using VERSION=v1.29.3+k3s1
Validated using COMMIT=3f906bee79cb32072bea061cbbf4e38142343c66

Infrastructure

• Cloud

Node(s) CPU architecture, OS, and version:

Linux 5.14.21-150500.53-default x86_64 GNU/Linux
PRETTY_NAME="SUSE Linux Enterprise Server 15 SP5"

Cluster Configuration:

NAME               STATUS   ROLES                       AGE   VERSION
ip-1-1-1-29        Ready    control-plane,etcd,master   35m   v1.29.3+k3s-3f906bee

Config.yaml:

token: YOUR_TOKEN_HERE
write-kubeconfig-mode: 644
debug: true
profile: cis
selinux: true
protect-kernel-defaults: true
cluster-init: true
embedded-registry: true

Reproduction

$ curl https://get.k3s.io --output install-"k3s".sh
$ sudo chmod +x install-"k3s".sh
$ sudo groupadd --system etcd && sudo useradd -s /sbin/nologin --system -g etcd etcd
$ sudo modprobe ip_vs_rr
$ sudo modprobe ip_vs_wrr
$ sudo modprobe ip_vs_sh
$ sudo printf "on_oovm.panic_on_oom=0 \nvm.overcommit_memory=1 \nkernel.panic=10 \nkernel.panic_ps=1 \nkernel.panic_on_oops=1 \n" > ~/90-kubelet.conf
$ sudo cp 90-kubelet.conf /etc/sysctl.d/
$ sudo systemctl restart systemd-sysctl
$ COMMIT=3f906bee79cb32072bea061cbbf4e38142343c66
$ sudo INSTALL_K3S_COMMIT=$COMMIT INSTALL_K3S_EXEC=server ./install-k3s.sh
$ kg storageclass -A
$ kd storageclass local-path

Results:

// current release annotations

$ kg storageclass -o yaml

apiVersion: v1
items:
- apiVersion: storage.k8s.io/v1
  kind: StorageClass
  metadata:
    annotations:
      objectset.rio.cattle.io/applied: H4sIAAAAAAAA/4yRT0CAAA
      objectset.rio.cattle.io/id: ""
      objectset.rio.cattle.io/owner-gvk: k3s.cattle.io/v1, Kind=Addon
      objectset.rio.cattle.io/owner-name: local-storage
      objectset.rio.cattle.io/owner-namespace: kube-system
      storageclass.kubernetes.io/is-default-class: "true"
    creationTimestamp: "2024-04-11T22:47:09Z"

// latest commit attention to the annotation for defaultVolumeType: local
$ kg storageclass -o yaml

apiVersion: v1
items:
- apiVersion: storage.k8s.io/v1
  kind: StorageClass
  metadata:
    annotations:
      defaultVolumeType: local
      objectset.rio.cattle.io/applied: H4sIAAAAAAAA/4yKo0kCAAA
      objectset.rio.cattle.io/id: ""
      objectset.rio.cattle.io/owner-gvk: k3s.cattle.io/v1, Kind=Addon
      objectset.rio.cattle.io/owner-name: local-storage
      objectset.rio.cattle.io/owner-namespace: kube-system
      storageclass.kubernetes.io/is-default-class: "true"
    creationTimestamp: "2024-04-11T22:47:12Z"

[brandond]

@VestigeJ you might also confirm the PV type when you create a PVC following this change - check that it is local instead of hostPath.

[xeor]

@VestigeJ @brandond this is nice! But when digging into this I also found that the default set to local was rolled back in the local-path-provisioner project because it broke rke.

That's why I suggested a flag. But if the rke project can handle this now everything is fine 🎉

[brandond]

RKE is not K3s so it doesn't really matter what we do here, it has no impact on that distro.