Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chart won't install if there's another external provisioner #104

Closed
mtb-xt opened this issue Mar 25, 2024 · 1 comment
Closed

Chart won't install if there's another external provisioner #104

mtb-xt opened this issue Mar 25, 2024 · 1 comment

Comments

@mtb-xt
Copy link

mtb-xt commented Mar 25, 2024

Hi everyone.

It seems that the chart tries to create resources with names, that might be used by another external provisioner:

hawara@phoenix ~]$ helm -n csi-s3 install csi-s3 yandex-s3/csi-s3 

Error: INSTALLATION FAILED:
Unable to continue with install:
ClusterRole "external-provisioner-runner" in namespace "" exists and cannot be imported into the current release:
invalid ownership metadata;
annotation validation error: key "meta.helm.sh/release-name" must equal "csi-s3": current value is "linstor";
annotation validation error: key "meta.helm.sh/release-namespace" must equal "csi-s3": current value is "linstor-system"

I do have Linstor CSI driver installed. It seemed to coexist okay with Ceph's CSI driver.

I managed to install the chart by rendering it as a template and patching the template like this:

(Please ignore the plugin path changes, this was necessary to make the driver work with k0s distribution, it would be nice to have this configurable, but probably belongs in it's own issue?)

--- csi-s3-render-orig.yaml	2024-03-25 20:27:18.355953792 +1300
+++ csi-s3-render.yaml	2024-03-25 20:11:40.119152324 +1300
@@ -10,7 +10,7 @@
 apiVersion: v1
 kind: ServiceAccount
 metadata:
-  name: csi-provisioner-sa
+  name: csi-provisioner-s3-sa
   namespace: kube-system
 ---
 # Source: csi-s3/templates/secret.yaml
@@ -54,7 +54,7 @@
 kind: ClusterRole
 apiVersion: rbac.authorization.k8s.io/v1
 metadata:
-  name: external-provisioner-runner
+  name: external-provisioner-s3-runner
 rules:
   - apiGroups: [""]
     resources: ["secrets"]
@@ -90,14 +90,14 @@
 kind: ClusterRoleBinding
 apiVersion: rbac.authorization.k8s.io/v1
 metadata:
-  name: csi-provisioner-role
+  name: csi-provisioner-s3-role
 subjects:
   - kind: ServiceAccount
-    name: csi-provisioner-sa
+    name: csi-provisioner-s3-sa
     namespace: kube-system
 roleRef:
   kind: ClusterRole
-  name: external-provisioner-runner
+  name: external-provisioner-s3-runner
   apiGroup: rbac.authorization.k8s.io
 ---
 # Source: csi-s3/templates/provisioner.yaml
@@ -148,7 +148,7 @@
             - name: ADDRESS
               value: /csi/csi.sock
             - name: DRIVER_REG_SOCK_PATH
-              value: /var/lib/kubelet/plugins/ru.yandex.s3.csi/csi.sock
+              value: /var/lib/k0s/kubelet/plugins/ru.yandex.s3.csi/csi.sock
             - name: KUBE_NODE_NAME
               valueFrom:
                 fieldRef:
@@ -171,6 +171,8 @@
             - "--nodeid=$(NODE_ID)"
             - "--v=4"
           env:
+            - name: PLUGIN_DIR
+              value: /var/lib/k0s/kubelet/plugins/ru.yandex.s3.csi
             - name: CSI_ENDPOINT
               value: unix:///csi/csi.sock
             - name: NODE_ID
@@ -181,10 +183,10 @@
             - name: plugin-dir
               mountPath: /csi
             - name: stage-dir
-              mountPath: /var/lib/kubelet/plugins/kubernetes.io/csi
+              mountPath: /var/lib/k0s/kubelet/plugins/kubernetes.io/csi
               mountPropagation: "Bidirectional"
             - name: pods-mount-dir
-              mountPath: /var/lib/kubelet/pods
+              mountPath: /var/lib/k0s/kubelet/pods
               mountPropagation: "Bidirectional"
             - name: fuse-device
               mountPath: /dev/fuse
@@ -193,19 +195,19 @@
       volumes:
         - name: registration-dir
           hostPath:
-            path: /var/lib/kubelet/plugins_registry/
+            path: /var/lib/k0s/kubelet/plugins_registry/
             type: DirectoryOrCreate
         - name: plugin-dir
           hostPath:
-            path: /var/lib/kubelet/plugins/ru.yandex.s3.csi
+            path: /var/lib/k0s/kubelet/plugins/ru.yandex.s3.csi
             type: DirectoryOrCreate
         - name: stage-dir
           hostPath:
-            path: /var/lib/kubelet/plugins/kubernetes.io/csi
+            path: /var/lib/k0s/kubelet/plugins/kubernetes.io/csi
             type: DirectoryOrCreate
         - name: pods-mount-dir
           hostPath:
-            path: /var/lib/kubelet/pods
+            path: /var/lib/k0s/kubelet/pods
             type: Directory
         - name: fuse-device
           hostPath:
@@ -232,7 +234,7 @@
       labels:
         app: csi-provisioner-s3
     spec:
-      serviceAccount: csi-provisioner-sa
+      serviceAccount: csi-provisioner-s3-sa
       tolerations:
         - key: node-role.kubernetes.io/master
           operator: Exists
@@ -246,11 +248,11 @@
             - "--v=4"
           env:
             - name: ADDRESS
-              value: /var/lib/kubelet/plugins/ru.yandex.s3.csi/csi.sock
+              value: /var/lib/k0s/kubelet/plugins/ru.yandex.s3.csi/csi.sock
           imagePullPolicy: "IfNotPresent"
           volumeMounts:
             - name: socket-dir
-              mountPath: /var/lib/kubelet/plugins/ru.yandex.s3.csi
+              mountPath: /var/lib/k0s/kubelet/plugins/ru.yandex.s3.csi
         - name: csi-s3
           image: cr.yandex/crp9ftr22d26age3hulg/yandex-cloud/csi-s3/csi-s3-driver:0.40.3
           imagePullPolicy: IfNotPresent
@@ -260,14 +262,14 @@
             - "--v=4"
           env:
             - name: CSI_ENDPOINT
-              value: unix:///var/lib/kubelet/plugins/ru.yandex.s3.csi/csi.sock
+              value: unix:///var/lib/k0s/kubelet/plugins/ru.yandex.s3.csi/csi.sock
             - name: NODE_ID
               valueFrom:
                 fieldRef:
                   fieldPath: spec.nodeName
           volumeMounts:
             - name: socket-dir
-              mountPath: /var/lib/kubelet/plugins/ru.yandex.s3.csi
+              mountPath: /var/lib/k0s/kubelet/plugins/ru.yandex.s3.csi
       volumes:
         - name: socket-dir
           emptyDir: {}
@vitalif
Copy link
Collaborator

vitalif commented Apr 22, 2024

Thanks, committed a fix in master

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants