Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mountpoint err when restart csi node-plugin #14

Open
hhstu opened this issue Dec 26, 2023 · 0 comments
Open

mountpoint err when restart csi node-plugin #14

hhstu opened this issue Dec 26, 2023 · 0 comments

Comments

@hhstu
Copy link

hhstu commented Dec 26, 2023

Start fuse process use systemd is one way to solve the problem

this is systemd example :
csi node-plugin shoud create it when NodeStageVolume

[root@k8s1 test]# cat  /run/systemd/transient/mfs-fuse-pvc-5be4a515-13a1-4c81-b3ac-b93713bc04de.service
# This is a transient unit file, created programmatically via the systemd API. Do not edit.
[Unit]
Description=mfs fuse mount of pvc-5be4a515-13a1-4c81-b3ac-b93713bc04de

[Service]
ExecStart=
ExecStart="mount" "-f" "-t" "moosefs" "10.109.148.254:9421:/liuchang-test" "/var/lib/kubelet/plugins/kubernetes.io/csi/liuchang-test.csi.moosefs.com/e7673584d954b10a7e08aea91ae532734ebc4300f3f2a621c7e907d03025b0ac/globalmount"
ExecStopPost=
ExecStopPost="/bin/umount" "-f" "-l" "/var/lib/kubelet/plugins/kubernetes.io/csi/liuchang-test.csi.moosefs.com/e7673584d954b10a7e08aea91ae532734ebc4300f3f2a621c7e907d03025b0ac/globalmount"

[Unit]
CollectMode=inactive-or-failed
[root@k8s1-dev-common1.pgy01.ynode.cn test]#

I have fix it in my env, this is a test that recreate node-plugin and mountpoint is healthy

[root@k8s1 demo]# kubectl  get  pod  -n mfs   -owide
NAME                                     READY   STATUS    RESTARTS   AGE     IP             NODE                              NOMINATED NODE   READINESS GATES
csi-moosefs-controller-liuchang-test-0   4/4     Running   0          20m     10.55.140.59   k8s1  <none>           <none>
csi-moosefs-node-liuchang-test-894jr     2/2     Running   0          20m     10.55.140.60   k8s2 <none>           <none>
csi-moosefs-node-liuchang-test-965c9     2/2     Running   0          20m     10.55.140.61   k8s3 <none>           <none>
csi-moosefs-node-liuchang-test-pq96r     2/2     Running   0          9m37s   10.55.140.59   k8s1  <none>           <none>
[root@k8s1-dev-common1.pgy01.ynode.cn demo]# kubectl  delete  pod  -n mfs  csi-moosefs-node-liuchang-test-pq96r
pod "csi-moosefs-node-liuchang-test-pq96r" deleted
[root@k8s1 demo]#
[root@k8s1 demo]#
[root@k8s1 demo]# systemctl  status  mfs-fuse-pvc-79be2cc4-411e-47ed-947f-dbd2150b40e6
● mfs-fuse-pvc-79be2cc4-411e-47ed-947f-dbd2150b40e6.service - mfs fuse mount of pvc-79be2cc4-411e-47ed-947f-dbd2150b40e6
     Loaded: loaded (/run/systemd/transient/mfs-fuse-pvc-79be2cc4-411e-47ed-947f-dbd2150b40e6.service; transient)
  Transient: yes
     Active: active (running) since Tue 2023-12-26 14:40:18 CST; 1min 56s ago
   Main PID: 203854 (mount)
      Tasks: 17 (limit: 153589)
     Memory: 263.6M
        CPU: 534ms
     CGroup: /system.slice/mfs-fuse-pvc-79be2cc4-411e-47ed-947f-dbd2150b40e6.service
             ├─203854 mount -f -t moosefs 10.109.148.254:9421:/liuchang-test /var/lib/kubelet/plugins/kubernetes.io/csi/liuchang-test.csi.moosefs.com/73d5febbce3f883aac9afb4f4311ff27dafc29eb61b4fd9c24a1a68327d59a9f/globalmount
             └─203855 "mfsmount (mounted on: /data1/kubelet/plugins/kubernetes.io/csi/liuchang-test.csi.moosefs.com/73d5febbce3f883aac9afb4f4311ff27dafc29eb61b4fd9c24a1a68327d59a9f/globalmount)"

Dec 26 14:40:18 k8s1 mount[203855]: mfsmount[203855]: out of memory killer disabled
Dec 26 14:40:18 k8s1 mount[203855]: mfsmount[203855]: monotonic clock function: clock_gettime
Dec 26 14:40:18 k8s1 mfsmount[203855]: setting glibc malloc arena max to 4
Dec 26 14:40:18 k8s1 mfsmount[203855]: setting glibc malloc arena test to 4
Dec 26 14:40:18 k8s1 mfsmount[203855]: out of memory killer disabled
Dec 26 14:40:18 k8s1 mfsmount[203855]: monotonic clock function: clock_gettime
Dec 26 14:40:18 k8s1 mount[203855]: mfsmount[203855]: monotonic clock speed: 331293 ops / 10 mili seconds
Dec 26 14:40:18 k8s1 mfsmount[203855]: monotonic clock speed: 331293 ops / 10 mili seconds
Dec 26 14:40:19 k8s1 mount[203855]: mfsmount[203855]: my st_dev: 2097256
Dec 26 14:40:19 k8s1 mfsmount[203855]: my st_dev: 2097256
[root@k8s1 demo]#
[root@k8s1demo]# kubectl  exec -it   mfs-subpath-6db68d88d4-6z8vw   -- df  -h
Filesystem                Size      Used Available Use% Mounted on
overlay                 198.9G     33.6G    165.3G  17% /
tmpfs                    64.0M         0     64.0M   0% /dev
mfs#10.109.148.254:9421
                         10.0G         0     10.0G   0% /data
/dev/sda2               198.9G     33.6G    165.3G  17% /etc/hosts
/dev/sda2               198.9G     33.6G    165.3G  17% /dev/termination-log
/dev/sda2               198.9G     33.6G    165.3G  17% /etc/hostname
/dev/sda2               198.9G     33.6G    165.3G  17% /etc/resolv.conf
shm                      64.0M         0     64.0M   0% /dev/shm
tmpfs                    18.6G     12.0K     18.6G   0% /run/secrets/kubernetes.io/serviceaccount
tmpfs                    11.7G         0     11.7G   0% /proc/acpi
tmpfs                    64.0M         0     64.0M   0% /proc/kcore
tmpfs                    64.0M         0     64.0M   0% /proc/keys
tmpfs                    64.0M         0     64.0M   0% /proc/timer_list
tmpfs                    11.7G         0     11.7G   0% /proc/scsi
tmpfs                    11.7G         0     11.7G   0% /sys/firmware
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant