Container image is missing /etc/protocols causing Output: mount.nfs: Protocol not supported on NFSv3

[strideynet]

What happened:

Attempting to mount a NFSv3 share produces:

Output: mount.nfs: Protocol not supported

What you expected to happen:

It to mount without error.

How to reproduce it:

Attempt to use a NFSv3 server.

Anything else we need to know?:

During investigation, I found that execing into the container and running apt install netbase which provides the /etc/protocols file means I am then able to successfully mount the share:

➜  kce-cpc-gitops git:(master) kubectl exec -n kube-system csi-nfs-node-2l579 -c nfs -it sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
# mkdir ./test
# mount  -t nfs -o nolock 10.11.1.8:/katapult/fsv_4uaqmYsHDkd8xyZt/pvc-123c08df-6b14-4481-8074-d1e61f5a57a8 ./test -vvv -o vers=3 -o tcp
mount.nfs: timeout set for Thu Dec  2 19:53:39 2021
mount.nfs: trying text-based options 'nolock,vers=3,tcp,addr=10.11.1.8'
mount.nfs: prog 100003, trying vers=3, prot=6
mount.nfs: trying 10.11.1.8 prog 100003 vers 3 prot TCP port 2049
mount.nfs: prog 100005, trying vers=3, prot=6
mount.nfs: trying 10.11.1.8 prog 100005 vers 3 prot TCP port 20048
mount.nfs: Protocol not supported
# cat /etc/protocols
cat: /etc/protocols: No such file or directory
# apt update
Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB]
Get:2 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB]
Get:3 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB]
Get:4 http://deb.debian.org/debian bullseye/main amd64 Packages [8180 kB]
Get:5 http://deb.debian.org/debian bullseye-updates/main amd64 Packages [2592 B]
Get:6 http://security.debian.org/debian-security bullseye-security/main amd64 Packages [97.5 kB]
Fetched 8479 kB in 2s (5450 kB/s)                     
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
8 packages can be upgraded. Run 'apt list --upgradable' to see them.
# apt install netbase -y
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
  netbase
0 upgraded, 1 newly installed, 0 to remove and 8 not upgraded.
Need to get 19.9 kB of archives.
After this operation, 42.0 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian bullseye/main amd64 netbase all 6.3 [19.9 kB]
Fetched 19.9 kB in 0s (1069 kB/s)   
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package netbase.
(Reading database ... 5097 files and directories currently installed.)
Preparing to unpack .../archives/netbase_6.3_all.deb ...

Progress: [  0%] [......................................................................................................................................................] 
Unpacking netbase (6.3) ...#####################........................................................................................................................] 

Setting up netbase (6.3) ...##################################################..........................................................................................] 

Progress: [ 60%] [##########################################################################################............................................................] 
# mount  -t nfs -o nolock 10.11.1.8:/katapult/fsv_4uaqmYsHDkd8xyZt/pvc-123c08df-6b14-4481-8074-d1e61f5a57a8 ./test -vvv -o vers=3 -o tcp
mount.nfs: timeout set for Thu Dec  2 19:54:03 2021
mount.nfs: trying text-based options 'nolock,vers=3,tcp,addr=10.11.1.8'
mount.nfs: prog 100003, trying vers=3, prot=6
mount.nfs: trying 10.11.1.8 prog 100003 vers 3 prot TCP port 2049
mount.nfs: prog 100005, trying vers=3, prot=6
mount.nfs: trying 10.11.1.8 prog 100005 vers 3 prot TCP port 20048
# 

We should probably install this package within the docker container, or manually provide a /etc/protocols file.

Environment:

• CSI Driver version: Using the v3.0.0 helm chart

  nfs:
    Container ID:  containerd://15dd003bfe4693e8f06c4c554cf19b5a99df426ffdc9f3e7c4a89cad1c654c25
    Image:         mcr.microsoft.com/k8s/csi/nfs-csi:v3.0.0
    Image ID:      mcr.microsoft.com/k8s/csi/nfs-csi@sha256:df491d3f8353b4bbffee11332239632b97417107787c79094c97e9eab98e5b63

• Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.7+k3s1", GitCommit:"aa768cbdabdb44c95c5c1d9562ea7f5ded073bc0", GitTreeState:"clean", BuildDate:"2021-05-20T01:07:13Z", GoVersion:"go1.15.12", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.7+k3s1", GitCommit:"aa768cbdabdb44c95c5c1d9562ea7f5ded073bc0", GitTreeState:"clean", BuildDate:"2021-05-20T01:07:13Z", GoVersion:"go1.15.12", Compiler:"gc", Platform:"linux/amd64"}

• OS (e.g. from /etc/os-release):

NAME="Ubuntu"
VERSION="20.04 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

• Kernel (e.g. uname -a): Linux kce-staging 5.4.0-26-generic #30-Ubuntu SMP Mon Apr 20 16:58:30 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
• Install tools:
• Others:

[strideynet]

Patched docker image available at https://github.com/strideynet/csi-nfs-driver-patched

Use with the following helm values:

image:
  nfs:
    repository: ghcr.io/strideynet/csi-nfs-driver-patched
    tag: v3.0.0-0

[andyzhangx]

thanks, I have worked out a PR: #253, pls take a look.

[strideynet]

@andyzhangx PR looks good, matches my own implementation of a fix.

[andyzhangx]

mcr.microsoft.com/k8s/csi/nfs-csi:v3.0.0 already fixed the issue, pls try with ImagePullPolicy: Always