MasterConfiguration parameters not propagated to /var/lib/kubelet/config.yaml

[ewanbarr]

What keywords did you search in kubeadm issues before filing this one?

"MasterConfiguration", "swap", "KubeletConfiguration"

This may be related to #1028 and #1066 although in both those cases there is an explanation provided for the non-overrides (security enforced defaults and typo, respectively).

Is this a BUG REPORT or FEATURE REQUEST?

BUG REPORT

Versions

kubeadm version (use kubeadm version):

kubeadm version:&version.Info{  
   Major:"1",
   Minor:"11",
   GitVersion:"v1.11.2",
   GitCommit:"bb9ffb1654d4a729bb4cec18ff088eacc153c239",
   GitTreeState:"clean",
   BuildDate:"2018-08-07T23:14:39Z",
   GoVersion:"go1.10.3",
   Compiler:"gc",
   Platform:"linux/amd64"
}

Environment:

• Kubernetes version (use kubectl version):

Client Version:version.Info{  
   Major:"1",
   Minor:"11",
   GitVersion:"v1.11.2",
   GitCommit:"bb9ffb1654d4a729bb4cec18ff088eacc153c239",
   GitTreeState:"clean",
   BuildDate:"2018-08-07T23:17:28Z",
   GoVersion:"go1.10.3",
   Compiler:"gc",
   Platform:"linux/amd64"
}

• Cloud provider or hardware configuration: Single VM node.

• OS (e.g. from /etc/os-release):

PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

• Kernel (e.g. uname -a):

Linux kubehead0 4.9.0-8-amd64 #1 SMP Debian 4.9.110-3+deb9u4 (2018-08-21) x86_64 GNU/Linux

What happened?

While trying to start a master node in a VM with swap on, parameters from the config passed to kubeadm init were not making it into /var/lib/kubelet/config.yaml. This resulted in the kubelet service remaining in a crashloop as the failSwapOn parameter was set to true.

The configuration passed to kubeadm init is below:

kind: MasterConfiguration
apiVersion: kubeadm.k8s.io/v1alpha1
kubeletConfiguration:
  failSwapOn: false
  podCIDR: 10.244.0.0/16

The error returned by kubelet was:

Aug 25 10:35:13 kubehead0 kubelet[40588]: F0825 10:35:13.022471   40588 server.go:262] failed to run Kubelet: Running with swap on is not supported, please disable swap! or set --fa
il-swap-on flag to false. /proc/swaps contained: [Filename                                Type                Size        Used        Priority /dev/dm-3                               partition        524284        73840        -1] 

This error stopped kubeadm init from completing successfully.

By manually editing the /var/lib/kubelet/config.yaml (setting failSwapOn: false) while kubelet was in a crashloop, the service was able to start and kubeadm init to finish successfully.

What you expected to happen?

I would expect the failSwapOn: false option specified in the configuration passed to kubeadm init to be reflected in /var/lib/kubelet/config.yaml.

How to reproduce it (as minimally and precisely as possible)?

swapon -a

cat > config.yaml << EOF
kind: MasterConfiguration
apiVersion: kubeadm.k8s.io/v1alpha1
kubeletConfiguration:
  failSwapOn: false
  podCIDR: 10.244.0.0/16
EOF

kubeadm init --config config.yaml --ignore-preflight-errors Swap

At this point kubelet does not exit the crashloop state due to the swap unsupported failure.

By editing /var/lib/kubelet/config.yaml at this point and setting failSwapOn: false, kubelet will start and kubeadm init will complete successfully.

[neolit123]

hi, you should be using kubeadm.k8s.io/v1alpha2 with 1.11.x

there are multiple ways to disable the swap fail:
https://kubernetes.io/docs/setup/independent/kubelet-integration/#the-kubelet-drop-in-file-for-systemd
in /etc/default/kubelet

or from the v1alpha2 config:
https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-init/

please re-open if the issue persists.
/close

[ewanbarr]

Thanks, issue resolved. For posterity the following worked:

swapon -a

cat > config.yaml << EOF
kind: MasterConfiguration
apiVersion: kubeadm.k8s.io/v1alpha2
kubeletConfiguration:
  baseConfig:
    failSwapOn: false
kubeProxy:
  config:
    clusterCIDR: 10.244.0.0/16
EOF

kubeadm init --config config.yaml --ignore-preflight-errors Swap