kubeadm uses deprecated kubelet flags

[albertvaka]

BUG REPORT

Versions

kubeadm version: 1.10.3

Environment:

• Kubernetes version: 1.10.3
• Cloud provider: GCE
• OS: Ubuntu 16.04
• Kernel: 4.13
• Others: Using calico CNI

What happened?

Installing kubeadm via apt (repo: https://packages.cloud.google.com/), as recommended in https://kubernetes.io/docs/tasks/tools/install-kubeadm/ results in the following files being installed:

# dpkg-query -L kubeadm
/.
/usr
/usr/bin
/usr/bin/kubeadm
/etc
/etc/systemd
/etc/systemd/system
/etc/systemd/system/kubelet.service.d
/etc/systemd/system/kubelet.service.d/10-kubeadm.conf

The file 10-kubeadm.conf has a series of parameters for kubelet, that should instead be passed in a config file:

# cat /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
Environment="KUBELET_SYSTEM_PODS_ARGS=--pod-manifest-path=/etc/kubernetes/manifests --allow-privileged=true"
Environment="KUBELET_NETWORK_ARGS=--network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin"
Environment="KUBELET_DNS_ARGS=--cluster-dns=10.96.0.10 --cluster-domain=cluster.local"
Environment="KUBELET_AUTHZ_ARGS=--authorization-mode=Webhook --client-ca-file=/etc/kubernetes/pki/ca.crt"
Environment="KUBELET_CADVISOR_ARGS=--cadvisor-port=0"
Environment="KUBELET_CERTIFICATE_ARGS=--rotate-certificates=true --cert-dir=/var/lib/kubelet/pki"
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_SYSTEM_PODS_ARGS $KUBELET_NETWORK_ARGS $KUBELET_DNS_ARGS $KUBELET_AUTHZ_ARGS $KUBELET_CADVISOR_ARGS $KUBELET_CERTIFICATE_ARGS $KUBELET_EXTRA_ARGS

Logs from kubelet starting:

# journalctl -u kubelet
-- Logs begin at Thu 2018-05-31 19:33:55 UTC, end at Thu 2018-05-31 19:55:01 UTC. --
May 31 19:37:18 vk-beta-node-v10-b8f1 systemd[1]: Started kubelet: The Kubernetes Node Agent.
May 31 19:37:18 vk-beta-node-v10-b8f1 kubelet[1635]: I0531 19:37:18.285823    1635 feature_gate.go:226] feature gates: &{{} map[]}
May 31 19:37:18 vk-beta-node-v10-b8f1 systemd[1]: Stopping kubelet: The Kubernetes Node Agent...
May 31 19:37:18 vk-beta-node-v10-b8f1 systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
May 31 19:37:18 vk-beta-node-v10-b8f1 systemd[1]: Started kubelet: The Kubernetes Node Agent.
May 31 19:37:39 vk-beta-node-v10-b8f1 kubelet[5667]: Flag --pod-manifest-path has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 31 19:37:39 vk-beta-node-v10-b8f1 kubelet[5667]: Flag --allow-privileged has been deprecated, will be removed in a future version
May 31 19:37:39 vk-beta-node-v10-b8f1 kubelet[5667]: Flag --cluster-dns has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 31 19:37:39 vk-beta-node-v10-b8f1 kubelet[5667]: Flag --cluster-domain has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 31 19:37:39 vk-beta-node-v10-b8f1 kubelet[5667]: Flag --authorization-mode has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 31 19:37:39 vk-beta-node-v10-b8f1 kubelet[5667]: Flag --client-ca-file has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 31 19:37:39 vk-beta-node-v10-b8f1 kubelet[5667]: Flag --cadvisor-port has been deprecated, The default will change to 0 (disabled) in 1.12, and the cadvisor port will be removed entirely in 1.13
May 31 19:37:39 vk-beta-node-v10-b8f1 kubelet[5667]: Flag --kubelet-cgroups has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.

[neolit123]

the kubelet is newer than what kubeadm expects.
you can fix that moving the params you need to the kubelet config:
https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/

the systemd drop-in from kubeadm would look like this in 1.11:

[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
EnvironmentFile-=/var/lib/kubelet/kubeadm-flags.env
EnvironmentFile-=/etc/default/kubelet
ExecStart=/usr/bin/kubelet_v111 $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS

i have no information if some of that is going to be cherry-picked in 1.10.x.

[luxas]

This is a main feature coming in v1.11 and working at HEAD. See: #822