join second control plane etcd bind error ip

[yixianhuang]

What keywords did you search in kubeadm issues before filing this one?

etcd、kubeadm、ip、network card
If you have found any duplicates, you should instead reply there and close this page.

If you have not found any duplicates, delete this section and continue on.
not found

Is this a BUG REPORT or FEATURE REQUEST?

Choose one: BUG REPORT or FEATURE REQUEST

Versions

kubeadm version (use kubeadm version):
[root@k8snode01 ~]# kubeadm version kubeadm version: &version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.2", GitCommit:"cff46ab41ff0bb44d8584413b598ad8360ec1def", GitTreeState:"clean", BuildDate:"2019-01-10T23:33:30Z", GoVersion:"go1.11.4", Compiler:"gc", Platform:"linux/amd64"}

Environment:

• Kubernetes version (use kubectl version):
  [root@k8snode01 ~]# kubectl version Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.2", GitCommit:"cff46ab41ff0bb44d8584413b598ad8360ec1def", GitTreeState:"clean", BuildDate:"2019-01-10T23:35:51Z", GoVersion:"go1.11.4", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.2", GitCommit:"cff46ab41ff0bb44d8584413b598ad8360ec1def", GitTreeState:"clean", BuildDate:"2019-01-10T23:28:14Z", GoVersion:"go1.11.4", Compiler:"gc", Platform:"linux/amd64"}
• Cloud provider or hardware configuration: vagrant in mac
• OS (e.g. from /etc/os-release):
  `[root@k8snode01 ~]# cat /etc/os-release
  NAME="CentOS Linux"
  VERSION="7 (Core)"
  ID="centos"
  ID_LIKE="rhel fedora"
  VERSION_ID="7"
  PRETTY_NAME="CentOS Linux 7 (Core)"
  ANSI_COLOR="0;31"
  CPE_NAME="cpe:/o:centos:centos:7"
  HOME_URL="https://www.centos.org/"
  BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"`

• Kernel (e.g. uname -a):
  [root@k8snode01 ~]# uname -a Linux k8snode01 3.10.0-957.1.3.el7.x86_64 #1 SMP Thu Nov 29 14:49:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
• Others:
  docker17.03.2-ce
  socat-1.7.3.2-2.el7.x86_64
  kubelet-1.10.0-0.x86_64
  kubernetes-cni-0.6.0-0.x86_64
  kubectl-1.10.0-0.x86_64
  kubeadm-1.10.0-0.x86_64

What happened?

I setup HA kubernetes by Stacked etcd model.refrence by:
https://kubernetes.io/docs/setup/independent/ha-topology/
in node01 ,i init my kubernetes:
kubeadm init --config=kubeadm-config.yaml
`[root@k8snode01 ~]# cat kubeadm-config.yaml
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
etcd:
local:
imageRepository: "registry.cn-hangzhou.aliyuncs.com/google_containers"
imageTag: "3.2.24"
dataDir: "/var/lib/etcd"
extraArgs:
listen-client-urls: "https://192.168.33.61:2379,http://127.0.0.1:2379"
advertise-client-urls: "https://192.168.33.61:2379"
initial-advertise-peer-urls: "https://192.168.33.61:2380"
initial-cluster: "k8snode01=https://192.168.33.61:2380"
listen-peer-urls: "https://192.168.33.61:2380"
serverCertSANs:
- "192.168.33.61"
- "192.168.33.66"
- "k8snode01"
peerCertSANs:
- "192.168.33.61"
- "192.168.33.66"
- "k8snode01"
kubernetesVersion: "v1.13.2"
apiServer:
certSANs:

• "192.168.33.66"
  controlPlaneEndpoint: "192.168.33.66:6443"
  networking:
  serviceSubnet: "10.96.0.0/12"
  podSubnet: "10.100.0.1/24"
  dnsDomain: "cluster.local"
  imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
  clusterName: "k8s-cluster"this sucdess.but when i use join in node02:[root@k8snode02 manifests]# kubeadm join 192.168.33.66:6443 --token xzffzn.2veb9rd3hzj5kfe8 --discovery-token-ca-cert-hash sha256:1e044b5cc57a86839bdfd0c71b42e37b7c6e9c5b786bbefe8ce9cabe3537a50a --experimental-control-plane --apiserver-advertise-address 192.168.33.62
  [preflight] Running pre-flight checks
  [discovery] Trying to connect to API Server "192.168.33.66:6443"
  [discovery] Created cluster-info discovery client, requesting info from "https://192.168.33.66:6443"
  [discovery] Requesting info from "https://192.168.33.66:6443" again to validate TLS against the pinned public key
  [discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "192.168.33.66:6443"
  [discovery] Successfully established connection with API Server "192.168.33.66:6443"
  [join] Reading configuration from the cluster...
  [join] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
  [join] Running pre-flight checks before initializing the new control plane instance
  [certs] Using the existing "front-proxy-client" certificate and key
  [certs] Using the existing "apiserver-etcd-client" certificate and key
  [certs] Using the existing "etcd/server" certificate and key
  [certs] Using the existing "etcd/peer" certificate and key
  [certs] Using the existing "etcd/healthcheck-client" certificate and key
  [certs] Using the existing "apiserver-kubelet-client" certificate and key
  [certs] Using the existing "apiserver" certificate and key
  [certs] valid certificates and keys now exist in "/etc/kubernetes/pki"
  [certs] Using the existing "sa" key
  [kubeconfig] Using existing up-to-date kubeconfig file: "/etc/kubernetes/admin.conf"
  [kubeconfig] Writing "controller-manager.conf" kubeconfig file
  [kubeconfig] Writing "scheduler.conf" kubeconfig file
  [etcd] Checking Etcd cluster health
  error syncing endpoints with etc: dial tcp 10.0.2.15:2379: connect: connection refused`

[root@k8snode02 manifests]# ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 52:54:00:84:81:d5 brd ff:ff:ff:ff:ff:ff inet 10.0.2.15/24 brd 10.0.2.255 scope global noprefixroute dynamic eth0 valid_lft 81548sec preferred_lft 81548sec inet6 fe80::5054:ff:fe84:81d5/64 scope link valid_lft forever preferred_lft forever 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 08:00:27:6c:ee:c3 brd ff:ff:ff:ff:ff:ff inet 192.168.33.62/24 brd 192.168.33.255 scope global noprefixroute eth1 valid_lft forever preferred_lft forever inet6 fe80::a00:27ff:fe6c:eec3/64 scope link valid_lft forever preferred_lft forever 4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:a6:0b:6a:a8 brd ff:ff:ff:ff:ff:ff inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 valid_lft forever preferred_lft forever

i think the etcd get error ip form eth0.the corrent is from eth1.

What you expected to happen?

success join the another master node.

How to reproduce it (as minimally and precisely as possible)?

1、vagrant have two network card
2、use kubeam init to init
3、in other node use join to join the master you can get this error

Anything else we need to know?

[fabriziopandini]

@yixianhuang if you are using local etcd, it is much easier to let kubeadm set etcd extra args and all the certificates sans. Kubeadm will give you same defaults. If you instead override this values in the config intentionally, you should know very well how ha works in kubeadm.

Finally, please take care of copying only the certificates specified in the documentation, otherwise you risk to have problems on the joining node

[yixianhuang]

if you use advertiseAddress ,you can get de corrent ip.but I get another problem.
apiVersion: kubeadm.k8s.io/v1beta1 kind: InitConfiguration localAPIEndpoint: advertiseAddress: "192.168.33.61" bindPort: 6443

[de1m]

error execution phase check-etcd: error syncing endpoints with etc: dial tcp 10.0.2.15:2379: connect: connection refused

I've the same issue, I also use the vagrant (2 vms) to install the HA Cluster.
I think, the problem is the first nic of vagrant vm, this is a NAT nic to communicate with the host. The ip of this nic is 10.0.2.15 on both vms.
I don't understand , what mean this message. Will the kubeadm connect the first master on ip 10.0.2.15 or local ip?

[mobby1984]

@de1m Were you able to ever fix or address this issue? Apparently, I ran into the same issue not too long ago. I am also using Vagrant to set up my virtual machines and trying to create an HA Kubernetes cluster for development purposes.

Vagrant scripts were downloaded from https://github.com/mmumshad/kubernetes-the-hard-way

[muratcankarakurt]

Hi @mobby1984

If you used the same configuration with https://github.com/mmumshad/kubernetes-the-hard-way add the advertise address to both kubeadm init and kubeadm join commands like this:

sudo kubeadm init --control-plane-endpoint=192.168.5.11:6443 --upload-certs --apiserver-advertise-address=192.168.5.11

sudo kubeadm join 192.168.5.11:6443 --token XXXXX --discovery-token-ca-cert-hash sha256:XXXXX --control-plane --certificate-key XXXXX --apiserver-advertise-address=192.168.5.12