minikube tries to talk on older IP of the VM

[surajssd]

Is this a BUG REPORT or FEATURE REQUEST?:

/kind bug

Environment:

minikube version: v0.22.1

OS:
NAME=Fedora
VERSION="25 (Workstation Edition)"
ID=fedora
VERSION_ID=25
PRETTY_NAME="Fedora 25 (Workstation Edition)"
ANSI_COLOR="0;34"
CPE_NAME="cpe:/o:fedoraproject:fedora:25"
HOME_URL="https://fedoraproject.org/"
SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=25
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=25
PRIVACY_POLICY_URL=https://fedoraproject.org/wiki/Legal:PrivacyPolicy
VARIANT="Workstation Edition"
VARIANT_ID=workstation

VM driver:
    "DriverName": "kvm",

ISO version
        "ISO": "/home/foobar/.minikube/machines/minikube/boot2docker.iso",
        "Boot2DockerURL": "file:///home/foobar/.minikube/cache/iso/minikube-v0.23.3.iso",

I am using version v0.10.0 of docker-machine-driver-kvm https://github.com/dhiltgen/docker-machine-kvm/releases/tag/v0.10.0
verify using the md5sum

$ md5sum `which docker-machine-driver-kvm `
fb2ded7b5b20400ef66f0adbc384364e  /usr/local/bin/docker-machine-driver-kvm

What happened:

When I stop minikube and try to start again it fails to start. When I ran it in debug mode, it seems like it is talking to VM on IP address which was allocated to VM on previous boot. While when the machine boots up it is getting a new IP address which minikube is not picking up.

What you expected to happen:

minikube should pickup the new IP address that VM has got on this boot time

How to reproduce it (as minimally and precisely as possible):

Now that I have stopped machine, this is the status

$ minikube status
minikube: Stopped
cluster: 
kubectl: 

IP from last boot

$ minikube ip
192.168.42.165

while I ran following command I also did run other commands given after this blob

$ minikube start -v 10
Starting local Kubernetes v1.7.5 cluster...
Starting VM...
Found binary path at /usr/local/bin/docker-machine-driver-kvm
Launching plugin server for driver kvm
Plugin server listening at address 127.0.0.1:43621
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(minikube) Calling .GetState
(minikube) DBG | Getting current state...
(minikube) DBG | Fetching VM...
(minikube) Calling .Start
(minikube) DBG | Starting VM minikube
(minikube) DBG | GetIP called for minikube
(minikube) DBG | Failed to retrieve dnsmasq leases from /var/lib/libvirt/dnsmasq/docker-machines.leases
(minikube) DBG | IP address: 192.168.42.165
(minikube) DBG | Unable to locate IP address for MAC 52:54:00:0d:09:c2
(minikube) Calling .GetConfigRaw
(minikube) Calling .DriverName
Waiting for SSH to be available...
Getting to WaitForSSH function...
(minikube) Calling .GetSSHHostname
(minikube) DBG | GetIP called for minikube
(minikube) DBG | Failed to retrieve dnsmasq leases from /var/lib/libvirt/dnsmasq/docker-machines.leases
(minikube) DBG | IP address: 192.168.42.165
(minikube) DBG | Unable to locate IP address for MAC 52:54:00:0d:09:c2
(minikube) Calling .GetSSHPort
(minikube) Calling .GetSSHKeyPath
(minikube) Calling .GetSSHKeyPath
(minikube) Calling .GetSSHUsername
Using SSH client type: native
&{{{<nil> 0 [] [] []} docker [0x844e00] <nil>  [] 0s} 192.168.42.165 22 <nil> <nil>}
About to run SSH command:
exit 0
Error dialing TCP: dial tcp 192.168.42.165:22: getsockopt: no route to host
Error dialing TCP: dial tcp 192.168.42.165:22: getsockopt: no route to host
Error dialing TCP: dial tcp 192.168.42.165:22: getsockopt: no route to host
Error dialing TCP: dial tcp 192.168.42.165:22: getsockopt: no route to host
Error dialing TCP: dial tcp 192.168.42.165:22: getsockopt: no route to host
Error dialing TCP: dial tcp 192.168.42.165:22: getsockopt: no route to host
Error dialing TCP: dial tcp 192.168.42.165:22: getsockopt: no route to host
Error dialing TCP: dial tcp 192.168.42.165:22: getsockopt: no route to host
Error dialing TCP: dial tcp 192.168.42.165:22: getsockopt: no route to host
^C

meanwhile above was running I did following

See the status it says that the config is mismatching

$ minikube status
minikube: Running
cluster: Stopped
kubectl: Misconfigured: pointing to stale minikube-vm.
To fix the kubectl context, run minikube update-context

so I updated the context, the IP has changed, but it has not changed in the above when minikube was coming up

$ minikube update-context 
Reconfigured kubeconfig IP, now pointing at 192.168.42.166

$ minikube status
minikube: Running
cluster: Stopped
kubectl: Correctly Configured: pointing to minikube-vm at 192.168.42.166

[surajnarwade]

issue reference: #951

[surajssd]

So this worked for me as workaround:

Start the VM, when it is hung for long exit using Ctrl + C

$ minikube start
Starting local Kubernetes v1.7.5 cluster...
Starting VM...
^C

new IP, which minikube does not know yet

$ minikube ip
192.168.42.172

this can fail, but try again!

$ minikube status
minikube: Running
cluster: Stopped
kubectl: Misconfigured: pointing to stale minikube-vm.
To fix the kubectl context, run minikube update-context

now update context

$ minikube update-context 
Reconfigured kubeconfig IP, now pointing at 192.168.42.172

once the context is updated minikube start works again

$ minikube start
Starting local Kubernetes v1.7.5 cluster...
Starting VM...
Getting VM IP address...
Moving files into cluster...
Setting up certs...
Connecting to cluster...
Setting up kubeconfig...
Starting cluster components...
Kubectl is now configured to use the cluster.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten
/remove-lifecycle stale

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close