Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

minikube incorrectly assumes the DOCKER_HOST is always a numeric IP address #10158

Closed
alanblairverint opened this issue Jan 16, 2021 · 6 comments · Fixed by #13300
Closed

minikube incorrectly assumes the DOCKER_HOST is always a numeric IP address #10158

alanblairverint opened this issue Jan 16, 2021 · 6 comments · Fixed by #13300
Assignees
@spowelljr
Labels
kind/bug Categorizes issue or PR as related to a bug. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
Milestone
1.25.0

Comments

@alanblairverint
Copy link

alanblairverint commented Jan 16, 2021 •
edited
Loading

I am unable to get minikube to start on Windows 10 using the following versions:

Minikube
minikube version: v1.16.0
commit: 9f1e482

Kubectl:
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.0", GitCommit:"af46c47ce925f4c4ad5cc8d1fca46c7b77d13b38", GitTreeState:"clean", BuildDate:"2020-12-08T17:59:43Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"windows/amd64"}

Docker:
Client: Docker Engine - Community
Cloud integration: 1.0.7
Version: 20.10.2
API version: 1.41
Go version: go1.13.15
Git commit: 2291f61
Built: Mon Dec 28 16:14:16 2020
OS/Arch: windows/amd64
Context: default
Experimental: true

Server: Docker Engine - Community
Engine:
Version: 20.10.2
API version: 1.41 (minimum version 1.12)
Go version: go1.13.15
Git commit: 8891c58
Built: Mon Dec 28 16:15:28 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.3
GitCommit: 269548fa27e0089a8b8278fc4fc781d7f65a939b
runc:
Version: 1.0.0-rc92
GitCommit: ff819c7e9184c13b7c2607fe6c30ae19403a7aff
docker-init:
Version: 0.19.0
GitCommit: de40ad0

minikube start --driver=docker --alsologtostderr

Steps to reproduce the issue:

  1. Fresh install of minikbe & docker desktop
  2. Run minikube start --driver=docker --alsologtostderr

Logs
c:>minikube start --driver=docker --alsologtostderr
I0116 18:32:49.280441 3812 out.go:221] Setting OutFile to fd 92 ...
I0116 18:32:49.284427 3812 out.go:268] TERM=,COLORTERM=, which probably does not support color
I0116 18:32:49.284427 3812 out.go:234] Setting ErrFile to fd 96...
I0116 18:32:49.284427 3812 out.go:268] TERM=,COLORTERM=, which probably does not support color
W0116 18:32:49.298444 3812 root.go:255] Error reading config file at C:\Users\ablair.minikube\config\config.json: open C:\Users\ablair.minikube\config\config.json: The system cannot find the file specified.
I0116 18:32:49.302427 3812 out.go:228] Setting JSON to false
I0116 18:32:49.311473 3812 start.go:104] hostinfo: {"hostname":"BFS-501507a","uptime":3910,"bootTime":1610818059,"procs":329,"os":"windows","platform":"Microsoft Windows 10 Enterprise","platformFamily":"Standalone Workstation","platformVersion":"10.0.17763 Build 17763","kernelVersion":"","virtualizationSystem":"","virtualizationRole":"","hostid":"10a241a0-29c8-4855-b8ca-750919e3c58f"}
W0116 18:32:49.311473 3812 start.go:112] gopshost.Virtualization returned error: not implemented yet
I0116 18:32:49.322643 3812 out.go:119] * minikube v1.16.0 on Microsoft Windows 10 Enterprise 10.0.17763 Build 17763

  • minikube v1.16.0 on Microsoft Windows 10 Enterprise 10.0.17763 Build 17763
    I0116 18:32:49.323410 3812 notify.go:126] Checking for updates...
    I0116 18:32:49.325408 3812 driver.go:303] Setting default libvirt URI to qemu:///system
    I0116 18:32:49.881408 3812 docker.go:117] docker version: linux-20.10.2
    I0116 18:32:49.915439 3812 cli_runner.go:111] Run: docker system info --format "{{json .}}"
    I0116 18:32:51.155016 3812 cli_runner.go:155] Completed: docker system info --format "{{json .}}": (1.2385773s)
    I0116 18:32:51.155016 3812 info.go:253] docker info: {ID:6UXK:GT6V:U6SE:VVRK:KOTF:2LQB:NRWV:KF7S:4OW5:26PQ:B5TJ:I5PF Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:1 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true]] SystemStatus: Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization: Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:true KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:50 OomKillDisable:true NGoroutines:53 SystemTime:2021-01-16 18:32:50.4786514 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:4 KernelVersion:4.19.121-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:2062274560 GenericResources: DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:20.10.2 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:269548fa27e0089a8b8278fc4fc781d7f65a939b Expected:269548fa27e0089a8b8278fc4fc781d7f65a939b} RuncCommit:{ID:ff819c7e9184c13b7c2607fe6c30ae19403a7aff Expected:ff819c7e9184c13b7c2607fe6c30ae19403a7aff} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=default] ProductLicense: Warnings: ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Experimental:true Name:app Path:C:\ProgramData\Docker\cli-plugins\docker-app.exe SchemaVersion:0.1.0 ShortDescription:Docker App Vendor:Docker Inc. Version:v0.9.1-beta3] map[Name:buildx Path:C:\ProgramData\Docker\cli-plugins\docker-buildx.exe SchemaVersion:0.1.0 ShortDescription:Build with BuildKit Vendor:Docker Inc. Version:v0.5.1-docker] map[Name:scan Path:C:\ProgramData\Docker\cli-plugins\docker-scan.exe SchemaVersion:0.1.0 ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.5.0]] Warnings:}}
    I0116 18:32:51.161028 3812 out.go:119] * Using the docker driver based on existing profile
  • Using the docker driver based on existing profile
    I0116 18:32:51.161028 3812 start.go:277] selected driver: docker
    I0116 18:32:51.161993 3812 start.go:686] validating driver "docker" against &{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.15-snapshot4@sha256:ef1f485b5a1cfa4c989bc05e153f0a8525968ec999e242efff871cbb31649c16 Memory:1966 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:minikube Namespace:default APIServerName:localhost APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.49.2 Port:8443 KubernetesVersion:v1.20.0 ControlPlane:true Worker:true}] Addons:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop: ExposedPorts:[] MultiNodeRequested:false}
    I0116 18:32:51.162995 3812 start.go:697] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error: Fix: Doc:}
    I0116 18:32:51.234027 3812 cli_runner.go:111] Run: docker system info --format "{{json .}}"
    I0116 18:32:52.423990 3812 cli_runner.go:155] Completed: docker system info --format "{{json .}}": (1.1899635s)
    I0116 18:32:52.423990 3812 info.go:253] docker info: {ID:6UXK:GT6V:U6SE:VVRK:KOTF:2LQB:NRWV:KF7S:4OW5:26PQ:B5TJ:I5PF Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:1 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true]] SystemStatus: Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization: Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:true KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:50 OomKillDisable:true NGoroutines:53 SystemTime:2021-01-16 18:32:51.7662762 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:4 KernelVersion:4.19.121-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:2062274560 GenericResources: DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:20.10.2 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:269548fa27e0089a8b8278fc4fc781d7f65a939b Expected:269548fa27e0089a8b8278fc4fc781d7f65a939b} RuncCommit:{ID:ff819c7e9184c13b7c2607fe6c30ae19403a7aff Expected:ff819c7e9184c13b7c2607fe6c30ae19403a7aff} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=default] ProductLicense: Warnings: ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Experimental:true Name:app Path:C:\ProgramData\Docker\cli-plugins\docker-app.exe SchemaVersion:0.1.0 ShortDescription:Docker App Vendor:Docker Inc. Version:v0.9.1-beta3] map[Name:buildx Path:C:\ProgramData\Docker\cli-plugins\docker-buildx.exe SchemaVersion:0.1.0 ShortDescription:Build with BuildKit Vendor:Docker Inc. Version:v0.5.1-docker] map[Name:scan Path:C:\ProgramData\Docker\cli-plugins\docker-scan.exe SchemaVersion:0.1.0 ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.5.0]] Warnings:}}
    I0116 18:32:53.630919 3812 start_flags.go:367] config:
    {Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.15-snapshot4@sha256:ef1f485b5a1cfa4c989bc05e153f0a8525968ec999e242efff871cbb31649c16 Memory:1966 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:minikube Namespace:default APIServerName:localhost APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.49.2 Port:8443 KubernetesVersion:v1.20.0 ControlPlane:true Worker:true}] Addons:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop: ExposedPorts:[] MultiNodeRequested:false}
    I0116 18:32:53.641148 3812 out.go:119] * Starting control plane node minikube in cluster minikube
  • Starting control plane node minikube in cluster minikube
    I0116 18:32:54.329453 3812 image.go:92] Found gcr.io/k8s-minikube/kicbase:v0.0.15-snapshot4@sha256:ef1f485b5a1cfa4c989bc05e153f0a8525968ec999e242efff871cbb31649c16 in local docker daemon, skipping pull
    I0116 18:32:54.329453 3812 cache.go:116] gcr.io/k8s-minikube/kicbase:v0.0.15-snapshot4@sha256:ef1f485b5a1cfa4c989bc05e153f0a8525968ec999e242efff871cbb31649c16 exists in daemon, skipping pull
    I0116 18:32:54.330473 3812 preload.go:97] Checking if preload exists for k8s version v1.20.0 and runtime docker
    I0116 18:32:54.331451 3812 preload.go:105] Found local preload: C:\Users\ablair.minikube\cache\preloaded-tarball\preloaded-images-k8s-v8-v1.20.0-docker-overlay2-amd64.tar.lz4
    I0116 18:32:54.332460 3812 cache.go:54] Caching tarball of preloaded images
    I0116 18:32:54.333448 3812 preload.go:131] Found C:\Users\ablair.minikube\cache\preloaded-tarball\preloaded-images-k8s-v8-v1.20.0-docker-overlay2-amd64.tar.lz4 in cache, skipping download
    I0116 18:32:54.334452 3812 cache.go:57] Finished verifying existence of preloaded tar for v1.20.0 on docker
    I0116 18:32:54.335453 3812 profile.go:147] Saving config to C:\Users\ablair.minikube\profiles\minikube\config.json ...
    I0116 18:32:54.348450 3812 cache.go:185] Successfully downloaded all kic artifacts
    I0116 18:32:54.350450 3812 start.go:314] acquiring machines lock for minikube: {Name:mk6cae157420339782da0bcc15f19ec4eb3a47e6 Clock:{} Delay:500ms Timeout:10m0s Cancel:}
    I0116 18:32:54.351459 3812 start.go:318] acquired machines lock for "minikube" in 1.0083ms
    I0116 18:32:54.352486 3812 start.go:94] Skipping create...Using existing machine configuration
    I0116 18:32:54.354453 3812 fix.go:54] fixHost starting:
    I0116 18:32:54.435447 3812 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
    I0116 18:32:55.130490 3812 fix.go:107] recreateIfNeeded on minikube: state=Running err=
    W0116 18:32:55.130490 3812 fix.go:133] unexpected machine state, will restart:
    I0116 18:32:55.136493 3812 out.go:119] * Updating the running docker "minikube" container ...
  • Updating the running docker "minikube" container ...
    I0116 18:32:55.141494 3812 machine.go:88] provisioning docker machine ...
    I0116 18:32:55.150499 3812 ubuntu.go:169] provisioning hostname "minikube"
    I0116 18:32:55.192489 3812 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
    I0116 18:32:55.855019 3812 main.go:119] libmachine: Using SSH client type: native
    I0116 18:32:55.857030 3812 main.go:119] libmachine: &{{{ 0 [] [] []} docker [0x178db80] 0x178db40 [] 0s} localhost 55003 }
    I0116 18:32:55.859025 3812 main.go:119] libmachine: About to run SSH command:
    sudo hostname minikube && echo "minikube" | sudo tee /etc/hostname
    I0116 18:32:56.031023 3812 main.go:119] libmachine: SSH cmd err, output: : minikube

I0116 18:32:56.068061 3812 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
I0116 18:32:56.697575 3812 main.go:119] libmachine: Using SSH client type: native
I0116 18:32:56.698544 3812 main.go:119] libmachine: &{{{ 0 [] [] []} docker [0x178db80] 0x178db40 [] 0s} localhost 55003 }
I0116 18:32:56.699578 3812 main.go:119] libmachine: About to run SSH command:

            if ! grep -xq '.*\sminikube' /etc/hosts; then
                    if grep -xq '127.0.1.1\s.*' /etc/hosts; then
                            sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 minikube/g' /etc/hosts;
                    else
                            echo '127.0.1.1 minikube' | sudo tee -a /etc/hosts;
                    fi
            fi

I0116 18:32:56.867055 3812 main.go:119] libmachine: SSH cmd err, output: :
I0116 18:32:56.867055 3812 ubuntu.go:175] set auth options {CertDir:C:\Users\ablair.minikube CaCertPath:C:\Users\ablair.minikube\certs\ca.pem CaPrivateKeyPath:C:\Users\ablair.minikube\certs\ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:C:\Users\ablair.minikube\machines\server.pem ServerKeyPath:C:\Users\ablair.minikube\machines\server-key.pem ClientKeyPath:C:\Users\ablair.minikube\certs\key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:C:\Users\ablair.minikube\certs\cert.pem ServerCertSANs:[] StorePath:C:\Users\ablair.minikube}
I0116 18:32:56.872067 3812 ubuntu.go:177] setting up certificates
I0116 18:32:56.878056 3812 provision.go:83] configureAuth start
I0116 18:32:56.920057 3812 cli_runner.go:111] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" minikube
I0116 18:32:57.566047 3812 provision.go:137] copyHostCerts
I0116 18:32:57.567048 3812 exec_runner.go:145] found C:\Users\ablair.minikube/ca.pem, removing ...
I0116 18:32:57.575048 3812 exec_runner.go:190] rm: C:\Users\ablair.minikube\ca.pem
I0116 18:32:57.577050 3812 exec_runner.go:152] cp: C:\Users\ablair.minikube\certs\ca.pem --> C:\Users\ablair.minikube/ca.pem (1078 bytes)
I0116 18:32:57.583049 3812 exec_runner.go:145] found C:\Users\ablair.minikube/cert.pem, removing ...
I0116 18:32:57.584057 3812 exec_runner.go:190] rm: C:\Users\ablair.minikube\cert.pem
I0116 18:32:57.593058 3812 exec_runner.go:152] cp: C:\Users\ablair.minikube\certs\cert.pem --> C:\Users\ablair.minikube/cert.pem (1119 bytes)
I0116 18:32:57.599103 3812 exec_runner.go:145] found C:\Users\ablair.minikube/key.pem, removing ...
I0116 18:32:57.606055 3812 exec_runner.go:190] rm: C:\Users\ablair.minikube\key.pem
I0116 18:32:57.609084 3812 exec_runner.go:152] cp: C:\Users\ablair.minikube\certs\key.pem --> C:\Users\ablair.minikube/key.pem (1675 bytes)
I0116 18:32:57.613073 3812 provision.go:111] generating server cert: C:\Users\ablair.minikube\machines\server.pem ca-key=C:\Users\ablair.minikube\certs\ca.pem private-key=C:\Users\ablair.minikube\certs\ca-key.pem org=ablair.minikube san=[192.168.49.2 localhost localhost 127.0.0.1 minikube minikube]
I0116 18:32:57.760055 3812 provision.go:165] copyRemoteCerts
I0116 18:32:57.800088 3812 ssh_runner.go:149] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0116 18:32:57.832053 3812 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
I0116 18:32:58.493112 3812 sshutil.go:48] new ssh client: &{IP:localhost Port:55003 SSHKeyPath:C:\Users\ablair.minikube\machines\minikube\id_rsa Username:docker}
I0116 18:32:58.594076 3812 ssh_runner.go:310] scp C:\Users\ablair.minikube\certs\ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0116 18:32:58.619083 3812 ssh_runner.go:310] scp C:\Users\ablair.minikube\machines\server.pem --> /etc/docker/server.pem (1208 bytes)
I0116 18:32:58.647081 3812 ssh_runner.go:310] scp C:\Users\ablair.minikube\machines\server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0116 18:32:58.673348 3812 provision.go:86] duration metric: configureAuth took 1.7932578s
I0116 18:32:58.673348 3812 ubuntu.go:193] setting minikube options for container-runtime
I0116 18:32:58.712383 3812 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
I0116 18:32:59.382348 3812 main.go:119] libmachine: Using SSH client type: native
I0116 18:32:59.382348 3812 main.go:119] libmachine: &{{{ 0 [] [] []} docker [0x178db80] 0x178db40 [] 0s} localhost 55003 }
I0116 18:32:59.385350 3812 main.go:119] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0116 18:32:59.565374 3812 main.go:119] libmachine: SSH cmd err, output: : overlay

I0116 18:32:59.565374 3812 ubuntu.go:71] root file system type: overlay
I0116 18:32:59.574894 3812 provision.go:296] Updating docker unit: /lib/systemd/system/docker.service ...
I0116 18:32:59.669918 3812 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
I0116 18:33:00.335395 3812 main.go:119] libmachine: Using SSH client type: native
I0116 18:33:00.336404 3812 main.go:119] libmachine: &{{{ 0 [] [] []} docker [0x178db80] 0x178db40 [] 0s} localhost 55003 }
I0116 18:33:00.338404 3812 main.go:119] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket

[Service]
Type=notify
Restart=on-failure
StartLimitBurst=3
StartLimitIntervalSec=60

This file is a systemd drop-in unit that inherits from the base dockerd configuration.

The base configuration already specifies an 'ExecStart=...' command. The first directive

here is to clear out that command inherited from the base configuration. Without this,

the command from the base configuration and the command specified here are treated as

a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

will catch this invalid input and refuse to start the service with an error like:

Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

container runtimes. If left unlimited, it may result in OOM issues with MySQL.

ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID

Having non-zero Limit*s causes performance problems due to accounting overhead

in the kernel. We recommend using cgroups to do container-local accounting.

LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

Uncomment TasksMax if your systemd version supports it.

Only systemd 226 and above support this version.

TasksMax=infinity
TimeoutStartSec=0

set delegate yes so that systemd does not reset the cgroups of docker containers

Delegate=yes

kill only the docker process, not all processes in the cgroup

KillMode=process

[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0116 18:33:00.504462 3812 main.go:119] libmachine: SSH cmd err, output: : [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket

[Service]
Type=notify
Restart=on-failure
StartLimitBurst=3
StartLimitIntervalSec=60

This file is a systemd drop-in unit that inherits from the base dockerd configuration.

The base configuration already specifies an 'ExecStart=...' command. The first directive

here is to clear out that command inherited from the base configuration. Without this,

the command from the base configuration and the command specified here are treated as

a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

will catch this invalid input and refuse to start the service with an error like:

Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

container runtimes. If left unlimited, it may result in OOM issues with MySQL.

ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP

Having non-zero Limit*s causes performance problems due to accounting overhead

in the kernel. We recommend using cgroups to do container-local accounting.

LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

Uncomment TasksMax if your systemd version supports it.

Only systemd 226 and above support this version.

TasksMax=infinity
TimeoutStartSec=0

set delegate yes so that systemd does not reset the cgroups of docker containers

Delegate=yes

kill only the docker process, not all processes in the cgroup

KillMode=process

[Install]
WantedBy=multi-user.target

I0116 18:33:00.539501 3812 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
I0116 18:33:01.205475 3812 main.go:119] libmachine: Using SSH client type: native
I0116 18:33:01.205475 3812 main.go:119] libmachine: &{{{ 0 [] [] []} docker [0x178db80] 0x178db40 [] 0s} localhost 55003 }
I0116 18:33:01.208462 3812 main.go:119] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0116 18:33:01.364357 3812 main.go:119] libmachine: SSH cmd err, output: :
I0116 18:33:01.364357 3812 machine.go:91] provisioned docker machine in 6.2138579s
I0116 18:33:01.367358 3812 start.go:268] post-start starting for "minikube" (driver="docker")
I0116 18:33:01.374356 3812 start.go:278] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0116 18:33:01.421382 3812 ssh_runner.go:149] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0116 18:33:01.455366 3812 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
I0116 18:33:02.064414 3812 sshutil.go:48] new ssh client: &{IP:localhost Port:55003 SSHKeyPath:C:\Users\ablair.minikube\machines\minikube\id_rsa Username:docker}
I0116 18:33:02.208438 3812 ssh_runner.go:149] Run: cat /etc/os-release
I0116 18:33:02.216403 3812 main.go:119] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0116 18:33:02.216403 3812 main.go:119] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0116 18:33:02.220413 3812 main.go:119] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0116 18:33:02.226414 3812 info.go:97] Remote host: Ubuntu 20.04.1 LTS
I0116 18:33:02.228420 3812 filesync.go:118] Scanning C:\Users\ablair.minikube\addons for local assets ...
I0116 18:33:02.239432 3812 filesync.go:118] Scanning C:\Users\ablair.minikube\files for local assets ...
I0116 18:33:02.243437 3812 start.go:271] post-start completed in 869.0815ms
I0116 18:33:02.298433 3812 ssh_runner.go:149] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0116 18:33:02.329403 3812 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
I0116 18:33:02.934397 3812 sshutil.go:48] new ssh client: &{IP:localhost Port:55003 SSHKeyPath:C:\Users\ablair.minikube\machines\minikube\id_rsa Username:docker}
I0116 18:33:03.033403 3812 fix.go:56] fixHost completed within 8.6789496s
I0116 18:33:03.033403 3812 start.go:81] releasing machines lock for "minikube", held for 8.681944s
I0116 18:33:03.077404 3812 cli_runner.go:111] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" minikube
I0116 18:33:03.677417 3812 ssh_runner.go:149] Run: curl -sS -m 2 https://k8s.gcr.io/
I0116 18:33:03.721477 3812 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
I0116 18:33:03.722432 3812 ssh_runner.go:149] Run: systemctl --version
I0116 18:33:03.766400 3812 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
I0116 18:33:04.533927 3812 sshutil.go:48] new ssh client: &{IP:localhost Port:55003 SSHKeyPath:C:\Users\ablair.minikube\machines\minikube\id_rsa Username:docker}
I0116 18:33:04.588928 3812 sshutil.go:48] new ssh client: &{IP:localhost Port:55003 SSHKeyPath:C:\Users\ablair.minikube\machines\minikube\id_rsa Username:docker}
I0116 18:33:04.865448 3812 ssh_runner.go:189] Completed: curl -sS -m 2 https://k8s.gcr.io/: (1.1880305s)
I0116 18:33:04.865448 3812 ssh_runner.go:189] Completed: systemctl --version: (1.1420116s)
I0116 18:33:04.915480 3812 ssh_runner.go:149] Run: sudo systemctl is-active --quiet service containerd
I0116 18:33:04.977443 3812 ssh_runner.go:149] Run: sudo systemctl cat docker.service
I0116 18:33:04.995443 3812 cruntime.go:200] skipping containerd shutdown because we are bound to it
I0116 18:33:05.045462 3812 ssh_runner.go:149] Run: sudo systemctl is-active --quiet service crio
I0116 18:33:05.110464 3812 ssh_runner.go:149] Run: sudo systemctl cat docker.service
I0116 18:33:05.177721 3812 ssh_runner.go:149] Run: sudo systemctl daemon-reload
I0116 18:33:05.300734 3812 ssh_runner.go:149] Run: sudo systemctl start docker
I0116 18:33:05.347775 3812 ssh_runner.go:149] Run: docker version --format {{.Server.Version}}
I0116 18:33:05.419858 3812 out.go:140] * Preparing Kubernetes v1.20.0 on Docker 20.10.0 ...

  • Preparing Kubernetes v1.20.0 on Docker 20.10.0 ...| I0116 18:33:05.458861 3812 cli_runner.go:111] Run: docker exec -t minikube dig +short host.docker.internal
    \ I0116 18:33:06.158819 3812 network.go:68] got host ip for mount in container by digging dns: 192.168.65.2
    I0116 18:33:06.212818 3812 ssh_runner.go:149] Run: grep 192.168.65.2 host.minikube.internal$ /etc/hosts
    | I0116 18:33:06.258821 3812 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "8443/tcp") 0).HostPort}}'" minikube
  • I0116 18:33:06.866819 3812 out.go:119]

W0116 18:33:06.878832 3812 out.go:181] X Exiting due to DRV_CP_ENDPOINT: failed to get API Server URL: failed to parse ip for "localhost"
X Exiting due to DRV_CP_ENDPOINT: failed to get API Server URL: failed to parse ip for "localhost"
W0116 18:33:06.892819 3812 out.go:181] * Suggestion:

Recreate the cluster by running:
minikube delete --profile=minikube
minikube start --profile=minikube

  • Suggestion:

    Recreate the cluster by running:
    minikube delete --profile=minikube
    minikube start --profile=minikube
    I0116 18:33:06.905851 3812 out.go:119]
@priyawadhwa priyawadhwa added the kind/support Categorizes issue or PR as a support question. label Jan 25, 2021
@spowelljr
Copy link
Member

Thank you for reporting this problem. This seems to be a known issue as we have an issue created to fix this #10357. Feel free to track that issue to know when it is solved.

@spowelljr
Copy link
Member

Hey @alanblairverint since this seems to be a duplicate of #10357 I'm going to close this issue, feel free to add additional information to that issue. However, if you think this issue is different and should be re-opened just leave a comment and I can re-open it, thanks!

@snickell
Copy link

snickell commented Jan 5, 2022

@spowelljr I think this issue is different, that issue involved a blank IP docker host, the issue at hand here (and I think the issue could be retitled: "minikube trying to parse valid hostnames, listed as a DOCKER_HOST, as if they were numeric IP addresses, which they don't have to be"

@snickell
Copy link

snickell commented Jan 5, 2022

I think this issue should be re-opened and titled something like: "minikube incorrectly tries to parse the DOCKER_HOST as a numeric IP address"

To give a more concrete example, I went to the trouble of publishing the IPs of one my docker hosts to a public DNS A entry as deletemenow3.ceresimaging.net (so no "VM and host don't share name resolution" issues can occur), and the same issue occurs:

**arm64 ~/.ssh export DOCKER_HOST="ssh://deletemenow3.ceresimaging.net"
arm64 ~/.ssh docker ps
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
arm64 ~/.ssh cd ~
arm64 ~ minikube start
😄  minikube v1.24.0 on Darwin 11.6 (arm64)
✨  Using the docker driver based on user configuration
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
    > gcr.io/k8s-minikube/kicbase: 0 B [_______________________] ?% ? p/s 200ms
    > index.docker.io/kicbase/sta...: 0 B [____________________] ?% ? p/s 200ms
    > gcr.io/k8s-minikube/kicbase...: 0 B [____________________] ?% ? p/s 200ms
    > index.docker.io/kicbase/sta...: 0 B [____________________] ?% ? p/s 300ms
🔥  Creating docker container (CPUs=2, Memory=4000MB) ...
❗  Listening to 0.0.0.0 on external docker host deletemenow3.ceresimaging.net. Please be advised
❗  Executing "docker ps -a --format {{.Names}}" took an unusually long time: 5.7801065s
💡  Restarting the docker service may improve performance.
❗  Executing "docker container inspect minikube --format={{.State.Status}}" took an unusually long time: 5.868468875s
💡  Restarting the docker service may improve performance.
🐳  Preparing Kubernetes v1.22.3 on Docker 20.10.8 ...

❌  Exiting due to DRV_CP_ENDPOINT: failed to get API Server URL: failed to parse ip for "deletemenow3.ceresimaging.net"**

However, setting the DOCKER_HOST to the equivalent IP address (34.105.17.162==deletemenow3.ceresimaging.net, again, an A record published to a public DNS server), minikube start works! This is consisently reproducible, using either fully resolved public hostnames like deletemenow3.ceresimaging.net, or with local hostnames like localhost, or myserver.local.

@spowelljr spowelljr reopened this Jan 5, 2022
@spowelljr spowelljr changed the title Exiting due to DRV_CP_ENDPOINT: failed to get API Server URL: failed to parse ip for "localhost" minikube incorrectly assumes the DOCKER_HOST is always a numeric IP address Jan 5, 2022
@spowelljr spowelljr self-assigned this Jan 5, 2022
@spowelljr spowelljr added kind/bug Categorizes issue or PR as related to a bug. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed kind/support Categorizes issue or PR as a support question. labels Jan 5, 2022
@spowelljr spowelljr added this to the 1.25.0 milestone Jan 5, 2022
@spowelljr spowelljr mentioned this issue Jan 5, 2022
Merged
@spowelljr
Copy link
Member

Thanks for outlining the issue @snickell, I created a PR that should resolve this issue.

@snickell
Copy link

snickell commented Jan 6, 2022

@spowelljr WOW! that is so awesome thank you for your work on this 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@spowelljr spowelljr

Labels
kind/bug Categorizes issue or PR as related to a bug. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
Projects
None yet
Milestone
1.25.0
Development

Successfully merging a pull request may close this issue.

Support DOCKER_HOST not being numeric IP spowelljr/minikube
4 participants
@snickell @priyawadhwa @alanblairverint @spowelljr