add clear documentation that service command on MacOs/Windows needs active terminal seassion #9016

simon-swanson-leapyear · 2020-08-18T01:05:23Z

This works on a colleague's computer, it works in our CI system, and even used to work on my machine. But something happened in the last month or so and now I can't get a service url programmatically. I have no idea where to start debugging, but would be willing to believe this is the result of something I did.

System

minikube and docker were both installed with Homebrew. Running Mac OS 10.15.6.

✓ minikube version
minikube version: v1.12.3
commit: 2243b4b97c131e3244c5f014faedca0d846599f5
✓ docker --version
Docker version 19.03.12, build 48a66213fe
✓ kubectl version 
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.8", GitCommit:"9f2892aab98fe339f3bd70e3c470144299398ace", GitTreeState:"clean", BuildDate:"2020-08-14T11:09:22Z", GoVersion:"go1.14.7", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.3", GitCommit:"2e7996e3e2712684bc73f0dec0200d64eec7fe40", GitTreeState:"clean", BuildDate:"2020-05-20T12:43:34Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

A "working" system I can use is a box from our CI system. It's running the same version of minikube (on Linux) and docker 19.03.8. All kubectl commands on the box are run with minikube kubectl -- .... I know it's not an apples to apples comparison, but it's what I can run reliably.

Steps to Reproduce

Start a new instance of minikube: minikube delete && minikube start

✓ minikube delete && minikube start 
🔥  Deleting "minikube" in docker ...
🔥  Deleting container "minikube" ...
🔥  Removing <user home>/.minikube/machines/minikube ...
💀  Removed all traces of the "minikube" cluster.
😄  minikube v1.12.3 on Darwin 10.15.6
    ▪ MINIKUBE_ACTIVE_DOCKERD=minikube
✨ Automatically selected the docker driver
👍  Starting control plane node minikube in cluster minikube
🔥  Creating docker container (CPUs=2, Memory=3892MB) ...
🐳  Preparing Kubernetes v1.18.3 on Docker 19.03.8 ...
🔎  Verifying Kubernetes components...
🌟  Enabled addons: default-storageclass, storage-provisioner
🏄  Done! kubectl is now configured to use "minikube"

Create a super basic nginx deployment+service:

✓ kubectl create deployment nginx --image=nginx && kubectl create service nodeport nginx --tcp=80:80
deployment.apps/nginx created
service/nginx created
✓ kubectl get svc
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP        31m
nginx        NodePort    10.98.108.121   <none>        80:32102/TCP   28m

Try to get the service url:
```
curl $(minikube service nginx --url)
```

Expected

I get the landing page html from nginx. From our CI system box:

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Notably, the service has a url in the service list:

$ minikube service list
|-------------|------------|--------------|-------------------------|
|  NAMESPACE  |    NAME    | TARGET PORT  |           URL           |
|-------------|------------|--------------|-------------------------|
| default     | kubernetes | No node port |
| default     | nginx      | 80-80/80     | http://172.17.0.4:30606 |
| kube-system | kube-dns   | No node port |
|-------------|------------|--------------|-------------------------|

Actual

The service has no url:

✓ minikube service list
|-------------|------------|--------------|-----|
|  NAMESPACE  |    NAME    | TARGET PORT  | URL |
|-------------|------------|--------------|-----|
| default     | kubernetes | No node port |
| default     | nginx      | 80-80/80     |     |
| kube-system | kube-dns   | No node port |
|-------------|------------|--------------|-----|

And my terminal hangs as minikube claims to form a tunnel:

✓ curl $(minikube service nginx --url)
❗  Because you are using a Docker driver on darwin, the terminal needs to be open to run it.

My terminal stops here and waits, but because I can't see the url exposed by the tunnel, I can't connect even in another terminal.

The text was updated successfully, but these errors were encountered:

tstromberg · 2020-08-20T21:54:22Z

I think this is related to the docker driver, which needs an active terminal session. We switched macOS to use this driver by default a few releases ago, if no local configuration is found. I believe you can get your original behavior by using the hyperkit driver on macOS:

minikube start --driver=hyperkit

You can also set it to the default using:

minikube config set driver hyperkit

Does this help at all?

tstromberg · 2020-08-20T21:55:37Z

Alternatively, you can also use kubectl port-forward to get a port.

medyagh · 2020-08-26T18:08:20Z

@simon-swanson-leapyear due to the docker networking limitations on MacOs you need to run minikube service command to get service.

I admit we can explain this better in the docs and make it more clear.

simon-swanson-leapyear · 2020-08-27T00:44:59Z

@tstromberg Apologies for the delay, missed this in my inbox. Using the hyperkit driver did indeed fix my problem, thanks so much!

@medyagh I don't know what docs would help particularly for someone as clueless as I, but it couldn't hurt to add more!

No need to keep this open any longer for my sake, though it looks like this will now be a place to talk about an upcoming docs change. :)

blueelvis · 2020-10-28T18:44:05Z

I just checked this on Windows (As per the title) and it is working fine.

PS D:\> minikube start
* minikube v1.13.1 on Microsoft Windows 10 Enterprise 10.0.18363 Build 18363
  - KUBECONFIG=C:\utilities\minikube_binaries\testhome\kubeconfig
  - MINIKUBE_HOME=C:\utilities\minikube_binaries\testhome
* Using the hyperv driver based on existing profile
* Starting control plane node minikube in cluster minikube
* Restarting existing hyperv VM for "minikube" ...
* Preparing Kubernetes v1.19.2 on Docker 19.03.12 ...
* Verifying Kubernetes components...
* Enabled addons: default-storageclass, storage-provisioner
* Done! kubectl is now configured to use "minikube" by default
PS D:\> kubectl create deployment nginx --image=nginx && kubectl create service nodeport nginx --tcp=80:80
deployment.apps/nginx created
service/nginx created
PS D:\> minikube service nginx --url
http://172.17.237.62:32766
PS D:\> curl http://172.17.237.62:32766
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

dev-michael-schmidt · 2020-11-09T14:20:58Z

I believe you can get your original behavior by using the hyperkit driver on macOS:

minikube start --driver=hyperkit

You can also set it to the default using:

minikube config set driver hyperkit

Does this help at all?

I tried this, and it SEEMS like this is the solution, but for my case (following the hello-world tutorial for testing sake) I got an page not found (but a proper launch without the msg). I suspect the IP or NAT has it wrong perhaps?

fejta-bot · 2021-02-07T15:48:27Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

PatrickGeyer · 2022-03-07T03:20:13Z

Wonder if there's a better fix than switching driver?

prajithsonu524 · 2022-03-17T11:13:28Z

i am using m1 chip which doesnt support hyperkit at all and using docker vm also is throwing the same problem pls help me out

AndrewAlscher · 2022-03-23T13:20:18Z

Yes, I'm also using m1 chip and try find the other solution.

SyedK1987 · 2022-03-31T14:02:39Z

Same, I also use M1 Chip and I also face the same problem... doesn't matter if I use the Service as NodePort/LoadBalancer the issue is same.... Not able to reach the Ip and it's not giving the URL as well. Could someone help please!

|-------------|------------|--------------|-----|
| NAMESPACE | NAME | TARGET PORT | URL |
|-------------|------------|--------------|-----|
| default | hello-world | 8080 | |
| default | kubernetes | No node port |
| kube-system | kube-dns | No node port |
|-------------|------------|--------------|-----|

stumbows · 2022-04-05T06:36:21Z

This is also affecting me on M1 Mac. Would love to know of a solution to work around this.

bergstenarn · 2022-04-06T06:23:17Z

Same problem for me using MacBook Air m1 and docker driver.

@medyagh , would you mind to re-open this issue?

arifterzioglu · 2022-04-07T09:09:46Z

@medyagh Same problem. I cannot also expose any service to external by using Macbook Pro M1. Could you propose any solution or workaround for the problem?

akshaychopra5207 · 2022-04-09T16:06:27Z

I am facing the same issue with Macbook M1 pro. I cannot use hyperkit since its not supported for arm architecture.

hoani · 2022-04-11T19:40:17Z

Same issue here on M1 Mac

bcranteverfi · 2022-04-11T22:10:01Z

Same issue on M1 Mac. Receiving the following error:

$ minikube service nginx-service

panic: runtime error: index out of range [0] with length 0

goroutine 1 [running]:
k8s.io/minikube/cmd/minikube/cmd.startKicServiceTunnel({0x14000d64d40, 0x1, 0x1}, {0x0, 0x0, 0x0}, {0x1020d9e2e, 0x8}, {0x14000dc089a, 0x6})
        /private/tmp/minikube-20220224-54309-1g1tid5/cmd/minikube/cmd/service.go:205 +0x340
k8s.io/minikube/cmd/minikube/cmd.glob..func35(0x103c25e20, {0x14000d64d40, 0x1, 0x1})
        /private/tmp/minikube-20220224-54309-1g1tid5/cmd/minikube/cmd/service.go:143 +0x594
github.com/spf13/cobra.(*Command).execute(0x103c25e20, {0x14000d64d10, 0x1, 0x1})
        /Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/spf13/cobra@v1.3.0/command.go:860 +0x640
github.com/spf13/cobra.(*Command).ExecuteC(0x103c25920)
        /Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/spf13/cobra@v1.3.0/command.go:974 +0x410
github.com/spf13/cobra.(*Command).Execute(...)
        /Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/spf13/cobra@v1.3.0/command.go:902
k8s.io/minikube/cmd/minikube/cmd.Execute()
        /private/tmp/minikube-20220224-54309-1g1tid5/cmd/minikube/cmd/root.go:157 +0xdb8
main.main()
        /private/tmp/minikube-20220224-54309-1g1tid5/cmd/minikube/main.go:86 +0x2a0

Edit: In case it is helpful, here is my output for minikube start --alsologtostderr -v=4 which I found in the docs.
link to gist

djru · 2022-04-19T04:08:29Z

Issue also happening for me on M1 MBA

AdnanAddy78 · 2022-04-19T12:10:00Z

likewise cant seem to figure out a solution.

vignesh-desk · 2022-04-19T18:54:17Z

minikube service go-service url
🏃 Starting tunnel for service go-service.
❗ Because you are using a Docker driver on darwin, the terminal needs to be open to run it.

Same issue but I start minikube with podman driver, instead of hyperkit

NAME READY STATUS RESTARTS AGE
pod/go-restapi-5bcd754564-8pvfl 1/1 Running 0 9h
pod/go-restapi-5bcd754564-g88hb 1/1 Running 0 9h

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/go-service NodePort x.x.x.x 9090:30120/TCP 9h
service/kubernetes ClusterIP x.x.x.x 443/TCP 8d

NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/go-restapi 2/2 2 2 9h

NAME DESIRED CURRENT READY AGE
replicaset.apps/go-restapi-5bcd754564 2 2 2 9h

Any suggestions ?

asungur · 2022-04-25T19:11:55Z

Try #13788 (comment)
This worked for me

vignesh-desk · 2022-04-26T14:28:58Z

@asungur Thanks
that solved the issue

shabirabasse · 2022-04-29T03:06:24Z

Experiencing the same issue with an m1 mac: ❗ Because you are using a Docker driver on darwin, the terminal needs to be open to run it.
This link didn't work for me.

mayank3srivastava · 2022-05-05T19:43:07Z

Try #13788 (comment)
Worked for me as well

amitk030 · 2022-05-13T11:39:54Z

This #13788 (comment)
worked for me as well.
What does it mean by: the terminal needs to be open to run it. ?

DiegoNovati · 2022-05-18T06:50:43Z

Apple M1 Pro here with the same error message:
❗ Because you are using a Docker driver on darwin, the terminal needs to be open to run it.

It seems that M1 cannot be used

septst · 2022-05-30T19:15:53Z

My-MacBook-Pro ~ % minikube start --driver=hyperkit
😄 minikube v1.25.2 on Darwin 12.4 (arm64)
✨ Using the hyperkit driver based on user configuration

❌ Exiting due to DRV_UNSUPPORTED_OS: The driver 'hyperkit' is not supported on darwin/arm64

Deleting MiniKube and running this didn't help me. Any alternatives?

jonathansds · 2022-06-02T11:16:58Z

I am also having the same issue on my Macbook Pro M1 chip and this comment didn't work for me. Any workaround?

amohsenter09-github · 2022-06-17T10:21:27Z

any updates on this issue or workaround, I am witnessing the same issue on M1 chip.

tkrish · 2022-07-08T08:01:00Z

My case "BigSur V11.16.7; Radeon Pro 555X 4 GB Intel UHD Graphics 630 1536 MB; 2.2 GHz 6-Core Intel Core i7 "
It worked with the below command: (ref documentation#https://minikube.sigs.k8s.io/docs/start/#loadbalancer-deployments)

minikube tunnel [NOTE: Please do not close this terminal as this process must stay alive for the tunnel to be accessible]
EXTERNAL-IP:service-port

tstromberg changed the title ~~No longer able to get service url programmatically on Darwin~~ docker on macOS: No longer able to get service url programmatically Aug 20, 2020

tstromberg added co/docker-driver Issues related to kubernetes in container co/service issues related to the service feature os/macos labels Aug 20, 2020

tstromberg added the triage/needs-information Indicates an issue needs more information in order to work on it. label Aug 20, 2020

medyagh changed the title ~~docker on macOS: No longer able to get service url programmatically~~ add clear documentation that service command on MacOs/Windows needs active terminal seassion Aug 26, 2020

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Feb 7, 2021

medyagh closed this as completed Feb 18, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add clear documentation that service command on MacOs/Windows needs active terminal seassion #9016

add clear documentation that service command on MacOs/Windows needs active terminal seassion #9016

simon-swanson-leapyear commented Aug 18, 2020

tstromberg commented Aug 20, 2020

tstromberg commented Aug 20, 2020

medyagh commented Aug 26, 2020

simon-swanson-leapyear commented Aug 27, 2020

blueelvis commented Oct 28, 2020

dev-michael-schmidt commented Nov 9, 2020

fejta-bot commented Feb 7, 2021

PatrickGeyer commented Mar 7, 2022

prajithsonu524 commented Mar 17, 2022

AndrewAlscher commented Mar 23, 2022

SyedK1987 commented Mar 31, 2022 •

edited

Loading

stumbows commented Apr 5, 2022

bergstenarn commented Apr 6, 2022 •

edited

Loading

arifterzioglu commented Apr 7, 2022 •

edited

Loading

akshaychopra5207 commented Apr 9, 2022

hoani commented Apr 11, 2022

bcranteverfi commented Apr 11, 2022 •

edited

Loading

djru commented Apr 19, 2022

AdnanAddy78 commented Apr 19, 2022

vignesh-desk commented Apr 19, 2022 •

edited

Loading

asungur commented Apr 25, 2022 •

edited

Loading

vignesh-desk commented Apr 26, 2022 •

edited

Loading

shabirabasse commented Apr 29, 2022

mayank3srivastava commented May 5, 2022

amitk030 commented May 13, 2022 •

edited

Loading

DiegoNovati commented May 18, 2022

septst commented May 30, 2022

jonathansds commented Jun 2, 2022

amohsenter09-github commented Jun 17, 2022

tkrish commented Jul 8, 2022 •

edited

Loading

add clear documentation that service command on MacOs/Windows needs active terminal seassion #9016

add clear documentation that service command on MacOs/Windows needs active terminal seassion #9016

Comments

simon-swanson-leapyear commented Aug 18, 2020

System

Steps to Reproduce

Expected

Actual

tstromberg commented Aug 20, 2020

tstromberg commented Aug 20, 2020

medyagh commented Aug 26, 2020

simon-swanson-leapyear commented Aug 27, 2020

blueelvis commented Oct 28, 2020

dev-michael-schmidt commented Nov 9, 2020

fejta-bot commented Feb 7, 2021

PatrickGeyer commented Mar 7, 2022

prajithsonu524 commented Mar 17, 2022

AndrewAlscher commented Mar 23, 2022

SyedK1987 commented Mar 31, 2022 • edited Loading

stumbows commented Apr 5, 2022

bergstenarn commented Apr 6, 2022 • edited Loading

arifterzioglu commented Apr 7, 2022 • edited Loading

akshaychopra5207 commented Apr 9, 2022

hoani commented Apr 11, 2022

bcranteverfi commented Apr 11, 2022 • edited Loading

djru commented Apr 19, 2022

AdnanAddy78 commented Apr 19, 2022

vignesh-desk commented Apr 19, 2022 • edited Loading

asungur commented Apr 25, 2022 • edited Loading

vignesh-desk commented Apr 26, 2022 • edited Loading

shabirabasse commented Apr 29, 2022

mayank3srivastava commented May 5, 2022

amitk030 commented May 13, 2022 • edited Loading

DiegoNovati commented May 18, 2022

septst commented May 30, 2022

jonathansds commented Jun 2, 2022

amohsenter09-github commented Jun 17, 2022

tkrish commented Jul 8, 2022 • edited Loading

SyedK1987 commented Mar 31, 2022 •

edited

Loading

bergstenarn commented Apr 6, 2022 •

edited

Loading

arifterzioglu commented Apr 7, 2022 •

edited

Loading

bcranteverfi commented Apr 11, 2022 •

edited

Loading

vignesh-desk commented Apr 19, 2022 •

edited

Loading

asungur commented Apr 25, 2022 •

edited

Loading

vignesh-desk commented Apr 26, 2022 •

edited

Loading

amitk030 commented May 13, 2022 •

edited

Loading

tkrish commented Jul 8, 2022 •

edited

Loading