minikube start: kubectl info: exec: exit status 2 #9976

bl-ue · 2020-12-16T16:32:22Z

Steps to reproduce the issue:

minikube start --vm=true --kubernetes-version='v1.20.0'

Full output of failed command:

$ minikube start --vm=true --kubernetes-version='v1.20.0'
😄  minikube v1.15.1 on Darwin 11.1
✨  Automatically selected the hyperkit driver. Other choices: vmware, vmwarefusion
👍  Starting control plane node minikube in cluster minikube
🔥  Creating hyperkit VM (CPUs=2, Memory=6000MB, Disk=20000MB) ...
🐳  Preparing Kubernetes v1.20.0 on Docker 19.03.13 ...
🔎  Verifying Kubernetes components...
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
E1216 11:24:31.467901   25710 start.go:266] kubectl info: exec: exit status 2
$

--alsologtostderr didn't help clarify this error. Last few lines of run with --alsologtostderr:

I1216 11:28:23.304254   25968 start.go:203] waiting for startup goroutines ...
I1216 11:28:23.408164   25968 out.go:110] 🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
E1216 11:28:23.408190   25968 start.go:266] kubectl info: exec: exit status 2
$

If neccessary I can attach the full log with --alsologtostderr.

Optional: Full output of minikube logs command:
logs.zip

The text was updated successfully, but these errors were encountered:

bl-ue · 2020-12-16T18:05:53Z

The log line is:

minikube/cmd/minikube/cmd/start.go

Line 271 in fdab252

klog.Errorf("kubectl info: %v", err)

minikube is trying to exec /usr/local/bin/kubectl version --client --output=json, but getting error 2 aka "No such file or directory".

kubectl is linked to minikube i.e.

lrwxr-xr-x 1 me admin 23 Dec 13 09:10 /usr/local/bin/kubectl -> /usr/local/bin/minikube*

bl-ue · 2020-12-16T18:08:01Z

Installing the Homebrew formula kubernetes-cli which provides kubectl binary and replacing the link with the actual binary fixes the issue.

afbjorklund · 2020-12-17T18:59:10Z

It is supposed to be possible to run "minikube kubectl" using a symlink like that.

Looks like something went wrong, when calling the actual kubectl binary ?

Try something like:

minikube --alsologtostderr kubectl version

It should have more info.

Like using and Running

afbjorklund · 2020-12-17T19:02:49Z

The command (from start) works OK on linux:

$ kubectl version --client --output=json
{
  "clientVersion": {
    "major": "1",
    "minor": "20",
    "gitVersion": "v1.20.0",
    "gitCommit": "af46c47ce925f4c4ad5cc8d1fca46c7b77d13b38",
    "gitTreeState": "clean",
    "buildDate": "2020-12-08T17:59:43Z",
    "goVersion": "go1.15.5",
    "compiler": "gc",
    "platform": "linux/amd64"
  }
}

Alternative invocation (without symlink):

minikube kubectl -- version --client --output=json

It should not return "exit status 2"

bl-ue · 2020-12-17T19:20:10Z

@afbjorklund Relevent logs from minikube --alsologtostderr kubectl version (can add more):

I1217 14:17:55.964853   62803 binary.go:56] Not caching binary, using https://storage.googleapis.com/kubernetes-release/release/v1.20.0/bin/darwin/amd64/kubectl?checksum=file:https://storage.googleapis.com/kubernetes-release/release/v1.20.0/bin/darwin/amd64/kubectl.sha256
I1217 14:17:55.964885   62803 kubectl.go:51] Running /Users/me/.minikube/cache/darwin/v1.20.0/kubectl [version]
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.0", GitCommit:"af46c47ce925f4c4ad5cc8d1fca46c7b77d13b38", GitTreeState:"clean", BuildDate:"2020-12-08T17:59:43Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.0", GitCommit:"af46c47ce925f4c4ad5cc8d1fca46c7b77d13b38", GitTreeState:"clean", BuildDate:"2020-12-08T17:51:19Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}

afbjorklund · 2020-12-17T19:22:06Z

Hmm, that command looks happy enough. How did we end up with the error ?

kubectl info: exec: exit status 2

bl-ue · 2020-12-17T19:33:32Z

The following simple program works, with ./symlink linking to /usr/local/bin/minikube:

package main

import (
	"fmt"
	"os/exec"
)

func main() {
	out, err := exec.Command("./symlink").Output()
	if err != nil {
		fmt.Printf("%v", err)
		return
	}
	fmt.Printf(string(out))
}

Let me investigate.

bl-ue · 2021-01-04T19:05:02Z

This is certainly some kind of bug.
/remove-kind support
/kind bug

ilya-zuyev · 2021-02-17T19:58:22Z

Hi @bl-ue! Any update on this issue?

bl-ue · 2021-02-17T20:01:09Z

Oh, I forgot about this one. Let me see...

medyagh · 2021-03-03T19:56:44Z

Hi @bl-ue c, I haven't heard back from you, I wonder if you still have this issue?
Regrettably, there isn't enough information in this issue to make it actionable, and a long enough duration has passed, so this issue is likely difficult to replicate.

I will close this issue for now but please feel free to reopen whenever you feel ready to provide more information.

bl-ue · 2021-03-03T20:03:25Z

Hi @medyagh! Yes, I'm still having the issue and it's still repeatable using the steps given above. I haven't had much time lately to investigate, and I also wouldn't consider myself a Go expert—therefore I haven't been able to take care of it.

bl-ue · 2021-03-03T20:03:45Z

I will close this issue for now but please feel free to reopen whenever you feel ready to provide more information.

I can't reopen this issue, because once a collaborator closes it, the OP can't reopen it 😕

bl-ue · 2021-03-03T20:07:33Z

FYI—it's not imparing usablity in the slightest bit; but the I opened an issue about it because it's a defect in the product, but a slight, hidden defect that's not likely to be noticed.

prezha · 2021-03-03T22:22:18Z

/reopen

@bl-ue we can reopen if you have time to continue working on this issue with us - thanks!

can you please confirm (#9976 (comment)):

kubectl is linked to minikube i.e.
lrwxr-xr-x 1 me admin 23 Dec 13 09:10 /usr/local/bin/kubectl -> /usr/local/bin/minikube*

this does not look right

~~also (#9976 (comment)):~~

Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.0", GitCommit:"af46c47ce925f4c4ad5cc8d1fca46c7b77d13b38", GitTreeState:"clean", BuildDate:"2020-12-08T17:59:43Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.0", GitCommit:"af46c47ce925f4c4ad5cc8d1fca46c7b77d13b38", GitTreeState:"clean", BuildDate:"2020-12-08T17:51:19Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}

~~it seems that you have kubectl for two different platforms: darwin/amd64 (client) and linux/amd64 (server) - is that correct?~~
pls ignore this 2nd one (minikube v1.15.1 on Darwin 11.1) - it is ok, I was looking at another issue...
have you tried to replicate the issue with the latest minikube v1.18?

k8s-ci-robot · 2021-03-03T22:22:26Z

@prezha: Reopened this issue.

In response to this:

/reopen
@bl-ue we can reopen if you have time to continue working on this issue with us - thanks!

can you please confirm (#9976 (comment)):
kubectl is linked to minikube i.e.
lrwxr-xr-x 1 me admin 23 Dec 13 09:10 /usr/local/bin/kubectl -> /usr/local/bin/minikube*
this does not look right

also (#9976 (comment)):
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.0", GitCommit:"af46c47ce925f4c4ad5cc8d1fca46c7b77d13b38", GitTreeState:"clean", BuildDate:"2020-12-08T17:59:43Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.0", GitCommit:"af46c47ce925f4c4ad5cc8d1fca46c7b77d13b38", GitTreeState:"clean", BuildDate:"2020-12-08T17:51:19Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
it seems that you have kubectl for two different platforms: darwin/amd64 and linux/amd64 - probably one is not good

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

bl-ue · 2021-03-03T22:42:21Z

Okay @prezha I'll take a look. 👍🏻

afbjorklund · 2021-03-04T07:20:03Z

It's not possible to reproduce this issue on linux, but it would be nice to re-visit after fixing #9593

There you would have a linux-amd64 client talking to a linux-arm64 server, maybe similar issues ?

bl-ue · 2021-03-04T15:18:41Z

So I tried it again, here's the output:

$ rm /usr/local/bin/kubectl
$ ln -s /usr/local/bin/minikube /usr/local/bin/kubectl
$ readlink /usr/local/bin/kubectl
/usr/local/bin/minikube
$ /usr/local/bin/minikube start --vm=true --kubernetes-version='v1.20.4'
😄  minikube v1.18.0 on Darwin 11.2.2
✨  Automatically selected the hyperkit driver. Other choices: vmware, ssh
👍  Starting control plane node minikube in cluster minikube
🔥  Creating hyperkit VM (CPUs=2, Memory=6000MB, Disk=20000MB) ...
🐳  Preparing Kubernetes v1.20.4 on Docker 20.10.3 ...
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v4
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
E0304 10:14:53.683681   84151 start.go:270] kubectl info: exec: exit status 2
$

I was doing some experimenting a few months ago but I didn't come to the slightest conclusion other than the symlinking is causing an issue, because kubectl being it's own binary works fine without this error.

prezha · 2021-03-04T15:34:12Z

@bl-ue thank you for your reply

i'm still a bit confused why are you linking kubectl to minikube - the error seems to root to then when, at the end (as shown in your logs), calling effectively minikube with version --client --output=json params, which are for kubectl and would not work for minikube:

minikube/cmd/minikube/cmd/start.go

Lines 269 to 271 in 18e215f

    
           if err := showKubectlInfo(kubeconfig, starter.Node.KubernetesVersion, starter.Cfg.Name); err != nil { 
        
           	klog.Errorf("kubectl info: %v", err) 
        
           }

->

minikube/cmd/minikube/cmd/start.go

Lines 448 to 451 in 18e215f

    
           gitVersion, err := kubectlVersion(path) 
        
           if err != nil { 
        
           	return err 
        
           }

->

minikube/cmd/minikube/cmd/start.go

Lines 520 to 526 in 18e215f

    
           j, err := exec.Command(path, "version", "--client", "--output=json").Output() 
        
           if err != nil { 
        
           	// really old Kubernetes clients did not have the --output parameter 
        
           	b, err := exec.Command(path, "version", "--client", "--short").Output() 
        
           	if err != nil { 
        
           		return "", errors.Wrap(err, "exec") 
        
           	}

why do you link them?

can you please try w/o linking and see if you still have the issue

bl-ue · 2021-03-04T15:43:16Z

why do you link them?

@prezha I'm linking them per here: https://minikube.sigs.k8s.io/docs/handbook/kubectl/

Quoting:

Alternatively, you can create a symbolic link to minikube’s binary named ‘kubectl’.
ln -s $(which minikube) /usr/local/bin/kubectl

can you please try w/o linking and see if you still have the issue

Yep. I don't have the issue when kubectl is an actual binary.

prezha · 2021-03-04T15:50:38Z

thanks for confirming, that's great, and also for referencing where the linking part comes from :)

there, if you don't have (or don't want to download) kubectl, alias kubectl="minikube kubectl --" would work just fine, but tbh, i'm not sure what was the idea in the handbook with linking it

bl-ue · 2021-03-04T16:04:41Z

Sure. I'll just use the binary. But I just verified: this does not happen when I specify --kubernetes-version='v1.19.0' 🤔

prezha · 2021-03-04T16:12:52Z

great

you can also use the alias, that should work too
this specific code was introduced about a year ago
now, with the link, if you specify any minikube-acceptable command/flag (like --kubernetes-version='v1.19.0') - it will work, but i don't expect it to work with any command/flag specific to kubectl (eg, get nodes -A)

if you don't mind we can close this issue now and check separately about the advice given in the handbook

bl-ue · 2021-03-04T16:20:14Z

No, I don't mind at all. Thank you for your help @prezha! ❤️

It's a little odd that it started with k8s v1.20, but...:shrug:

redolf250 · 2023-01-12T21:19:39Z

minikube start guves this error. it was working fine and all of the sudden it started giving this error

😄 minikube v1.28.0 on Microsoft Windows 10 Pro 10.0.19044 Build 19044
✨ Using the docker driver based on existing profile
👍 Starting control plane node minikube in cluster minikube
🚜 Pulling base image ...
🤷 docker "minikube" container is missing, will recreate.
🔥 Creating docker container (CPUs=2, Memory=3000MB) ...
❗ This container is having trouble accessing https://registry.k8s.io
💡 To pull new external images, you may need to configure a proxy: https://minikube.sigs.k8s.io/docs/reference/networking/proxy/
🐳 Preparing Kubernetes v1.25.3 on Docker 20.10.20 ...
🔎 Verifying Kubernetes components...
▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
▪ Using image docker.io/kubernetesui/dashboard:v2.7.0
▪ Using image docker.io/kubernetesui/metrics-scraper:v1.0.8
💡 Some dashboard features require the metrics-server addon. To enable all features please run:

    minikube addons enable metrics-server

🌟 Enabled addons: storage-provisioner, default-storageclass, dashboard
🏄 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
E0112 21:10:09.077945 4792 start.go:276] kubectl info: exec: exit status 0xffffffff

priyawadhwa added the kind/support Categorizes issue or PR as a support question. label Dec 28, 2020

k8s-ci-robot added kind/bug Categorizes issue or PR as related to a bug. and removed kind/support Categorizes issue or PR as a support question. labels Jan 4, 2021

tstromberg added the cmd/kubectl Issues with the "kubectl" command label Jan 6, 2021

tstromberg changed the title ~~minikube start - E1216 start.go:266] kubectl info: exec: exit status 2~~ minikube start: kubectl info: exec: exit status 2 Jan 6, 2021

tstromberg added the priority/awaiting-more-evidence Lowest priority. Possibly useful, but not yet enough support to actually get it done. label Jan 6, 2021

medyagh closed this as completed Mar 3, 2021

k8s-ci-robot reopened this Mar 3, 2021

bl-ue closed this as completed Mar 4, 2021

prezha mentioned this issue Mar 4, 2021

website/handbook: don't advise users to create a symbolic link to minikube’s binary named ‘kubectl’ #10717

Closed

fardjad mentioned this issue Dec 3, 2021

minikube status command - always gets an error fardjad/node-minikube-docker-port-mapper#1

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

minikube start: kubectl info: exec: exit status 2 #9976

minikube start: kubectl info: exec: exit status 2 #9976

bl-ue commented Dec 16, 2020 •

edited

Loading

bl-ue commented Dec 16, 2020

bl-ue commented Dec 16, 2020

afbjorklund commented Dec 17, 2020

afbjorklund commented Dec 17, 2020

bl-ue commented Dec 17, 2020

afbjorklund commented Dec 17, 2020

bl-ue commented Dec 17, 2020

bl-ue commented Jan 4, 2021 •

edited

Loading

ilya-zuyev commented Feb 17, 2021

bl-ue commented Feb 17, 2021 •

edited

Loading

medyagh commented Mar 3, 2021

bl-ue commented Mar 3, 2021

bl-ue commented Mar 3, 2021

bl-ue commented Mar 3, 2021

prezha commented Mar 3, 2021 •

edited

Loading

k8s-ci-robot commented Mar 3, 2021

bl-ue commented Mar 3, 2021

afbjorklund commented Mar 4, 2021

bl-ue commented Mar 4, 2021

prezha commented Mar 4, 2021 •

edited

Loading

bl-ue commented Mar 4, 2021 •

edited

Loading

prezha commented Mar 4, 2021

bl-ue commented Mar 4, 2021 •

edited

Loading

prezha commented Mar 4, 2021

bl-ue commented Mar 4, 2021

redolf250 commented Jan 12, 2023

minikube start: kubectl info: exec: exit status 2 #9976

minikube start: kubectl info: exec: exit status 2 #9976

Comments

bl-ue commented Dec 16, 2020 • edited Loading

bl-ue commented Dec 16, 2020

bl-ue commented Dec 16, 2020

afbjorklund commented Dec 17, 2020

afbjorklund commented Dec 17, 2020

bl-ue commented Dec 17, 2020

afbjorklund commented Dec 17, 2020

bl-ue commented Dec 17, 2020

bl-ue commented Jan 4, 2021 • edited Loading

ilya-zuyev commented Feb 17, 2021

bl-ue commented Feb 17, 2021 • edited Loading

medyagh commented Mar 3, 2021

bl-ue commented Mar 3, 2021

bl-ue commented Mar 3, 2021

bl-ue commented Mar 3, 2021

prezha commented Mar 3, 2021 • edited Loading

k8s-ci-robot commented Mar 3, 2021

bl-ue commented Mar 3, 2021

afbjorklund commented Mar 4, 2021

bl-ue commented Mar 4, 2021

prezha commented Mar 4, 2021 • edited Loading

bl-ue commented Mar 4, 2021 • edited Loading

prezha commented Mar 4, 2021

bl-ue commented Mar 4, 2021 • edited Loading

prezha commented Mar 4, 2021

bl-ue commented Mar 4, 2021

redolf250 commented Jan 12, 2023

bl-ue commented Dec 16, 2020 •

edited

Loading

bl-ue commented Jan 4, 2021 •

edited

Loading

bl-ue commented Feb 17, 2021 •

edited

Loading

prezha commented Mar 3, 2021 •

edited

Loading

prezha commented Mar 4, 2021 •

edited

Loading

bl-ue commented Mar 4, 2021 •

edited

Loading

bl-ue commented Mar 4, 2021 •

edited

Loading