Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reword error message when cluster state exit code is not found #9390

Merged
merged 3 commits into from
Nov 10, 2020

Conversation

prezha
Copy link
Contributor

@prezha prezha commented Oct 4, 2020

fixes: #9374
fixes #8933

as described by @medyagh in issue #9374, json output contains unable to convert exit code to int: strconv.Atoi: parsing "": invalid syntax error messages while the container is starting, and they should be treated as non-errors (consequentially not showing in json output)

original output (before pr) - example:

medya@cloudshell:~/campwiz2$ minikube status -o=json -l=cluster | jq .StatusName
E1001 22:38:25.301197  184671 status.go:533] unable to convert exit code to int: strconv.Atoi: parsing "": invalid syntax
E1001 22:38:25.302112  184671 status.go:533] unable to convert exit code to int: strconv.Atoi: parsing "": invalid syntax
E1001 22:38:25.302145  184671 status.go:533] unable to convert exit code to int: strconv.Atoi: parsing "": invalid syntax

new output (after pr):

  • minikube status -o=json -l=cluster | jq .StatusName does not contain this error
  • minikube status -o=json -l=cluster --alsologtostderr would log it as message from data map (transient code 100 ("Starting"): Waiting for container(s) to start (exitcode=""))

edit: i wasn't able to replicate case when data["exitcode"] was set to "", just the complete absence of data["exitcode"] (that manifests the same like "" value in maps) - eg:

data[message:❗  docker is currently using the btrfs storage driver, consider switching to overlay2 for better performance]
data[message:❗  Multi-node clusters are currently experimental and might exhibit unintended behavior.]

therefore i've amended pr to, in case that data["exitcode"] is set to "" or is unset (which boils down again to be "", just being explicit), print warning entry with the data map instead of the specific message (and potentially wrong/misleading message i've initially mentioned above - ie, waiting for container to start...)

  • example outputs:

-- with default (docker) driver:

I1005 03:18:10.947344  387936 system_pods.go:161] Checking kubelet status ...
I1005 03:18:10.947464  387936 ssh_runner.go:148] Run: sudo systemctl is-active --quiet service kubelet
I1005 03:18:10.964020  387936 status.go:338] minikube-m03 kubelet status = Running
I1005 03:18:10.964046  387936 status.go:213] minikube-m03 status: &{Name:minikube-m03 Host:Running Kubelet:Running APIServer:Irrelevant Kubeconfig:Irrelevant Worker:true}
I1005 03:18:10.964918  387936 status.go:522] transient code 100 ("Starting") for step: map[currentstep:0 message:😄  minikube v1.13.1 on "Opensuse-Tumbleweed" name:Initial Minikube Setup totalsteps:12]
I1005 03:18:10.964964  387936 status.go:522] transient code 100 ("Starting") for step: map[currentstep:1 message:✨  Automatically selected the docker driver name:Selecting Driver totalsteps:12]
W1005 03:18:10.964983  387936 status.go:532] transient code 100 ("Starting"): map[message:❗  docker is currently using the btrfs storage driver, consider switching to overlay2 for better performance]
I1005 03:18:10.965007  387936 status.go:522] transient code 100 ("Starting") for step: map[currentstep:3 message:👍  Starting control plane node minikube in cluster minikube name:Starting Node totalsteps:12]
I1005 03:18:10.965031  387936 status.go:522] transient code 100 ("Starting") for step: map[currentstep:6 message:🔥  Creating docker container (CPUs=8, Memory=16384MB) ... name:Creating Container totalsteps:12]
I1005 03:18:10.965061  387936 status.go:522] transient code 100 ("Starting") for step: map[currentstep:8 message:🐳  Preparing Kubernetes v1.19.2 on Docker 19.03.8 ... name:Preparing Kubernetes totalsteps:12]
I1005 03:18:10.965090  387936 status.go:522] transient code 100 ("Starting") for step: map[currentstep:10 message:🔎  Verifying Kubernetes components... name:Verifying Kubernetes totalsteps:12]
I1005 03:18:10.965114  387936 status.go:522] transient code 100 ("Starting") for step: map[currentstep:11 message:🌟  Enabled addons: storage-provisioner, default-storageclass name:Enabling Addons totalsteps:12]
W1005 03:18:10.965131  387936 status.go:532] transient code 100 ("Starting"): map[message:❗  Multi-node clusters are currently experimental and might exhibit unintended behavior.]
I1005 03:18:10.965152  387936 status.go:522] transient code 100 ("Starting") for step: map[currentstep:11 message:📘  To track progress on multi-node clusters, see https://github.com/kubernetes/minikube/issues/7538. name:Enabling Addons totalsteps:12]
I1005 03:18:10.965175  387936 status.go:522] transient code 100 ("Starting") for step: map[currentstep:3 message:👍  Starting node minikube-m02 in cluster minikube name:Starting Node totalsteps:12]
I1005 03:18:10.965199  387936 status.go:522] transient code 100 ("Starting") for step: map[currentstep:6 message:🔥  Creating docker container (CPUs=8, Memory=16384MB) ... name:Creating Container totalsteps:12]
I1005 03:18:10.965223  387936 status.go:522] transient code 100 ("Starting") for step: map[currentstep:6 message:🌐  Found network options: name:Creating Container totalsteps:12]
I1005 03:18:10.965247  387936 status.go:522] transient code 100 ("Starting") for step: map[currentstep:8 message:🐳  Preparing Kubernetes v1.19.2 on Docker 19.03.8 ... name:Preparing Kubernetes totalsteps:12]
I1005 03:18:10.965271  387936 status.go:522] transient code 100 ("Starting") for step: map[currentstep:10 message:🔎  Verifying Kubernetes components... name:Verifying Kubernetes totalsteps:12]
I1005 03:18:10.965295  387936 status.go:522] transient code 100 ("Starting") for step: map[currentstep:3 message:👍  Starting node minikube-m03 in cluster minikube name:Starting Node totalsteps:12]
I1005 03:18:10.965319  387936 status.go:522] transient code 100 ("Starting") for step: map[currentstep:6 message:🔥  Creating docker container (CPUs=8, Memory=16384MB) ... name:Creating Container totalsteps:12]
I1005 03:18:10.965342  387936 status.go:522] transient code 100 ("Starting") for step: map[currentstep:6 message:🌐  Found network options: name:Creating Container totalsteps:12]
I1005 03:18:10.965365  387936 status.go:522] transient code 100 ("Starting") for step: map[currentstep:8 message:🐳  Preparing Kubernetes v1.19.2 on Docker 19.03.8 ... name:Preparing Kubernetes totalsteps:12]
I1005 03:18:10.965391  387936 status.go:522] transient code 100 ("Starting") for step: map[currentstep:10 message:🔎  Verifying Kubernetes components... name:Verifying Kubernetes totalsteps:12]
I1005 03:18:10.965416  387936 status.go:522] transient code 0 ("") for step: map[currentstep:12 message:🏄  Done! kubectl is now configured to use "minikube" by default name:Done totalsteps:12]
{"Name":"minikube","StatusCode":200,"StatusName":"OK","Step":"Done","StepDetail":"🏄  Done! kubectl is now configured to use \"minikube\" by default","BinaryVersion":"v1.13.1","Components":{"kubeconfig":{"Name":"kubeconfig","StatusCode":200,"StatusName":""}},"Nodes":[{"Name":"minikube","StatusCode":200,"StatusName":"OK","Components":{"apiserver":{"Name":"apiserver","StatusCode":200,"StatusName":"OK"},"kubelet":{"Name":"kubelet","StatusCode":200,"StatusName":"OK"}}},{"Name":"minikube-m02","StatusCode":200,"StatusName":"OK","Components":{"kubelet":{"Name":"kubelet","StatusCode":200,"StatusName":"OK"}}},{"Name":"minikube-m03","StatusCode":200,"StatusName":"OK","Components":{"kubelet":{"Name":"kubelet","StatusCode":200,"StatusName":"OK"}}}]}

-- with kvm2 driver:

1005 03:14:16.542353  356179 system_pods.go:161] Checking kubelet status ...
I1005 03:14:16.542420  356179 ssh_runner.go:148] Run: sudo systemctl is-active --quiet service kubelet
I1005 03:14:16.557881  356179 status.go:338] minikube-m03 kubelet status = Running
I1005 03:14:16.557919  356179 status.go:213] minikube-m03 status: &{Name:minikube-m03 Host:Running Kubelet:Running APIServer:Irrelevant Kubeconfig:Irrelevant Worker:true}
I1005 03:14:16.559581  356179 status.go:522] transient code 100 ("Starting") for step: map[currentstep:0 message:😄  minikube v1.13.1 on "Opensuse-Tumbleweed" name:Initial Minikube Setup totalsteps:12]
I1005 03:14:16.559684  356179 status.go:522] transient code 100 ("Starting") for step: map[currentstep:1 message:✨  Using the kvm2 driver based on user configuration name:Selecting Driver totalsteps:12]
I1005 03:14:16.559759  356179 status.go:522] transient code 100 ("Starting") for step: map[currentstep:1 message:💿  Downloading VM boot image ... name:Selecting Driver totalsteps:12]
I1005 03:14:16.559826  356179 status.go:522] transient code 100 ("Starting") for step: map[currentstep:3 message:👍  Starting control plane node minikube in cluster minikube name:Starting Node totalsteps:12]
I1005 03:14:16.559892  356179 status.go:522] transient code 100 ("Starting") for step: map[currentstep:3 message:💾  Downloading Kubernetes v1.19.2 preload ... name:Starting Node totalsteps:12]
I1005 03:14:16.559956  356179 status.go:522] transient code 100 ("Starting") for step: map[currentstep:7 message:🔥  Creating kvm2 VM (CPUs=8, Memory=16384MB, Disk=32768MB) ... name:Creating VM totalsteps:12]
I1005 03:14:16.560020  356179 status.go:522] transient code 100 ("Starting") for step: map[currentstep:8 message:🐳  Preparing Kubernetes v1.19.2 on Docker 19.03.12 ... name:Preparing Kubernetes totalsteps:12]
I1005 03:14:16.560083  356179 status.go:522] transient code 100 ("Starting") for step: map[currentstep:10 message:🔎  Verifying Kubernetes components... name:Verifying Kubernetes totalsteps:12]
I1005 03:14:16.560153  356179 status.go:522] transient code 100 ("Starting") for step: map[currentstep:11 message:🌟  Enabled addons: storage-provisioner, default-storageclass name:Enabling Addons totalsteps:12]
W1005 03:14:16.560207  356179 status.go:532] transient code 100 ("Starting"): map[message:❗  Multi-node clusters are currently experimental and might exhibit unintended behavior.]
I1005 03:14:16.560269  356179 status.go:522] transient code 100 ("Starting") for step: map[currentstep:11 message:📘  To track progress on multi-node clusters, see https://github.com/kubernetes/minikube/issues/7538. name:Enabling Addons totalsteps:12]
I1005 03:14:16.560332  356179 status.go:522] transient code 100 ("Starting") for step: map[currentstep:3 message:👍  Starting node minikube-m02 in cluster minikube name:Starting Node totalsteps:12]
I1005 03:14:16.560400  356179 status.go:522] transient code 100 ("Starting") for step: map[currentstep:7 message:🔥  Creating kvm2 VM (CPUs=8, Memory=16384MB, Disk=32768MB) ... name:Creating VM totalsteps:12]
I1005 03:14:16.560464  356179 status.go:522] transient code 100 ("Starting") for step: map[currentstep:7 message:🌐  Found network options: name:Creating VM totalsteps:12]
I1005 03:14:16.560528  356179 status.go:522] transient code 100 ("Starting") for step: map[currentstep:8 message:🐳  Preparing Kubernetes v1.19.2 on Docker 19.03.12 ... name:Preparing Kubernetes totalsteps:12]
I1005 03:14:16.560592  356179 status.go:522] transient code 100 ("Starting") for step: map[currentstep:10 message:🔎  Verifying Kubernetes components... name:Verifying Kubernetes totalsteps:12]
I1005 03:14:16.560659  356179 status.go:522] transient code 100 ("Starting") for step: map[currentstep:3 message:👍  Starting node minikube-m03 in cluster minikube name:Starting Node totalsteps:12]
I1005 03:14:16.560723  356179 status.go:522] transient code 100 ("Starting") for step: map[currentstep:7 message:🔥  Creating kvm2 VM (CPUs=8, Memory=16384MB, Disk=32768MB) ... name:Creating VM totalsteps:12]
I1005 03:14:16.560786  356179 status.go:522] transient code 100 ("Starting") for step: map[currentstep:7 message:🌐  Found network options: name:Creating VM totalsteps:12]
I1005 03:14:16.560850  356179 status.go:522] transient code 100 ("Starting") for step: map[currentstep:8 message:🐳  Preparing Kubernetes v1.19.2 on Docker 19.03.12 ... name:Preparing Kubernetes totalsteps:12]
I1005 03:14:16.560913  356179 status.go:522] transient code 100 ("Starting") for step: map[currentstep:10 message:🔎  Verifying Kubernetes components... name:Verifying Kubernetes totalsteps:12]
I1005 03:14:16.560980  356179 status.go:522] transient code 0 ("") for step: map[currentstep:12 message:🏄  Done! kubectl is now configured to use "minikube" by default name:Done totalsteps:12]
W1005 03:14:16.561017  356179 status.go:552] event stream is too old (2020-10-05 03:02:27.515235027 +0100 BST) to be considered a transient state
{"Name":"minikube","StatusCode":200,"StatusName":"OK","BinaryVersion":"v1.13.1","Components":{"kubeconfig":{"Name":"kubeconfig","StatusCode":200,"StatusName":""}},"Nodes":[{"Name":"minikube","StatusCode":200,"StatusName":"OK","Components":{"apiserver":{"Name":"apiserver","StatusCode":200,"StatusName":"OK"},"kubelet":{"Name":"kubelet","StatusCode":200,"StatusName":"OK"}}},{"Name":"minikube-m02","StatusCode":200,"StatusName":"OK","Components":{"kubelet":{"Name":"kubelet","StatusCode":200,"StatusName":"OK"}}},{"Name":"minikube-m03","StatusCode":200,"StatusName":"OK","Components":{"kubelet":{"Name":"kubelet","StatusCode":200,"StatusName":"OK"}}}]}

@k8s-ci-robot k8s-ci-robot added do-not-merge/invalid-commit-message Indicates that a PR should not merge because it has an invalid commit message. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Oct 4, 2020
@k8s-ci-robot
Copy link
Contributor

Hi @prezha. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Oct 4, 2020
@k8s-ci-robot k8s-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Oct 4, 2020
@minikube-bot
Copy link
Collaborator

Can one of the admins verify this patch?

@prezha
Copy link
Contributor Author

prezha commented Oct 5, 2020

/assign @priyawadhwa

Copy link

@priyawadhwa priyawadhwa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@priyawadhwa
Copy link

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Oct 5, 2020
@minikube-pr-bot
Copy link

kvm2 Driver
error collecting results for kvm2 driver: timing run 0 with Minikube (PR 9390): timing cmd: [/home/performance-monitor/.minikube/minikube-binaries/9390/minikube start --driver=kvm2]: starting cmd: fork/exec /home/performance-monitor/.minikube/minikube-binaries/9390/minikube: exec format error
docker Driver
error collecting results for docker driver: timing run 0 with Minikube (PR 9390): timing cmd: [/home/performance-monitor/.minikube/minikube-binaries/9390/minikube start --driver=docker]: starting cmd: fork/exec /home/performance-monitor/.minikube/minikube-binaries/9390/minikube: exec format error

Copy link
Member

@medyagh medyagh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry for this, but I belive I created the issue on json pollution , dont think that is an issue

@prezha
Copy link
Contributor Author

prezha commented Oct 6, 2020

no worries @medyagh: i believe those are actual warning messages that came through (some examples are given in the pr description above), ie, not related to a container being created/starting up?

@medyagh
Copy link
Member

medyagh commented Oct 6, 2020

no worries @medyagh: i believe those are actual warning messages that came through (some examples are given in the pr description above), ie, not related to a container being created/starting up?

what I got wrong is, We explicitly called out that stderr will always have unexpected artifacts, like panic's, in the design doc

I think we had expected the user to parse only the stdout for json and ignore the stderr

@prezha
Copy link
Contributor Author

prezha commented Oct 6, 2020

ok, that makes perfect sense for a requested json format output
so, we can either fix that (ie, json output ignoring any /unstructured/ stderr messages) and/or convert these specific ones to the format that is 'digestible' in expected json format (tbh, i've done the latter unintentionally in my second commit - before this one, by returning complete data map structure that contained only a message field, but not the exitcode as it was not provided in these specific cases, which in turn would cause the json output pollution)
what you think would be the best approach here @medyagh?

@k8s-ci-robot k8s-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. approved Indicates a PR has been approved by an approver from all required OWNERS files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Oct 13, 2020
@minikube-pr-bot
Copy link

kvm2 Driver
Times for minikube: 62.1s 59.0s 62.5s
Average time for minikube: 61.2s

Times for Minikube (PR 9390): 24.0s 62.0s 59.7s
Average time for Minikube (PR 9390): 48.6s

Averages Time Per Log

+--------------------------------+----------+--------------------+
|              LOG               | MINIKUBE | MINIKUBE (PR 9390) |
+--------------------------------+----------+--------------------+
| * minikube v1.14.0 on Debian   | 0.1s     |                    |
|                           9.11 |          |                    |
| * Using the kvm2 driver based  | 0.0s     | 0.0s               |
| on user configuration          |          |                    |
| * Starting control plane node  | 0.0s     | 0.0s               |
| minikube in cluster minikube   |          |                    |
| * Creating kvm2 VM (CPUs=2,    | 36.2s    | 23.6s              |
| Memory=3700MB, Disk=20000MB)   |          |                    |
| ...                            |          |                    |
| * Preparing Kubernetes v1.19.2 | 23.0s    | 23.0s              |
| on Docker 19.03.12 ...         |          |                    |
| * Verifying Kubernetes         | 1.6s     | 1.5s               |
| components...                  |          |                    |
| * Enabled addons:              | 0.3s     | 0.4s               |
| storage-provisioner,           |          |                    |
| default-storageclass           |          |                    |
|                                | 0.0s     | 0.0s               |
| * Want kubectl v1.19.2? Try    |          |                    |
| 'minikube kubectl -- get pods  |          |                    |
| -A'                            |          |                    |
| * Done! kubectl is now         |          |                    |
| configured to use "minikube"   |          |                    |
| by default                     |          |                    |
+--------------------------------+----------+--------------------+

docker Driver
Times for minikube: 31.0s 28.9s 28.8s
Average time for minikube: 29.6s

Times for Minikube (PR 9390): 30.1s 27.5s 27.1s
Average time for Minikube (PR 9390): 28.2s

Averages Time Per Log

+--------------------------------+----------+--------------------+
|              LOG               | MINIKUBE | MINIKUBE (PR 9390) |
+--------------------------------+----------+--------------------+
| * minikube v1.14.0 on Debian   | 0.2s     |                    |
|                           9.11 |          |                    |
| * Using the docker driver      | 0.1s     | 0.1s               |
| based on user configuration    |          |                    |
| * Starting control plane node  | 0.1s     | 0.1s               |
| minikube in cluster minikube   |          |                    |
| * Creating docker container    | 9.2s     | 9.1s               |
| (CPUs=2, Memory=3700MB) ...    |          |                    |
| * Preparing Kubernetes v1.19.2 | 18.8s    | 17.7s              |
| on Docker 19.03.8 ...          |          |                    |
| * Verifying Kubernetes         | 1.1s     | 1.0s               |
| components...                  |          |                    |
| * Enabled addons:              | 0.1s     | 0.1s               |
| storage-provisioner,           |          |                    |
| default-storageclass           |          |                    |
|                                | 0.0s     | 0.0s               |
| * Want kubectl v1.19.2? Try    |          |                    |
| 'minikube kubectl -- get pods  |          |                    |
| -A'                            |          |                    |
| * Done! kubectl is now         |          |                    |
| configured to use "minikube"   |          |                    |
| by default                     |          |                    |
+--------------------------------+----------+--------------------+

@minikube-pr-bot
Copy link

kvm2 Driver
Times for minikube: 60.6s 61.1s 59.3s
Average time for minikube: 60.3s

Times for Minikube (PR 9390): 60.0s 61.3s 62.3s
Average time for Minikube (PR 9390): 61.2s

Averages Time Per Log

+--------------------------------+----------+--------------------+
|              LOG               | MINIKUBE | MINIKUBE (PR 9390) |
+--------------------------------+----------+--------------------+
| * minikube v1.14.0 on Debian   | 0.0s     |                    |
|                           9.11 |          |                    |
| * Using the kvm2 driver based  | 0.0s     | 0.0s               |
| on user configuration          |          |                    |
| * Starting control plane node  | 0.0s     | 0.0s               |
| minikube in cluster minikube   |          |                    |
| * Creating kvm2 VM (CPUs=2,    | 35.8s    | 35.0s              |
| Memory=3700MB, Disk=20000MB)   |          |                    |
| ...                            |          |                    |
| * Preparing Kubernetes v1.19.2 | 22.6s    | 23.8s              |
| on Docker 19.03.12 ...         |          |                    |
| * Verifying Kubernetes         | 1.5s     | 1.6s               |
| components...                  |          |                    |
| * Enabled addons:              | 0.4s     |                    |
| storage-provisioner,           |          |                    |
| default-storageclass           |          |                    |
|                                | 0.0s     | 0.0s               |
| * Want kubectl v1.19.2? Try    |          |                    |
| 'minikube kubectl -- get pods  |          |                    |
| -A'                            |          |                    |
| * Done! kubectl is now         |          |                    |
| configured to use "minikube"   |          |                    |
| by default                     |          |                    |
+--------------------------------+----------+--------------------+

docker Driver
Times for minikube: 30.1s 27.2s 28.3s
Average time for minikube: 28.5s

Times for Minikube (PR 9390): 28.4s 27.8s 28.6s
Average time for Minikube (PR 9390): 28.3s

Averages Time Per Log

+--------------------------------+----------+--------------------+
|              LOG               | MINIKUBE | MINIKUBE (PR 9390) |
+--------------------------------+----------+--------------------+
| * minikube v1.14.0 on Debian   | 0.2s     |                    |
|                           9.11 |          |                    |
| * Using the docker driver      | 0.1s     | 0.1s               |
| based on user configuration    |          |                    |
| * Starting control plane node  | 0.1s     | 0.1s               |
| minikube in cluster minikube   |          |                    |
| * Creating docker container    | 8.8s     | 8.6s               |
| (CPUs=2, Memory=3700MB) ...    |          |                    |
| * Preparing Kubernetes v1.19.2 | 18.3s    | 18.0s              |
| on Docker 19.03.8 ...          |          |                    |
| * Verifying Kubernetes         | 1.0s     | 1.1s               |
| components...                  |          |                    |
| * Enabled addons:              | 0.1s     | 0.2s               |
| default-storageclass,          |          |                    |
| storage-provisioner            |          |                    |
|                                | 0.0s     | 0.1s               |
| * Want kubectl v1.19.2? Try    |          |                    |
| 'minikube kubectl -- get pods  |          |                    |
| -A'                            |          |                    |
| * Done! kubectl is now         |          |                    |
| configured to use "minikube"   |          |                    |
| by default                     |          |                    |
+--------------------------------+----------+--------------------+

@prezha
Copy link
Contributor Author

prezha commented Oct 18, 2020

I think we had expected the user to parse only the stdout for json and ignore the stderr

@medyagh instead of 'forcing' user to use something like 2>/dev/null to ignore errors in command output in json format, i've proposed changes in this pr, as follows:

  • in general, to avoid errors polluting json format command output, i've amended SetJSON in pkg/minikube/out/out.go: in addition to making sure that logtostderr is set to false, and alsologtostderr is set to false, also stderrthreshold has to be set to FATAL, ie, above ERROR level; ref: https://pkg.go.dev/github.com/kubernetes/klog#pkg-overview:

Log events at or above this severity are logged to standard error as well as to files.

  • SetJSON is then called from cmd/minikube/cmd/status.go, if output flag was set to json

example

  • for the following command sequence:
    ❯ minikube start --kubernetes-version='latest' --driver='kvm2' --nodes=2 --cpus=4 --memory='8gb' --disk-size='16gb' --extra-config=apiserver.service-node-port-range=1-65535 --addons='metrics-server,dashboard'
    ❯ minikube stop
    ❯ minikube start

  • corresponding json output (running in a separate session):
    ❯ while sleep 1; do minikube status --output json -l=cluster -v=0 | jq .StatusName; done

null
null
null
...
null
null
null
"Starting"
"Starting"
"Starting"
...
"Starting"
"Starting"
"Starting"
"OK"
"OK"
"OK"
...
"OK"
"OK"
"OK"
"Stopping"
"Stopping"
"Stopping"
"Unknown"
"Stopped"
"Stopped"
"Stopped"
...
"Stopped"
"Stopped"
"Stopped"
"Starting"
"Starting"
"Starting"
...
"Starting"
"Starting"
"Starting"
"OK"
"OK"
"OK"
...

@minikube-pr-bot
Copy link

kvm2 Driver
Times for minikube: 61.7s 60.3s 61.2s
Average time for minikube: 61.1s

Times for Minikube (PR 9390): 59.5s 57.3s 61.2s
Average time for Minikube (PR 9390): 59.4s

Averages Time Per Log

+--------------------------------+----------+--------------------+
|              LOG               | MINIKUBE | MINIKUBE (PR 9390) |
+--------------------------------+----------+--------------------+
| * minikube v1.14.0 on Debian   | 0.0s     |                    |
|                           9.11 |          |                    |
| * Using the kvm2 driver based  | 0.0s     | 0.0s               |
| on user configuration          |          |                    |
| * Starting control plane node  | 0.0s     | 0.0s               |
| minikube in cluster minikube   |          |                    |
| * Creating kvm2 VM (CPUs=2,    | 36.0s    | 33.4s              |
| Memory=3700MB, Disk=20000MB)   |          |                    |
| ...                            |          |                    |
| * Preparing Kubernetes v1.19.2 | 23.4s    | 24.2s              |
| on Docker 19.03.12 ...         |          |                    |
| * Verifying Kubernetes         | 1.5s     | 1.6s               |
| components...                  |          |                    |
| * Enabled addons:              | 0.1s     |                    |
| default-storageclass,          |          |                    |
| storage-provisioner            |          |                    |
|                                | 0.0s     | 0.0s               |
| * Want kubectl v1.19.2? Try    |          |                    |
| 'minikube kubectl -- get pods  |          |                    |
| -A'                            |          |                    |
| * Done! kubectl is now         |          |                    |
| configured to use "minikube"   |          |                    |
| by default                     |          |                    |
+--------------------------------+----------+--------------------+

docker Driver
Times for minikube: 30.0s 28.7s 28.2s
Average time for minikube: 29.0s

Times for Minikube (PR 9390): 28.9s 28.9s 29.6s
Average time for Minikube (PR 9390): 29.2s

Averages Time Per Log

+--------------------------------+----------+--------------------+
|              LOG               | MINIKUBE | MINIKUBE (PR 9390) |
+--------------------------------+----------+--------------------+
| * minikube v1.14.0 on Debian   | 0.2s     |                    |
|                           9.11 |          |                    |
| * Using the docker driver      | 0.1s     | 0.1s               |
| based on user configuration    |          |                    |
| * Starting control plane node  | 0.1s     | 0.1s               |
| minikube in cluster minikube   |          |                    |
| * Creating docker container    | 9.0s     | 8.8s               |
| (CPUs=2, Memory=3700MB) ...    |          |                    |
| * Preparing Kubernetes v1.19.2 | 18.4s    | 18.8s              |
| on Docker 19.03.8 ...          |          |                    |
| * Verifying Kubernetes         | 1.1s     | 1.1s               |
| components...                  |          |                    |
| * Enabled addons:              | 0.1s     | 0.1s               |
| storage-provisioner,           |          |                    |
| default-storageclass           |          |                    |
|                                | 0.0s     | 0.0s               |
| * Want kubectl v1.19.2? Try    |          |                    |
| 'minikube kubectl -- get pods  |          |                    |
| -A'                            |          |                    |
| * Done! kubectl is now         |          |                    |
| configured to use "minikube"   |          |                    |
| by default                     |          |                    |
+--------------------------------+----------+--------------------+

additionally:
 - issue warning insted of error if 'exitCode' is not found
 - rebase: use klog instead of glog

don't pollute status json output with errors
@prezha prezha force-pushed the json-output-cleanup branch from a96c94f to e7f9b3e Compare October 18, 2020 18:56
@k8s-ci-robot k8s-ci-robot removed do-not-merge/invalid-commit-message Indicates that a PR should not merge because it has an invalid commit message. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Oct 18, 2020
@prezha prezha requested a review from medyagh October 18, 2020 19:03
@prezha prezha changed the title don't pollute json output for status with error don't pollute json output with errors Oct 18, 2020
Copy link
Contributor

@tstromberg tstromberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR has some nice things - but please keep in mind that stderr is not designed to be parsed in UNIX. There will always be unhandled messages, such as panics, that are emitted to stderr - for any command-line, not just minikube.

Most JSON users will be coming from languages where stdout and stderr are handled separately because of this. Even shells handle this properly.

@@ -191,6 +191,17 @@ func SetOutFile(w fdWriter) {
func SetJSON(j bool) {
klog.Infof("Setting JSON to %v", j)
JSON = j
if JSON {
if err := flag.Set("logtostderr", "false"); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Go, libraries should never set or read flags - only main.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for this comment - i've removed it altogether (as per your general comment), and this specifically is because libraries should never (unexpectedly and uncontrollably) change the behaviour of the mail app?

cmd/minikube/cmd/status.go Show resolved Hide resolved
@prezha
Copy link
Contributor Author

prezha commented Oct 22, 2020

@tstromberg thank you very much for your comments!
re: suppressing errors from within code & handling stderr in general - it makes a lot of sense, and what i've proposed would make it behave unexpectedly, so i've removed that part

@k8s-ci-robot k8s-ci-robot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Oct 22, 2020
@minikube-pr-bot
Copy link

kvm2 Driver
Times for minikube: 63.0s 63.1s 60.0s
Average time for minikube: 62.0s

Times for Minikube (PR 9390): 63.9s 60.2s 62.2s
Average time for Minikube (PR 9390): 62.1s

Averages Time Per Log

+--------------------------------+----------+--------------------+
|              LOG               | MINIKUBE | MINIKUBE (PR 9390) |
+--------------------------------+----------+--------------------+
| * minikube v1.14.0 on Debian   | 0.1s     | 0.0s               |
|                           9.11 |          |                    |
| * Using the kvm2 driver based  | 0.0s     | 0.0s               |
| on user configuration          |          |                    |
| * Starting control plane node  | 0.0s     | 0.0s               |
| minikube in cluster minikube   |          |                    |
| * Creating kvm2 VM (CPUs=2,    | 35.6s    | 35.7s              |
| Memory=3700MB, Disk=20000MB)   |          |                    |
| ...                            |          |                    |
| * Preparing Kubernetes v1.19.2 | 24.0s    | 24.1s              |
| on Docker 19.03.12 ...         |          |                    |
| * Verifying Kubernetes         | 1.6s     | 1.6s               |
| components...                  |          |                    |
| * Enabled addons:              | 0.7s     | 1.0s               |
| storage-provisioner,           |          |                    |
| default-storageclass           |          |                    |
|                                | 0.0s     | 0.4s               |
| * Want kubectl v1.19.2? Try    |          |                    |
| 'minikube kubectl -- get pods  |          |                    |
| -A'                            |          |                    |
| * Done! kubectl is now         |          |                    |
| configured to use "minikube"   |          |                    |
| by default                     |          |                    |
+--------------------------------+----------+--------------------+

docker Driver
Times for minikube: 30.0s 28.7s 29.5s
Average time for minikube: 29.4s

Times for Minikube (PR 9390): 28.5s 27.3s 29.3s
Average time for Minikube (PR 9390): 28.4s

Averages Time Per Log

+--------------------------------+----------+--------------------+
|              LOG               | MINIKUBE | MINIKUBE (PR 9390) |
+--------------------------------+----------+--------------------+
| * minikube v1.14.0 on Debian   | 0.2s     | 0.2s               |
|                           9.11 |          |                    |
| * Using the docker driver      | 0.1s     | 0.1s               |
| based on user configuration    |          |                    |
| * Starting control plane node  | 0.1s     | 0.1s               |
| minikube in cluster minikube   |          |                    |
| * Creating docker container    | 9.0s     | 9.0s               |
| (CPUs=2, Memory=3700MB) ...    |          |                    |
| * Preparing Kubernetes v1.19.2 | 18.7s    | 17.9s              |
| on Docker 19.03.8 ...          |          |                    |
| * Verifying Kubernetes         | 1.1s     | 1.1s               |
| components...                  |          |                    |
| * Enabled addons:              | 0.1s     | 0.1s               |
| storage-provisioner,           |          |                    |
| default-storageclass           |          |                    |
|                                | 0.0s     | 0.0s               |
| * Want kubectl v1.19.2? Try    |          |                    |
| 'minikube kubectl -- get pods  |          |                    |
| -A'                            |          |                    |
| * Done! kubectl is now         |          |                    |
| configured to use "minikube"   |          |                    |
| by default                     |          |                    |
+--------------------------------+----------+--------------------+

Copy link

@priyawadhwa priyawadhwa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now it looks like all this PR is doing is slightly changing the wording of an error message.

I'd suggest renaming the PR to more accurately describe that.

@prezha prezha changed the title don't pollute json output with errors reword error message when cluster state exit code is not found Nov 6, 2020
@prezha
Copy link
Contributor Author

prezha commented Nov 6, 2020

Right now it looks like all this PR is doing is slightly changing the wording of an error message.

I'd suggest renaming the PR to more accurately describe that.

yes, you are right - i've renamed the pr accordingly

@prezha prezha requested a review from priyawadhwa November 6, 2020 22:37
@minikube-pr-bot
Copy link

kvm2 Driver
Times for minikube: 71.6s 68.3s 68.6s
Average time for minikube: 69.5s

Times for Minikube (PR 9390): 54.6s 57.5s 56.4s
Average time for Minikube (PR 9390): 56.2s

Averages Time Per Log

+--------------------------------+----------+--------------------+
|              LOG               | MINIKUBE | MINIKUBE (PR 9390) |
+--------------------------------+----------+--------------------+
| * minikube v1.14.2 on Debian   | 0.0s     |                    |
|                           9.11 |          |                    |
| * Using the kvm2 driver based  | 0.0s     | 0.0s               |
| on user configuration          |          |                    |
| * Starting control plane node  | 0.0s     | 0.0s               |
| minikube in cluster minikube   |          |                    |
| * Creating kvm2 VM (CPUs=2,    | 31.1s    | 32.8s              |
| Memory=3700MB, Disk=20000MB)   |          |                    |
| ...                            |          |                    |
| * Preparing Kubernetes v1.19.2 | 36.6s    | 21.4s              |
| on Docker 19.03.12 ...         |          |                    |
| * Verifying Kubernetes         | 1.4s     | 1.4s               |
| components...                  |          |                    |
| * Enabled addons:              | 0.2s     | 0.7s               |
| storage-provisioner,           |          |                    |
| default-storageclass           |          |                    |
|                                | 0.0s     | 0.0s               |
| * Want kubectl v1.19.2? Try    |          |                    |
| 'minikube kubectl -- get pods  |          |                    |
| -A'                            |          |                    |
| * Done! kubectl is now         |          |                    |
| configured to use "minikube"   |          |                    |
| cluster and "default"          |          |                    |
| namespace by default           |          |                    |
+--------------------------------+----------+--------------------+

docker Driver
Times for minikube: 52.7s 44.8s 44.5s
Average time for minikube: 47.3s

Times for Minikube (PR 9390): 29.1s 27.1s 27.6s
Average time for Minikube (PR 9390): 27.9s

Averages Time Per Log

+--------------------------------+----------+--------------------+
|              LOG               | MINIKUBE | MINIKUBE (PR 9390) |
+--------------------------------+----------+--------------------+
| * minikube v1.14.2 on Debian   | 0.2s     |                    |
|                           9.11 |          |                    |
| * Using the docker driver      | 0.1s     | 0.1s               |
| based on user configuration    |          |                    |
| * Starting control plane node  | 0.1s     | 0.1s               |
| minikube in cluster minikube   |          |                    |
| * Pulling base image ...       | 9.5s     |                    |
| * Creating docker container    | 9.0s     | 8.7s               |
| (CPUs=2, Memory=3700MB) ...    |          |                    |
| * Preparing Kubernetes v1.19.2 | 33.7s    |                    |
| on Docker 19.03.13 ...         |          |                    |
| * Verifying Kubernetes         | 1.1s     | 1.1s               |
| components...                  |          |                    |
| * Enabled addons:              | 0.1s     | 0.1s               |
| storage-provisioner,           |          |                    |
| default-storageclass           |          |                    |
|                                | 0.0s     | 0.0s               |
| * Want kubectl v1.19.2? Try    |          |                    |
| 'minikube kubectl -- get pods  |          |                    |
| -A'                            |          |                    |
| * Done! kubectl is now         |          |                    |
| configured to use "minikube"   |          |                    |
| cluster and "default"          |          |                    |
| namespace by default           |          |                    |
+--------------------------------+----------+--------------------+

@prezha prezha requested a review from tstromberg November 6, 2020 22:53
Copy link

@priyawadhwa priyawadhwa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: prezha, priyawadhwa

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@TravisBuddy
Copy link

Travis tests have failed

Hey @prezha,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

View build log

make test
PATH="/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.15.2.linux.amd64/bin:/home/travis/bin:/home/travis/bin:/home/travis/.local/bin:/usr/local/lib/jvm/openjdk11/bin:/opt/pyenv/shims:/home/travis/.phpenv/shims:/home/travis/perl5/perlbrew/bin:/home/travis/.nvm/versions/node/v8.12.0/bin:/home/travis/.rvm/gems/ruby-2.5.3/bin:/home/travis/.rvm/gems/ruby-2.5.3@global/bin:/home/travis/.rvm/rubies/ruby-2.5.3/bin:/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.11.1.linux.amd64/bin:/usr/local/maven-3.6.3/bin:/usr/local/cmake-3.12.4/bin:/usr/local/clang-7.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/home/travis/.rvm/bin:/home/travis/.phpenv/bin:/opt/pyenv/bin:/home/travis/.yarn/bin:/home/travis/gopath/bin" go-bindata -nomemcopy -o pkg/minikube/assets/assets.go -pkg assets deploy/addons/...
gofmt -s -w pkg/minikube/assets/assets.go
PATH="/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.15.2.linux.amd64/bin:/home/travis/bin:/home/travis/bin:/home/travis/.local/bin:/usr/local/lib/jvm/openjdk11/bin:/opt/pyenv/shims:/home/travis/.phpenv/shims:/home/travis/perl5/perlbrew/bin:/home/travis/.nvm/versions/node/v8.12.0/bin:/home/travis/.rvm/gems/ruby-2.5.3/bin:/home/travis/.rvm/gems/ruby-2.5.3@global/bin:/home/travis/.rvm/rubies/ruby-2.5.3/bin:/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.11.1.linux.amd64/bin:/usr/local/maven-3.6.3/bin:/usr/local/cmake-3.12.4/bin:/usr/local/clang-7.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/home/travis/.rvm/bin:/home/travis/.phpenv/bin:/opt/pyenv/bin:/home/travis/.yarn/bin:/home/travis/gopath/bin" go-bindata -nomemcopy -o pkg/minikube/translate/translations.go -pkg translate translations/...
gofmt -s -w pkg/minikube/translate/translations.go
MINIKUBE_LDFLAGS="-X k8s.io/minikube/pkg/version.version=v1.14.2 -X k8s.io/minikube/pkg/version.isoVersion=v1.14.0 -X k8s.io/minikube/pkg/version.isoPath=minikube/iso -X k8s.io/minikube/pkg/version.gitCommitID="6c3e8280ecfc71df102c001ec9ff401a0e66374c" -X k8s.io/minikube/pkg/version.storageProvisionerVersion=v3" ./test.sh
= schema_check ==========================================================
go: downloading github.com/xeipuuv/gojsonschema v0.0.0-20180618132009-1d523034197f
go: downloading github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415
go: downloading github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f
ok
= go test ===============================================================
go: downloading github.com/juju/mutex v0.0.0-20180619145857-d21b13acf4bf
go: downloading github.com/machine-drivers/machine v0.7.1-0.20200810185219-7d42fed1b770
go: downloading github.com/cheggaaa/pb v1.0.27
go: downloading github.com/blang/semver v3.5.0+incompatible
go: downloading golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a
go: downloading cloud.google.com/go v0.57.0
go: downloading github.com/afbjorklund/go-getter v1.4.1-0.20201020145846-c0da14b4bffe
go: downloading github.com/cheggaaa/pb/v3 v3.0.1
go: downloading github.com/afbjorklund/go-containerregistry v0.0.0-20200902152226-fbad78ec2813
go: downloading github.com/shirou/gopsutil v2.18.12+incompatible
go: downloading k8s.io/client-go v0.17.3
go: downloading k8s.io/api v0.17.3
go: downloading github.com/mattn/go-isatty v0.0.12
go: downloading github.com/mitchellh/go-testing-interface v1.0.0
go: downloading github.com/ulikunitz/xz v0.5.7
go: downloading github.com/spf13/cobra v1.0.0
go: downloading golang.org/x/crypto v0.0.0-20200820211705-5c72a883971a
go: downloading github.com/cloudevents/sdk-go/v2 v2.1.0
go: downloading github.com/otiai10/copy v1.0.2
go: downloading github.com/cloudfoundry-attic/jibber_jabber v0.0.0-20151120183258-bcc4c8345a21
go: downloading github.com/mattn/go-colorable v0.1.6
go: downloading github.com/aws/aws-sdk-go v1.31.6
go: downloading github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2
go: downloading github.com/hashicorp/go-version v1.2.1
go: downloading cloud.google.com/go/storage v1.8.0
go: downloading github.com/hashicorp/go-safetemp v1.0.0
go: downloading github.com/machine-drivers/docker-machine-driver-vmware v0.1.1
go: downloading k8s.io/apimachinery v0.17.3
go: downloading go.opencensus.io v0.22.3
go: downloading github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51
go: downloading github.com/fatih/color v1.9.0
go: downloading github.com/juju/clock v0.0.0-20190205081909-9c5c9712527c
go: downloading github.com/VividCortex/ewma v1.1.1
go: downloading gopkg.in/yaml.v2 v2.3.0
go: downloading github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d
go: downloading github.com/hashicorp/go-cleanhttp v0.5.1
go: downloading k8s.io/kubernetes v1.18.5
go: downloading github.com/spf13/pflag v1.0.5
go: downloading go.uber.org/zap v1.10.0
go: downloading golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2
go: downloading k8s.io/klog v1.0.0
go: downloading github.com/googleapis/gax-go v2.0.2+incompatible
go: downloading google.golang.org/api v0.25.0
go: downloading github.com/mattn/go-runewidth v0.0.7
go: downloading github.com/googleapis/gax-go/v2 v2.0.5
go: downloading github.com/juju/errors v0.0.0-20190806202954-0232dcc7464d
go: downloading golang.org/x/sys v0.0.0-20200523222454-059865788121
go: downloading github.com/docker/cli v0.0.0-20200303162255-7d407207c304
go: downloading google.golang.org/genproto v0.0.0-20200527145253-8367513e4ece
go: downloading github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e
go: downloading golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6
go: downloading github.com/olekukonko/tablewriter v0.0.4
go: downloading github.com/mitchellh/go-ps v0.0.0-20190716172923-621e5597135b
go: downloading google.golang.org/grpc v1.29.1
go: downloading github.com/docker/docker v1.4.2-0.20190924003213-a8608b5b67c7
go: downloading golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d
go: downloading github.com/intel-go/cpuid v0.0.0-20181003105527-1a4a6f06a1c6
go: downloading k8s.io/klog/v2 v2.4.0
go: downloading github.com/davecgh/go-spew v1.1.1
go: downloading k8s.io/component-base v0.17.3
go: downloading github.com/pkg/browser v0.0.0-20160118053552-9302be274faa
go: downloading github.com/mitchellh/go-homedir v1.1.0
go: downloading github.com/google/gofuzz v1.1.0
go: downloading github.com/docker/go-connections v0.4.0
go: downloading github.com/gogo/protobuf v1.3.1
go: downloading github.com/go-logr/logr v0.3.0
go: downloading github.com/google/uuid v1.1.1
go: downloading github.com/pkg/profile v0.0.0-20161223203901-3a8809bd8a80
go: downloading github.com/sayboras/dockerclient v1.0.0
go: downloading go.uber.org/multierr v1.1.0
go: downloading k8s.io/cluster-bootstrap v0.17.3
go: downloading google.golang.org/protobuf v1.24.0
go: downloading github.com/opencontainers/go-digest v1.0.0-rc1
go: downloading github.com/modern-go/reflect2 v1.0.1
go: downloading github.com/pkg/errors v0.9.1
go: downloading github.com/golang/protobuf v1.4.2
go: downloading github.com/sirupsen/logrus v1.6.0
go: downloading github.com/hooklift/iso9660 v0.0.0-20170318115843-1cf07e5970d8
go: downloading sigs.k8s.io/sig-storage-lib-external-provisioner v4.0.0+incompatible
go: downloading github.com/google/slowjam v0.0.0-20200530021616-df27e642fe7b
go: downloading k8s.io/utils v0.0.0-20200324210504-a9aa75ae1b89
go: downloading sigs.k8s.io/sig-storage-lib-external-provisioner/v5 v5.0.0
go: downloading github.com/docker/go-units v0.4.0
go: downloading github.com/spf13/viper v1.7.0
go: downloading github.com/cenkalti/backoff v2.2.1+incompatible
go: downloading github.com/containerd/containerd v1.3.1-0.20191213020239-082f7e3aed57
go: downloading github.com/libvirt/libvirt-go v3.4.0+incompatible
go: downloading github.com/google/go-github v17.0.0+incompatible
go: downloading golang.org/x/text v0.3.2
go: downloading gopkg.in/inf.v0 v0.9.1
go: downloading github.com/spf13/cast v1.3.1
go: downloading gopkg.in/ini.v1 v1.56.0
go: downloading github.com/lightstep/tracecontext.go v0.0.0-20181129014701-1757c391b1ac
go: downloading github.com/docker/distribution v2.7.1+incompatible
go: downloading github.com/golang-collections/collections v0.0.0-20130729185459-604e922904d3
go: downloading github.com/hashicorp/hcl v1.0.0
go: downloading github.com/prometheus/client_golang v1.4.1
go: downloading github.com/google/go-querystring v1.0.0
go: downloading github.com/fsnotify/fsnotify v1.4.9
go: downloading github.com/opencontainers/image-spec v1.0.1
go: downloading github.com/spf13/jwalterweatherman v1.1.0
go: downloading k8s.io/kubectl v0.17.3
go: downloading github.com/prometheus/common v0.9.1
go: downloading github.com/cespare/xxhash v1.1.0
go: downloading github.com/cpuguy83/go-md2man v1.0.10
go: downloading github.com/imdario/mergo v0.3.9
go: downloading github.com/spf13/afero v1.2.2
go: downloading github.com/cespare/xxhash/v2 v2.1.1
go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.0
go: downloading github.com/russross/blackfriday v1.5.3-0.20200218234912-41c5fccfd6f6
go: downloading github.com/googleapis/gnostic v0.3.0
go: downloading github.com/pelletier/go-toml v1.8.0
go: downloading golang.org/x/time v0.0.0-20200416051211-89c76fbcd5d1
go: downloading github.com/subosito/gotenv v1.2.0
go: downloading github.com/jmespath/go-jmespath v0.3.0
go: downloading github.com/magiconair/properties v1.8.1
go: downloading github.com/russross/blackfriday/v2 v2.0.1
go: downloading github.com/mitchellh/mapstructure v1.3.1
go: downloading github.com/prometheus/procfs v0.0.8
go: downloading github.com/miekg/dns v1.1.27
go: downloading go.uber.org/atomic v1.4.0
go: downloading github.com/docker/docker-credential-helpers v0.6.3
go: downloading sigs.k8s.io/yaml v1.2.0
go: downloading github.com/json-iterator/go v1.1.9
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
go: downloading github.com/prometheus/client_model v0.2.0
go: downloading k8s.io/kube-openapi v0.0.0-20200410145947-61e04a5be9a6
go: downloading github.com/shurcooL/sanitized_anchor_name v1.0.0
go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: downloading github.com/google/go-cmp v0.5.2
go: downloading github.com/c4milo/gotoolkit v0.0.0-20170318115440-bcc06269efa9
go: downloading github.com/hashicorp/golang-lru v0.5.3
go: downloading github.com/beorn7/perks v1.0.1
go: downloading github.com/MakeNowJust/heredoc v0.0.0-20170808103936-bb23615498cd
go: downloading github.com/mitchellh/go-wordwrap v1.0.0
go: downloading github.com/docker/spdystream v0.0.0-20160310174837-449fdfce4d96
go: downloading golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543
go: downloading github.com/evanphx/json-patch v4.5.0+incompatible
go: downloading github.com/pmezard/go-difflib v1.0.0
--- FAIL: TestGenerateDocs (0.02s)
    --- FAIL: TestGenerateDocs/addons (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 17 identical lines
              	```
              	
            - 	### Options
            + 	### Options inherited from parent commands
              	
              	```
            - 	  -h, --help   help for addons
            - 	```
            - 	
            - 	### Options inherited from parent commands
            - 	
            - 	```
              	      --add_dir_header                   If true, adds the file directory to the header of the log messages
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 22 identical lines
              	```
              	
            - 	### Options
            + 	### Options inherited from parent commands
              	
              	```
            - 	  -h, --help   help for configure
            - 	```
            - 	
            - 	### Options inherited from parent commands
            - 	
            - 	```
              	      --add_dir_header                   If true, adds the file directory to the header of the log messages
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 22 identical lines
              	```
              	
            - 	### Options
            + 	### Options inherited from parent commands
              	
              	```
            - 	  -h, --help   help for disable
            - 	```
            - 	
            - 	### Options inherited from parent commands
            - 	
            - 	```
              	      --add_dir_header                   If true, adds the file directory to the header of the log messages
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 9 identical lines
              	      --vmodule moduleSpec               comma-separated list of pattern=N settings for file-filtered logging
              	```
            - 	
            - 	## minikube addons enable
            - 	
            - 	Enables the addon w/ADDON_NAME within minikube (example: minikube addons enable dashboard). For a list of available addons use: minikube addons list 
              	
            - 	### Synopsis
            + 	## minikube addons enable
              	... // 136 identical, 19 removed, and 13 inserted lines
              	"""
              )
    --- FAIL: TestGenerateDocs/cache (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 13 identical lines
              	Add, delete, or push a local image into minikube
              	
            - 	### Options
            + 	### Options inherited from parent commands
              	
              	```
            - 	  -h, --help   help for cache
            - 	```
            - 	
            - 	### Options inherited from parent commands
            - 	
            - 	```
              	      --add_dir_header                   If true, adds the file directory to the header of the log messages
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 22 identical lines
              	```
              	
            - 	### Options
            + 	### Options inherited from parent commands
              	
              	```
            - 	  -h, --help   help for add
            - 	```
            - 	
            - 	### Options inherited from parent commands
            - 	
            - 	```
              	      --add_dir_header                   If true, adds the file directory to the header of the log messages
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 22 identical lines
              	```
              	
            - 	### Options
            + 	### Options inherited from parent commands
              	
              	```
            - 	  -h, --help   help for delete
            - 	```
            - 	
            - 	### Options inherited from parent commands
            - 	
            - 	```
              	      --add_dir_header                   If true, adds the file directory to the header of the log messages
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 9 identical lines
              	      --vmodule moduleSpec               comma-separated list of pattern=N settings for file-filtered logging
              	```
            - 	
            - 	## minikube cache help
            - 	
            - 	Help about any command
              	
            - 	### Synopsis
            + 	## minikube cache help
              	... // 94 identical, 19 removed, and 13 inserted lines
              	"""
              )
    --- FAIL: TestGenerateDocs/completion (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 36 identical lines
              	```
              	minikube completion SHELL [flags]
            - 	```
            - 	
            - 	### Options
            - 	
            - 	```
            - 	  -h, --help   help for completion
              	```
              	
              	... // 4 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 13 identical lines
              	"""
              )
    --- FAIL: TestGenerateDocs/config (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 47 identical lines
              	```
              	
            - 	### Options
            + 	### Options inherited from parent commands
              	
              	```
            - 	  -h, --help   help for config
            - 	```
            - 	
            - 	### Options inherited from parent commands
            - 	
            - 	```
              	      --add_dir_header                   If true, adds the file directory to the header of the log messages
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 28 identical lines
              	
              	```
            - 	  -h, --help            help for defaults
              	      --output string   Output format. Accepted values: [json]
              	```
              	... // 5 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 22 identical lines
              	```
              	
            - 	### Options
            + 	### Options inherited from parent commands
              	
              	```
            - 	  -h, --help   help for get
            - 	```
            - 	
            - 	### Options inherited from parent commands
            - 	
            - 	```
              	      --add_dir_header                   If true, adds the file directory to the header of the log messages
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 23 identical lines
              	```
              	
            - 	### Options
            + 	### Options inherited from parent commands
              	
              	```
            - 	  -h, --help   help for help
            - 	```
            - 	
            - 	### Options inherited from parent commands
            - 	
            - 	```
              	      --add_dir_header                   If true, adds the file directory to the header of the log messages
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 9 identical lines
              	      --vmodule moduleSpec               comma-separated list of pattern=N settings for file-filtered logging
              	```
            - 	
            - 	## minikube config set
            - 	
            - 	Sets an individual value in a minikube config file
              	... // 95 identical, 20 removed, and 14 inserted lines
              	"""
              )
    --- FAIL: TestGenerateDocs/dashboard (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   strings.Join({
              	... // 20 identical lines
              	"",
              	"```",
            - 	"  -h, --help   help for dashboard",
            - 	"      --url    Display dashboard URL instead of opening a browser",
            + 	"      --url   Display dashboard URL instead of opening a browser",
              	"```",
              	"",
              	... // 4 identical lines
              	"      --alsologtostderr                  log to standard error as well as files",
              	`  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")`,
            + 	"  -h, --help                             ",
              	"      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)",
              	"      --log_dir string                   If non-empty, write log files in this directory",
              	... // 13 identical lines
              }, "\n")
    --- FAIL: TestGenerateDocs/delete (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 22 identical lines
              	```
              	      --all     Set flag to delete all profiles
            - 	  -h, --help    help for delete
              	      --purge   Set this flag to delete the '.minikube' folder from your user directory.
              	```
              	... // 5 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 13 identical lines
              	"""
              )
    --- FAIL: TestGenerateDocs/docker-env (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
            �� 	"""
              	... // 20 identical lines
              	
              	```
            - 	  -h, --help           help for docker-env
              	      --no-proxy       Add machine IP to NO_PROXY environment variable
              	      --shell string   Force environment to be configured for a specified shell: [fish, cmd, powershell, tcsh, bash, zsh], default is auto-detect
              	... // 7 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 13 identical lines
              	"""
              )
    --- FAIL: TestGenerateDocs/ip (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 15 identical lines
              	```
              	minikube ip [flags]
            - 	```
            - 	
            - 	### Options
            - 	
            - 	```
            - 	  -h, --help   help for ip
              	```
              	
              	... // 4 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 13 identical lines
              	"""
              )
    --- FAIL: TestGenerateDocs/kubectl (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 19 identical lines
              	```
              	minikube kubectl [flags]
            - 	```
            - 	
            - 	### Options
            - 	
            - 	```
            - 	  -h, --help   help for kubectl
              	```
              	
              	... // 4 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 13 identical lines
              	"""
              )
    --- FAIL: TestGenerateDocs/logs (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 21 identical lines
              	```
              	  -f, --follow        Show only the most recent journal entries, and continuously print new entries as they are appended to the journal.
            - 	  -h, --help          help for logs
              	  -n, --length int    Number of lines back to go within the log (default 60)
              	      --node string   The node to get logs from. Defaults to the primary control plane.
              	... // 7 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 13 identical lines
              	"""
              )
    --- FAIL: TestGenerateDocs/mount (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 22 identical lines
              	      --9p-version string   Specify the 9p version that the mount should use (default "9p2000.L")
              	      --gid string          Default group id used for the mount (default "docker")
            - 	  -h, --help                help for mount
              	      --ip string           Specify the ip that the mount should be setup on
              	      --kill                Kill the mount process spawned by minikube start
              	... // 11 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 13 identical lines
              	"""
              )
    --- FAIL: TestGenerateDocs/node (0.01s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 17 identical lines
              	```
              	
            - 	### Options
            + 	### Options inherited from parent commands
              	
              	```
            - 	  -h, --help   help for node
            - 	```
            - 	
            - 	### Options inherited from parent commands
            - 	
            - 	```
              	      --add_dir_header                   If true, adds the file directory to the header of the log messages
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 27 identical lines
              	      --control-plane       If true, the node added will also be a control plane in addition to a worker.
              	      --delete-on-failure   If set, delete the current cluster if start fails and try again. Defaults to false.
            - 	  -h, --help                help for add
              	      --worker              If true, the added node will be marked for work. Defaults to true. (default true)
              	```
              	... // 5 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 22 identical lines
              	```
              	
            - 	### Options
            + 	### Options inherited from parent commands
              	
              	```
            - 	  -h, --help   help for delete
            - 	```
            - 	
            - 	### Options inherited from parent commands
            - 	
            - 	```
              	      --add_dir_header                   If true, adds the file directory to the header of the log messages
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 23 identical lines
              	```
              	
            - 	### Options
            + 	### Options inherited from parent commands
              	
              	```
            - 	  -h, --help   help for help
            - 	```
            - 	
            - 	### Options inherited from parent commands
            - 	
            - 	```
              	      --add_dir_header                   If true, adds the file directory to the header of the log messages
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 9 identical lines
              	      --vmodule moduleSpec               comma-separated list of pattern=N settings for file-filtered logging
              	```
            - 	
            - 	## minikube node list
            - 	
            - 	List nodes.
              	... // 95 identical, 18 removed, and 12 inserted lines
              	"""
              )
    --- FAIL: TestGenerateDocs/pause (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 22 identical lines
              	  -n, ----namespaces strings   namespaces to pause (default [kube-system,kubernetes-dashboard,storage-gluster,istio-operator])
              	  -A, --all-namespaces         If set, pause all namespaces
            - 	  -h, --help                   help for pause
              	  -o, --output string          Format to print stdout in. Options include: [text,json] (default "text")
              	```
              	... // 5 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 13 identical lines
              	"""
              )
    --- FAIL: TestGenerateDocs/podman-env (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 20 identical lines
              	
              	```
            - 	  -h, --help           help for podman-env
              	      --shell string   Force environment to be configured for a specified shell: [fish, cmd, powershell, tcsh, bash, zsh], default is auto-detect
              	  -u, --unset          Unset variables instead of setting them
              	... // 6 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 13 identical lines
              	"""
              )
    --- FAIL: TestGenerateDocs/profile (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 17 identical lines
              	```
              	
            - 	### Options
            + 	### Options inherited from parent commands
              	
              	```
            - 	  -h, --help   help for profile
            - 	```
            - 	
            - 	### Options inherited from parent commands
            - 	
            - 	```
              	      --add_dir_header                   If true, adds the file directory to the header of the log messages
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 13 identical lines
              	
              	Help about any command
            - 	
            - 	### Synopsis
            - 	
            - 	Help provides help for any command in the application.
            - 	Simply type profile help [path to command] for full details.
              	
            - 	```
            - 	minikube profile help [command] [flags]
            - 	```
            + 	### Synopsis
              	
            - 	### Options
            + 	Help provides help for any command in the application.
            + 	Simply type profile help [path to command] for full details.
              	
              	```
            - 	  -h, --help   help for help
            + 	minikube profile help [command] [flags]
              	```
              	
              	... // 4 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 25 identical lines
              	
              	```
            - 	  -h, --help            help for list
              	  -o, --output string   The output format. One of 'json', 'table' (default "table")
              	```
              	... // 5 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 13 identical lines
              	"""
              )
    --- FAIL: TestGenerateDocs/service (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 21 identical lines
              	```
              	      --format string      Format to output service URL in. This format will be applied to each url individually and they will be printed one at a time. (default "http://{{.IP}}:{{.Port}}")
            - 	  -h, --help               help for service
              	      --https              Open the service URL with https instead of http (defaults to "false")
              	      --interval int       The initial time interval for each check that wait performs in seconds (default 1)
              	... // 9 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 18 identical lines
              	Help provides help for any command in the application.
              	Simply type service help [path to command] for full details.
            - 	
            - 	```
            - 	minikube service help [command] [flags]
            - 	```
            - 	
            - 	### Options
              	
              	```
            - 	  -h, --help   help for help
            + 	minikube service help [command] [flags]
              	```
              	
              	... // 5 identical lines
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
              	      --format string                    Format to output service URL in. This format will be applied to each url individually and they will be printed one at a time. (default "http://{{.IP}}:{{.Port}}")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 25 identical lines
              	
              	```
            - 	  -h, --help               help for list
              	  -n, --namespace string   The services namespace
              	```
              	... // 6 identical lines
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
              	      --format string                    Format to output service URL in. This format will be applied to each url individually and they will be printed one at a time. (default "http://{{.IP}}:{{.Port}}")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 13 identical lines
              	"""
              )
    --- FAIL: TestGenerateDocs/ssh (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 20 identical lines
              	
              	```
            - 	  -h, --help          help for ssh
              	      --native-ssh    Use native Golang SSH client (default true). Set to 'false' to use the command line 'ssh' command when accessing the docker machine. Useful for the machine drivers when they will not start with 'Waiting for SSH'. (default true)
              	  -n, --node string   The node to ssh into. Defaults to the primary control plane.
              	... // 6 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 13 identical lines
              	"""
              )
    --- FAIL: TestGenerateDocs/ssh-key (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 15 identical lines
              	```
              	minikube ssh-key [flags]
            - 	```
            - 	
            - 	### Options
            - 	
            - 	```
            - 	  -h, --help   help for ssh-key
              	```
              	
              	... // 4 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 13 identical lines
              	"""
              )
    --- FAIL: TestGenerateDocs/start (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 51 identical lines
              	      --force                             Force minikube to perform possibly dangerous operations
              	      --force-systemd                     If set, force the container runtime to use sytemd as cgroup manager. Currently available for docker and crio. Defaults to false.
            - 	  -h, --help                              help for start
              	      --host-dns-resolver                 Enable host resolver for NAT DNS requests (virtualbox driver only) (default true)
              	      --host-only-cidr string             The CIDR to be used for the minikube VM (virtualbox driver only) (default "192.168.99.1/24")
              	... // 45 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 13 identical lines
              	"""
              )
    --- FAIL: TestGenerateDocs/status (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 24 identical lines
              	  -f, --format string   Go template format string for the status output.  The format for Go templates can be found here: https://golang.org/pkg/text/template/
              	                        For the list accessible variables for the template, see the struct values here: https://godoc.org/k8s.io/minikube/cmd/minikube/cmd#Status (default "{{.Name}}\ntype: Control Plane\nhost: {{.Host}}\nkubelet: {{.Kubelet}}\napiserver: {{.APIServer}}\nkubeconfig: {{.Kubeconfig}}\n\n")
            - 	  -h, --help            help for status
              	  -l, --layout string   output layout (EXPERIMENTAL, JSON only): 'nodes' or 'cluster' (default "nodes")
              	  -n, --node string     The node to check status for. Defaults to control plane. Leave blank with default format for status on all nodes.
              	... // 7 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 13 identical lines
              	"""
              )
    --- FAIL: TestGenerateDocs/stop (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 21 identical lines
              	```
              	      --all                   Set flag to stop all profiles (clusters)
            - 	  -h, --help                  help for stop
              	      --keep-context-active   keep the kube-context active after cluster is stopped. Defaults to false.
              	  -o, --output string         Format to print stdout in. Options include: [text,json] (default "text")
              	... // 6 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 13 identical lines
              	"""
              )
    --- FAIL: TestGenerateDocs/tunnel (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 21 identical lines
              	```
              	  -c, --cleanup   call with cleanup=true to remove old tunnels (default true)
            - 	  -h, --help      help for tunnel
              	```
              	
              	... // 4 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 13 identical lines
              	"""
              )
    --- FAIL: TestGenerateDocs/unpause (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 22 identical lines
              	  -n, ----namespaces strings   namespaces to unpause (default [kube-system,kubernetes-dashboard,storage-gluster,istio-operator])
              	  -A, --all-namespaces         If set, unpause all namespaces
            - 	  -h, --help                   help for unpause
              	  -o, --output string          Format to print stdout in. Options include: [text,json] (default "text")
              	```
              	... // 5 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 13 identical lines
              	"""
              )
    --- FAIL: TestGenerateDocs/update-check (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 15 identical lines
              	```
              	minikube update-check [flags]
            - 	```
            - 	
            - 	### Options
            - 	
            - 	```
            - 	  -h, --help   help for update-check
              	```
              	
              	... // 4 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 13 identical lines
              	"""
              )
    --- FAIL: TestGenerateDocs/update-context (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 16 identical lines
              	```
              	minikube update-context [flags]
            - 	```
            - 	
            - 	### Options
            - 	
            - 	```
            - 	  -h, --help   help for update-context
              	```
              	
              	... // 4 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 13 identical lines
              	"""
              )
    --- FAIL: TestGenerateDocs/version (0.00s)
        generate-docs_test.go:47: Docs are not updated. Please run `make generate-docs` to update commands documentation:   (
              	"""
              	... // 20 identical lines
              	
              	```
            - 	  -h, --help            help for version
              	  -o, --output string   One of 'yaml' or 'json'.
              	      --short           Print just the version number.
              	... // 6 identical lines
              	      --alsologtostderr                  log to standard error as well as files
              	  -b, --bootstrapper string              The name of the cluster bootstrapper that will set up the Kubernetes cluster. (default "kubeadm")
            + 	  -h, --help                             
              	      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
              	      --log_dir string                   If non-empty, write log files in this directory
              	... // 13 identical lines
              	"""
              )
E1106 23:41:51.439541   12099 out.go:142] [unset errFile]: ! Local proxy ignored: not passing HTTP_PROXY=127.0.0.1:3128 to docker env.
E1106 23:41:51.440067   12099 out.go:142] [unset errFile]: ! Local proxy ignored: not passing HTTP_PROXY=127.0.0.1:3128 to docker env.
E1106 23:41:51.440204   12099 out.go:142] [unset errFile]: ! Local proxy ignored: not passing HTTP_PROXY=localhost:3128 to docker env.
E1106 23:41:51.440385   12099 out.go:142] [unset errFile]: ! Local proxy ignored: not passing HTTP_PROXY=localhost:3128 to docker env.
E1106 23:41:51.440488   12099 out.go:142] [unset errFile]: ! Local proxy ignored: not passing HTTP_PROXY=http://localhost:3128 to docker env.
E1106 23:41:51.440662   12099 out.go:142] [unset errFile]: ! Local proxy ignored: not passing HTTP_PROXY=http://localhost:3128 to docker env.
E1106 23:41:51.440790   12099 out.go:142] [unset errFile]: ! Local proxy ignored: not passing HTTP_PROXY=http://127.0.0.1:3128 to docker env.
E1106 23:41:51.440971   12099 out.go:142] [unset errFile]: ! Local proxy ignored: not passing HTTP_PROXY=http://127.0.0.1:3128 to docker env.
FAIL
coverage: 18.5% of statements
FAIL	k8s.io/minikube/cmd/minikube/cmd	3.818s
ok  	k8s.io/minikube/cmd/minikube/cmd/config	0.040s	coverage: 21.0% of statements
ok  	k8s.io/minikube/pkg/addons	0.075s	coverage: 50.9% of statements
ok  	k8s.io/minikube/pkg/drivers	0.011s	coverage: 19.6% of statements
ok  	k8s.io/minikube/pkg/drivers/hyperkit	0.007s	coverage: 77.3% of statements
ok  	k8s.io/minikube/pkg/drivers/kic/oci	0.017s	coverage: 6.4% of statements
ok  	k8s.io/minikube/pkg/drivers/kvm	0.033s	coverage: 2.3% of statements
ok  	k8s.io/minikube/pkg/minikube/bootstrapper	0.616s	coverage: 55.7% of statements
ok  	k8s.io/minikube/pkg/minikube/bootstrapper/bsutil	0.114s	coverage: 62.4% of statements
ok  	k8s.io/minikube/pkg/minikube/bootstrapper/bsutil/ktmpl	0.022s	coverage: 100.0% of statements
ok  	k8s.io/minikube/pkg/minikube/bootstrapper/images	0.014s	coverage: 82.0% of statements
ok  	k8s.io/minikube/pkg/minikube/cluster	0.066s	coverage: 14.5% of statements
ok  	k8s.io/minikube/pkg/minikube/command	0.034s	coverage: 4.2% of statements
ok  	k8s.io/minikube/pkg/minikube/config	0.139s	coverage: 68.1% of statements
ok  	k8s.io/minikube/pkg/minikube/cruntime	0.052s	coverage: 36.7% of statements
ok  	k8s.io/minikube/pkg/minikube/docker	0.057s	coverage: 20.8% of statements
ok  	k8s.io/minikube/pkg/minikube/driver	0.028s	coverage: 40.1% of statements
ok  	k8s.io/minikube/pkg/minikube/extract	0.006s	coverage: 58.6% of statements
ok  	k8s.io/minikube/pkg/minikube/image	0.029s	coverage: 3.0% of statements
ok  	k8s.io/minikube/pkg/minikube/kubeconfig	0.039s	coverage: 81.3% of statements
ok  	k8s.io/minikube/pkg/minikube/localpath	0.011s	coverage: 49.3% of statements
ok  	k8s.io/minikube/pkg/minikube/logs	0.041s	coverage: 1.3% of statements
ok  	k8s.io/minikube/pkg/minikube/machine	1.124s	coverage: 35.8% of statements
ok  	k8s.io/minikube/pkg/minikube/mustload	0.085s	coverage: 10.5% of statements
ok  	k8s.io/minikube/pkg/minikube/notify	0.020s	coverage: 92.9% of statements
ok  	k8s.io/minikube/pkg/minikube/out	0.024s	coverage: 66.0% of statements
ok  	k8s.io/minikube/pkg/minikube/out/register	0.010s	coverage: 54.5% of statements
ok  	k8s.io/minikube/pkg/minikube/perf	4.015s	coverage: 21.1% of statements
ok  	k8s.io/minikube/pkg/minikube/proxy	0.011s	coverage: 68.7% of statements
ok  	k8s.io/minikube/pkg/minikube/reason	0.008s	coverage: 80.0% of statements
ok  	k8s.io/minikube/pkg/minikube/registry	0.010s	coverage: 75.5% of statements
ok  	k8s.io/minikube/pkg/minikube/service	0.068s	coverage: 84.2% of statements
ok  	k8s.io/minikube/pkg/minikube/shell	0.006s	coverage: 97.1% of statements
ok  	k8s.io/minikube/pkg/minikube/storageclass	0.038s	coverage: 100.0% of statements
ok  	k8s.io/minikube/pkg/minikube/style	0.003s	coverage: 100.0% of statements
ok  	k8s.io/minikube/pkg/minikube/sysinit	0.043s	coverage: 6.5% of statements
ok  	k8s.io/minikube/pkg/minikube/translate	0.009s	coverage: 10.5% of statements
ok  	k8s.io/minikube/pkg/minikube/tunnel	1.965s	coverage: 64.2% of statements
ok  	k8s.io/minikube/pkg/util	1.203s	coverage: 75.7% of statements
ok  	k8s.io/minikube/pkg/util/lock	0.006s	coverage: 22.2% of statements
ok  	k8s.io/minikube/pkg/util/retry	0.005s	coverage: 0.0% of statements
FAIL
Makefile:315: recipe for target 'test' failed
make: *** [test] Error 32
TravisBuddy Request Identifier: e8b0ee90-2089-11eb-a20b-45518f503c27

@tstromberg tstromberg merged commit b9615bf into kubernetes:master Nov 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

do not pollute the json output for status with with Error Don't spit out glog.Errorf in json output
8 participants