Make sure multinode clusters can survive restarts #7973

sharifelgamal · 2020-05-02T01:05:53Z

Fixes #7496

k8s-ci-robot · 2020-05-02T01:06:25Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sharifelgamal

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [sharifelgamal]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

codecov-io · 2020-05-04T21:55:53Z

Codecov Report

Merging #7973 into master will decrease coverage by 0.08%.
The diff coverage is 34.00%.

@@            Coverage Diff             @@
##           master    #7973      +/-   ##
==========================================
- Coverage   34.60%   34.51%   -0.09%     
==========================================
  Files         147      147              
  Lines        9378     9401      +23     
==========================================
  Hits         3245     3245              
- Misses       5736     5757      +21     
- Partials      397      399       +2

Impacted Files	Coverage Δ
cmd/minikube/cmd/node_add.go	`13.88% <0.00%> (-3.97%)`	⬇️
cmd/minikube/cmd/node_start.go	`9.37% <0.00%> (-0.63%)`	⬇️
cmd/minikube/cmd/node_stop.go	`13.33% <0.00%> (-0.96%)`	⬇️
cmd/minikube/cmd/start.go	`15.02% <0.00%> (-0.08%)`	⬇️
cmd/minikube/cmd/status.go	`23.20% <ø> (ø)`
pkg/minikube/bootstrapper/bootstrapper.go	`0.00% <ø> (ø)`
pkg/minikube/bootstrapper/certs.go	`42.74% <20.00%> (-1.53%)`	⬇️
pkg/minikube/machine/start.go	`59.57% <64.28%> (-0.88%)`	⬇️
pkg/minikube/machine/fix.go	`38.31% <77.77%> (ø)`

…restart

sharifelgamal · 2020-05-20T22:59:46Z

/ok-to-test

minikube-pr-bot · 2020-05-20T23:03:53Z

kvm2 Driver
docker Driver

minikube-pr-bot · 2020-05-20T23:24:01Z

kvm2 Driver
Times for minikube: [64.84174844799999 65.17188296499998 66.3974307]
Average time for minikube: 65.47035403766667

Times for Minikube (PR 7973): [65.484962536 66.415054937 64.840697597]
Average time for Minikube (PR 7973): 65.58023835666667

Averages Time Per Log

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 7973) |
+--------------------------------+-----------+--------------------+
| * minikube v1.10.1 on Debian   |  0.062908 |           0.067353 |
|                           9.11 |           |                    |
| * Using the kvm2 driver based  |  0.022304 |           0.023450 |
| on existing profile            |           |                    |
| * Starting control plane node  |  0.005639 |           0.003553 |
| minikube in cluster minikube   |           |                    |
| * Creating kvm2 VM (CPUs=2,    | 40.262553 |          40.162894 |
| Memory=3700MB, Disk=20000MB)   |           |                    |
| ...                            |           |                    |
| * Preparing Kubernetes v1.18.2 | 23.062026 |          23.088246 |
| on Docker 19.03.8 ...          |           |                    |
| * Verifying Kubernetes         |  1.415995 |           1.542455 |
| components...                  |           |                    |
| * Enabled addons:              |  0.553068 |           0.604292 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.081772 |           0.081391 |
| configured to use "minikube"   |           |                    |
|                                |  0.004089 |           0.006604 |
+--------------------------------+-----------+--------------------+

docker Driver
Times for minikube: [27.660365914000003 27.900675310999997 28.19733573]
Average time for minikube: 27.919458985000002

Times for Minikube (PR 7973): [24.235588917 26.600426017 24.261345107999997]
Average time for Minikube (PR 7973): 25.032453347333334

Averages Time Per Log

+----------------------------------------+-----------+--------------------+
|                  LOG                   | MINIKUBE  | MINIKUBE (PR 7973) |
+----------------------------------------+-----------+--------------------+
| * minikube v1.10.1 on Debian           |  0.083920 |           0.079219 |
|                                   9.11 |           |                    |
| * Using the docker driver              |  0.003586 |           0.002825 |
| based on existing profile              |           |                    |
| * Starting control plane node          |  0.063420 |           0.066936 |
| minikube in cluster minikube           |           |                    |
| * Creating docker container            |  8.127957 |           5.611310 |
| (CPUs=2, Memory=3700MB) ...            |           |                    |
| * Preparing Kubernetes v1.18.2         |  0.129629 |           0.132332 |
| on Docker 19.03.2 ...                  |           |                    |
|   -                                    | 18.230095 |          18.885668 |
| kubeadm.pod-network-cidr=10.244.0.0/16 |           |                    |
| * Verifying Kubernetes                 |  1.200356 |           0.174730 |
| components...                          |           |                    |
| * Enabled addons:                      |  0.003360 |           0.003011 |
| default-storageclass,                  |           |                    |
| storage-provisioner                    |           |                    |
| * Done! kubectl is now                 |  0.072513 |           0.072390 |
| configured to use "minikube"           |           |                    |
|                                        |  0.004623 |           0.004034 |
+----------------------------------------+-----------+--------------------+

minikube-pr-bot · 2020-05-21T00:14:26Z

kvm2 Driver
Times for minikube: [65.89063012899999 66.469415068 67.14821262999999]
Average time for minikube: 66.50275260899998

Times for Minikube (PR 7973): [65.22296300699999 65.79751244799999 65.75328363500002]
Average time for Minikube (PR 7973): 65.59125303

Averages Time Per Log

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 7973) |
+--------------------------------+-----------+--------------------+
| * minikube v1.10.1 on Debian   |  0.065244 |           0.068737 |
|                           9.11 |           |                    |
| * Using the kvm2 driver based  |  0.022132 |           0.021629 |
| on existing profile            |           |                    |
| * Starting control plane node  |  0.003892 |           0.003857 |
| minikube in cluster minikube   |           |                    |
| * Creating kvm2 VM (CPUs=2,    | 41.708675 |          40.558488 |
| Memory=3700MB, Disk=20000MB)   |           |                    |
| ...                            |           |                    |
| * Preparing Kubernetes v1.18.2 | 22.301099 |          22.519155 |
| on Docker 19.03.8 ...          |           |                    |
| * Verifying Kubernetes         |  1.432243 |           1.466024 |
| components...                  |           |                    |
| * Enabled addons:              |  0.887646 |           0.860892 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.078074 |           0.087611 |
| configured to use "minikube"   |           |                    |
|                                |  0.003748 |           0.004860 |
+--------------------------------+-----------+--------------------+

docker Driver
Times for minikube: [27.573862473000002 28.551838548000003 27.948781505000003]
Average time for minikube: 28.024827508666664

Times for Minikube (PR 7973): [30.159359658 27.129887715000002 28.234326952]
Average time for Minikube (PR 7973): 28.507858108333334

Averages Time Per Log

+----------------------------------------+-----------+--------------------+
|                  LOG                   | MINIKUBE  | MINIKUBE (PR 7973) |
+----------------------------------------+-----------+--------------------+
| * minikube v1.10.1 on Debian           |  0.079457 |           0.080914 |
|                                   9.11 |           |                    |
| * Using the docker driver              |  0.002844 |           0.002784 |
| based on existing profile              |           |                    |
| * Starting control plane node          |  0.062767 |           0.064160 |
| minikube in cluster minikube           |           |                    |
| * Creating docker container            |  8.556789 |           8.390278 |
| (CPUs=2, Memory=3700MB) ...            |           |                    |
| * Preparing Kubernetes v1.18.2         |  0.129716 |           0.156370 |
| on Docker 19.03.2 ...                  |           |                    |
|   -                                    | 18.080399 |          18.735599 |
| kubeadm.pod-network-cidr=10.244.0.0/16 |           |                    |
| * Verifying Kubernetes                 |  0.979252 |           0.975357 |
| components...                          |           |                    |
| * Enabled addons:                      |  0.059650 |           0.027562 |
| default-storageclass,                  |           |                    |
| storage-provisioner                    |           |                    |
| * Done! kubectl is now                 |  0.068949 |           0.068855 |
| configured to use "minikube"           |           |                    |
|                                        |  0.005005 |           0.005979 |
+----------------------------------------+-----------+--------------------+

medyagh · 2020-05-21T01:34:38Z

on kvm I see this error:
https://storage.googleapis.com/minikube-builds/logs/7973/023b2dc/KVM_Linux.html#fail_TestMultiNode%2fserial%2fStartAfterStop

-- stdout --
	multinode-20200520172223-30114
	type: Control Plane
	host: Running
	kubelet: Running
	apiserver: Running
	kubeconfig: Configured
	
	multinode-20200520172223-30114-m02
	type: Worker
	host: Running
	kubelet: Running
	
	multinode-20200520172223-30114-m03
	type: Worker
	host: Running
	kubelet: Stopped
	
-- /stdout --
multinode_test.go:165: failed to run minikube status

so it seems like is failing because one of the node's kubelet is "stopped" the question is, is that a normal thing ? is that okay for a node's kubelet to be stopped ?

secondly we need to add this to all the subtests, so we could have seen the logs for the kublet

defer PostMortemLogs(t, profile)

so please add this so we know why it failed

minikube-pr-bot · 2020-05-21T02:05:10Z

kvm2 Driver
Times for minikube: [69.094022169 64.64200985 68.303222807]
Average time for minikube: 67.34641827533333

Times for Minikube (PR 7973): [66.440837307 65.258476876 66.138815794]
Average time for Minikube (PR 7973): 65.94604332566666

Averages Time Per Log

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 7973) |
+--------------------------------+-----------+--------------------+
| * minikube v1.10.1 on Debian   |  0.070982 |           0.062946 |
|                           9.11 |           |                    |
| * Using the kvm2 driver based  |  0.021840 |           0.021469 |
| on existing profile            |           |                    |
| * Starting control plane node  |  0.011021 |           0.003186 |
| minikube in cluster minikube   |           |                    |
| * Creating kvm2 VM (CPUs=2,    | 41.240252 |          40.421340 |
| Memory=3700MB, Disk=20000MB)   |           |                    |
| ...                            |           |                    |
| * Preparing Kubernetes v1.18.2 | 23.772028 |          23.219289 |
| on Docker 19.03.8 ...          |           |                    |
| * Verifying Kubernetes         |  1.478877 |           1.433424 |
| components...                  |           |                    |
| * Enabled addons:              |  0.663317 |           0.708251 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.083905 |           0.071454 |
| configured to use "minikube"   |           |                    |
|                                |  0.004195 |           0.004684 |
+--------------------------------+-----------+--------------------+

docker Driver
Times for minikube: [28.287869485 27.550556663 28.875351937999994]
Average time for minikube: 28.237926028666664

Times for Minikube (PR 7973): [26.781780021 27.878319794 28.062798046999998]
Average time for Minikube (PR 7973): 27.574299287333332

Averages Time Per Log

+----------------------------------------+-----------+--------------------+
|                  LOG                   | MINIKUBE  | MINIKUBE (PR 7973) |
+----------------------------------------+-----------+--------------------+
| * minikube v1.10.1 on Debian           |  0.075923 |           0.082838 |
|                                   9.11 |           |                    |
| * Using the docker driver              |  0.002504 |           0.003141 |
| based on existing profile              |           |                    |
| * Starting control plane node          |  0.064468 |           0.061863 |
| minikube in cluster minikube           |           |                    |
| * Creating docker container            |  7.972385 |           8.033783 |
| (CPUs=2, Memory=3700MB) ...            |           |                    |
| * Preparing Kubernetes v1.18.2         |  0.136577 |           0.127828 |
| on Docker 19.03.2 ...                  |           |                    |
|   -                                    | 18.904683 |          18.286286 |
| kubeadm.pod-network-cidr=10.244.0.0/16 |           |                    |
| * Verifying Kubernetes                 |  0.978632 |           0.820805 |
| components...                          |           |                    |
| * Enabled addons:                      |  0.021925 |           0.087068 |
| default-storageclass,                  |           |                    |
| storage-provisioner                    |           |                    |
| * Done! kubectl is now                 |  0.074047 |           0.065739 |
| configured to use "minikube"           |           |                    |
|                                        |  0.006781 |           0.004950 |
+----------------------------------------+-----------+--------------------+

…estart

minikube-pr-bot · 2020-05-21T22:06:46Z

kvm2 Driver
Times for minikube: [67.27415193099999 63.901112678999986 66.238276093]
Average time for minikube: 65.80451356766667

Times for Minikube (PR 7973): [64.470353064 63.528211406 66.112156886]
Average time for Minikube (PR 7973): 64.70357378533333

Averages Time Per Log

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 7973) |
+--------------------------------+-----------+--------------------+
| * minikube v1.10.1 on Debian   |  0.061293 |           0.058992 |
|                           9.11 |           |                    |
| * Using the kvm2 driver based  |  0.018461 |           0.020271 |
| on existing profile            |           |                    |
| * Starting control plane node  |  0.003351 |           0.005656 |
| minikube in cluster minikube   |           |                    |
| * Creating kvm2 VM (CPUs=2,    | 41.869861 |          40.534710 |
| Memory=3700MB, Disk=20000MB)   |           |                    |
| ...                            |           |                    |
| * Preparing Kubernetes v1.18.2 | 21.725706 |          22.225829 |
| on Docker 19.03.8 ...          |           |                    |
| * Verifying Kubernetes         |  1.689087 |           1.227469 |
| components...                  |           |                    |
| * Enabled addons:              |  0.351653 |           0.557322 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.080516 |           0.069570 |
| configured to use "minikube"   |           |                    |
|                                |  0.004585 |           0.003754 |
+--------------------------------+-----------+--------------------+

docker Driver
Times for minikube: [25.793777625999997 25.759186884000002 27.124523473]
Average time for minikube: 26.225829327666663

Times for Minikube (PR 7973): [26.455807898000003 28.316644833 26.156449873]
Average time for Minikube (PR 7973): 26.976300868

Averages Time Per Log

+----------------------------------------+-----------+--------------------+
|                  LOG                   | MINIKUBE  | MINIKUBE (PR 7973) |
+----------------------------------------+-----------+--------------------+
| * minikube v1.10.1 on Debian           |  0.080368 |           0.073804 |
|                                   9.11 |           |                    |
| * Using the docker driver              |  0.002868 |           0.002566 |
| based on existing profile              |           |                    |
| * Starting control plane node          |  0.057502 |           0.059059 |
| minikube in cluster minikube           |           |                    |
| * Creating docker container            |  7.565144 |           7.680765 |
| (CPUs=2, Memory=3700MB) ...            |           |                    |
| * Preparing Kubernetes v1.18.2         |  0.119105 |           0.119908 |
| on Docker 19.03.2 ...                  |           |                    |
|   -                                    | 16.779161 |          17.918437 |
| kubeadm.pod-network-cidr=10.244.0.0/16 |           |                    |
| * Verifying Kubernetes                 |  1.444630 |           0.936461 |
| components...                          |           |                    |
| * Enabled addons:                      |  0.110595 |           0.114348 |
| default-storageclass,                  |           |                    |
| storage-provisioner                    |           |                    |
| * Done! kubectl is now                 |  0.063008 |           0.064386 |
| configured to use "minikube"           |           |                    |
|                                        |  0.003447 |           0.006568 |
+----------------------------------------+-----------+--------------------+

minikube-pr-bot · 2020-05-21T22:56:53Z

kvm2 Driver
Times for minikube: [61.824525185 66.166375301 65.93158536]
Average time for minikube: 64.64082861533332

Times for Minikube (PR 7973): [66.80655133799999 64.81349420400001 63.33818678499999]
Average time for Minikube (PR 7973): 64.98607744233334

Averages Time Per Log

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 7973) |
+--------------------------------+-----------+--------------------+
| * minikube v1.10.1 on Debian   |  0.063572 |           0.060604 |
|                           9.11 |           |                    |
| * Using the kvm2 driver based  |  0.020147 |           0.019294 |
| on existing profile            |           |                    |
| * Starting control plane node  |  0.003578 |           0.003229 |
| minikube in cluster minikube   |           |                    |
| * Creating kvm2 VM (CPUs=2,    | 39.840189 |          39.993163 |
| Memory=3700MB, Disk=20000MB)   |           |                    |
| ...                            |           |                    |
| * Preparing Kubernetes v1.18.2 | 22.676972 |          22.870146 |
| on Docker 19.03.8 ...          |           |                    |
| * Verifying Kubernetes         |  1.366835 |           1.288459 |
| components...                  |           |                    |
| * Enabled addons:              |  0.590443 |           0.665858 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.074712 |           0.079987 |
| configured to use "minikube"   |           |                    |
|                                |  0.004381 |           0.005337 |
+--------------------------------+-----------+--------------------+

docker Driver
Times for minikube: [27.440570063000003 26.178439203 27.717908791]
Average time for minikube: 27.112306019000002

Times for Minikube (PR 7973): [27.084916353 26.40355993 25.450216769999997]
Average time for Minikube (PR 7973): 26.312897684333333

Averages Time Per Log

+----------------------------------------+-----------+--------------------+
|                  LOG                   | MINIKUBE  | MINIKUBE (PR 7973) |
+----------------------------------------+-----------+--------------------+
| * minikube v1.10.1 on Debian           |  0.075381 |           0.074286 |
|                                   9.11 |           |                    |
| * Using the docker driver              |  0.002656 |           0.002461 |
| based on existing profile              |           |                    |
| * Starting control plane node          |  0.058707 |           0.058461 |
| minikube in cluster minikube           |           |                    |
| * Creating docker container            |  7.722725 |           7.811579 |
| (CPUs=2, Memory=3700MB) ...            |           |                    |
| * Preparing Kubernetes v1.18.2         |  0.147674 |           0.145643 |
| on Docker 19.03.2 ...                  |           |                    |
|   -                                    | 17.631841 |          17.296636 |
| kubeadm.pod-network-cidr=10.244.0.0/16 |           |                    |
| * Verifying Kubernetes                 |  1.402025 |           0.851299 |
| components...                          |           |                    |
| * Enabled addons:                      |  0.002836 |           0.002571 |
| default-storageclass,                  |           |                    |
| storage-provisioner                    |           |                    |
| * Done! kubectl is now                 |  0.064455 |           0.064891 |
| configured to use "minikube"           |           |                    |
|                                        |  0.004005 |           0.005071 |
+----------------------------------------+-----------+--------------------+

TravisBuddy · 2020-05-22T00:19:08Z

Travis tests have failed

Hey @sharifelgamal,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

View build log

make test

which go-bindata || GO111MODULE=off GOBIN="/home/travis/gopath/bin" go get github.com/jteeuwen/go-bindata/...
PATH="/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.13.8.linux.amd64/bin:/home/travis/bin:/home/travis/bin:/home/travis/.local/bin:/usr/local/lib/jvm/openjdk11/bin:/opt/pyenv/shims:/home/travis/.phpenv/shims:/home/travis/perl5/perlbrew/bin:/home/travis/.nvm/versions/node/v8.12.0/bin:/home/travis/.rvm/gems/ruby-2.5.3/bin:/home/travis/.rvm/gems/ruby-2.5.3@global/bin:/home/travis/.rvm/rubies/ruby-2.5.3/bin:/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.11.1.linux.amd64/bin:/usr/local/maven-3.6.0/bin:/usr/local/cmake-3.12.4/bin:/usr/local/clang-7.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/home/travis/.rvm/bin:/home/travis/.phpenv/bin:/opt/pyenv/bin:/home/travis/.yarn/bin:/home/travis/gopath/bin" go-bindata -nomemcopy -o pkg/minikube/assets/assets.go -pkg assets deploy/addons/...
gofmt -s -w pkg/minikube/assets/assets.go
which go-bindata || GO111MODULE=off GOBIN="/home/travis/gopath/bin" go get github.com/jteeuwen/go-bindata/...
/home/travis/gopath/bin/go-bindata
PATH="/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.13.8.linux.amd64/bin:/home/travis/bin:/home/travis/bin:/home/travis/.local/bin:/usr/local/lib/jvm/openjdk11/bin:/opt/pyenv/shims:/home/travis/.phpenv/shims:/home/travis/perl5/perlbrew/bin:/home/travis/.nvm/versions/node/v8.12.0/bin:/home/travis/.rvm/gems/ruby-2.5.3/bin:/home/travis/.rvm/gems/ruby-2.5.3@global/bin:/home/travis/.rvm/rubies/ruby-2.5.3/bin:/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.11.1.linux.amd64/bin:/usr/local/maven-3.6.0/bin:/usr/local/cmake-3.12.4/bin:/usr/local/clang-7.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/home/travis/.rvm/bin:/home/travis/.phpenv/bin:/opt/pyenv/bin:/home/travis/.yarn/bin:/home/travis/gopath/bin" go-bindata -nomemcopy -o pkg/minikube/translate/translations.go -pkg translate translations/...
gofmt -s -w pkg/minikube/translate/translations.go
MINIKUBE_LDFLAGS="-X k8s.io/minikube/pkg/version.version=v1.10.1 -X k8s.io/minikube/pkg/version.isoVersion=v1.10.0 -X k8s.io/minikube/pkg/version.isoPath=minikube/iso -X k8s.io/minikube/pkg/version.gitCommitID="de11522107a3a435cc1e4878b5fa39766e046db9"" ./test.sh
= schema_check ==========================================================
go: downloading github.com/xeipuuv/gojsonschema v0.0.0-20180618132009-1d523034197f
go: extracting github.com/xeipuuv/gojsonschema v0.0.0-20180618132009-1d523034197f
go: downloading github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415
go: extracting github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415
go: downloading github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f
go: extracting github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f
go: finding github.com/xeipuuv/gojsonschema v0.0.0-20180618132009-1d523034197f
go: finding github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415
go: finding github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f
ok
= go test ===============================================================
go: downloading github.com/machine-drivers/docker-machine-driver-vmware v0.1.1
go: downloading github.com/machine-drivers/machine v0.7.1-0.20200323212942-41eb826190d8
go: downloading k8s.io/apimachinery v0.17.3
go: downloading github.com/spf13/cobra v1.0.0
go: downloading github.com/docker/go-units v0.4.0
go: downloading github.com/blang/semver v3.5.0+incompatible
go: downloading github.com/shirou/gopsutil v2.18.12+incompatible
go: downloading github.com/spf13/pflag v1.0.5
go: downloading github.com/pkg/profile v0.0.0-20161223203901-3a8809bd8a80
go: extracting github.com/blang/semver v3.5.0+incompatible
go: downloading github.com/mitchellh/go-ps v0.0.0-20170309133038-4fdf99ab2936
go: extracting github.com/spf13/pflag v1.0.5
go: downloading github.com/spf13/viper v1.6.1
go: extracting github.com/machine-drivers/docker-machine-driver-vmware v0.1.1
go: extracting github.com/pkg/profile v0.0.0-20161223203901-3a8809bd8a80
go: extracting github.com/mitchellh/go-ps v0.0.0-20170309133038-4fdf99ab2936
go: extracting github.com/shirou/gopsutil v2.18.12+incompatible
go: extracting github.com/docker/go-units v0.4.0
go: downloading k8s.io/kubectl v0.17.3
go: extracting github.com/spf13/viper v1.6.1
go: extracting github.com/spf13/cobra v1.0.0
go: extracting github.com/machine-drivers/machine v0.7.1-0.20200323212942-41eb826190d8
go: extracting k8s.io/apimachinery v0.17.3
go: downloading github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51
go: downloading github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2
go: downloading golang.org/x/crypto v0.0.0-20200302210943-78000ba7a073
go: extracting github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2
go: extracting github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51
go: downloading github.com/mitchellh/mapstructure v1.1.2
go: downloading k8s.io/client-go v0.17.3
go: downloading github.com/mattn/go-isatty v0.0.11
go: downloading k8s.io/api v0.17.3
go: extracting github.com/mitchellh/mapstructure v1.1.2
go: downloading cloud.google.com/go v0.45.1
go: downloading github.com/cloudfoundry-attic/jibber_jabber v0.0.0-20151120183258-bcc4c8345a21
go: downloading golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e
go: downloading github.com/google/go-containerregistry v0.0.0-20200131185320-aec8da010de2
go: extracting github.com/mattn/go-isatty v0.0.11
go: downloading github.com/magiconair/properties v1.8.1
go: extracting github.com/cloudfoundry-attic/jibber_jabber v0.0.0-20151120183258-bcc4c8345a21
go: extracting golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e
go: downloading github.com/subosito/gotenv v1.2.0
go: extracting k8s.io/kubectl v0.17.3
go: extracting github.com/subosito/gotenv v1.2.0
go: downloading github.com/pkg/errors v0.9.1
go: extracting golang.org/x/crypto v0.0.0-20200302210943-78000ba7a073
go: extracting github.com/magiconair/properties v1.8.1
go: downloading github.com/juju/mutex v0.0.0-20180619145857-d21b13acf4bf
go: extracting github.com/pkg/errors v0.9.1
go: downloading github.com/spf13/jwalterweatherman v1.1.0
go: extracting github.com/google/go-containerregistry v0.0.0-20200131185320-aec8da010de2
go: extracting k8s.io/client-go v0.17.3
go: extracting github.com/spf13/jwalterweatherman v1.1.0
go: downloading github.com/sayboras/dockerclient v0.0.0-20191231050035-015626177a97
go: extracting github.com/juju/mutex v0.0.0-20180619145857-d21b13acf4bf
go: downloading k8s.io/klog v1.0.0
go: extracting k8s.io/klog v1.0.0
go: downloading github.com/libvirt/libvirt-go v3.4.0+incompatible
go: extracting github.com/sayboras/dockerclient v0.0.0-20191231050035-015626177a97
go: downloading github.com/golang-collections/collections v0.0.0-20130729185459-604e922904d3
go: extracting github.com/libvirt/libvirt-go v3.4.0+incompatible
go: downloading github.com/cheggaaa/pb v1.0.27
go: extracting github.com/cheggaaa/pb v1.0.27
go: downloading github.com/cheggaaa/pb/v3 v3.0.1
go: downloading github.com/afbjorklund/go-getter v1.4.1-0.20190910175809-eb9f6c26742c
go: extracting github.com/golang-collections/collections v0.0.0-20130729185459-604e922904d3
go: downloading github.com/pelletier/go-toml v1.6.0
go: extracting github.com/pelletier/go-toml v1.6.0
go: extracting github.com/afbjorklund/go-getter v1.4.1-0.20190910175809-eb9f6c26742c
go: downloading github.com/otiai10/copy v1.0.2
go: extracting github.com/cheggaaa/pb/v3 v3.0.1
go: downloading github.com/gogo/protobuf v1.3.1
go: extracting github.com/otiai10/copy v1.0.2
go: downloading github.com/google/gofuzz v1.0.0
go: extracting cloud.google.com/go v0.45.1
go: extracting github.com/google/gofuzz v1.0.0
go: downloading github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: downloading github.com/MakeNowJust/heredoc v0.0.0-20170808103936-bb23615498cd
go: downloading github.com/hooklift/iso9660 v0.0.0-20170318115843-1cf07e5970d8
go: extracting k8s.io/api v0.17.3
go: downloading golang.org/x/sys v0.0.0-20200124204421-9fbb57f87de9
go: extracting github.com/MakeNowJust/heredoc v0.0.0-20170808103936-bb23615498cd
go: downloading github.com/docker/cli v0.0.0-20200303162255-7d407207c304
go: downloading github.com/hashicorp/hcl v1.0.0
go: extracting github.com/hooklift/iso9660 v0.0.0-20170318115843-1cf07e5970d8
go: downloading github.com/mitchellh/go-testing-interface v1.0.0
go: extracting github.com/mitchellh/go-testing-interface v1.0.0
go: downloading github.com/aws/aws-sdk-go v1.27.1
go: extracting github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: downloading golang.org/x/time v0.0.0-20190308202827-9d24e82272b4
go: extracting github.com/hashicorp/hcl v1.0.0
go: extracting golang.org/x/time v0.0.0-20190308202827-9d24e82272b4
go: downloading github.com/fatih/color v1.7.0
go: extracting github.com/fatih/color v1.7.0
go: extracting github.com/docker/cli v0.0.0-20200303162255-7d407207c304
go: downloading github.com/juju/clock v0.0.0-20190205081909-9c5c9712527c
go: extracting golang.org/x/sys v0.0.0-20200124204421-9fbb57f87de9
go: extracting github.com/juju/clock v0.0.0-20190205081909-9c5c9712527c
go: downloading github.com/modern-go/reflect2 v1.0.1
go: extracting github.com/modern-go/reflect2 v1.0.1
go: downloading gopkg.in/yaml.v2 v2.2.8
go: extracting github.com/gogo/protobuf v1.3.1
go: extracting gopkg.in/yaml.v2 v2.2.8
go: downloading golang.org/x/text v0.3.2
go: downloading google.golang.org/api v0.9.0
go: downloading google.golang.org/genproto v0.0.0-20200117163144-32f20d992d24
go: downloading github.com/juju/errors v0.0.0-20190806202954-0232dcc7464d
go: downloading github.com/hashicorp/go-safetemp v1.0.0
go: downloading github.com/mitchellh/go-wordwrap v1.0.0
go: downloading github.com/googleapis/gax-go/v2 v2.0.5
go: downloading k8s.io/kubernetes v1.17.3
go: extracting golang.org/x/text v0.3.2
go: extracting github.com/juju/errors v0.0.0-20190806202954-0232dcc7464d
go: extracting github.com/googleapis/gax-go/v2 v2.0.5
go: downloading google.golang.org/grpc v1.26.0
go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: extracting github.com/mitchellh/go-wordwrap v1.0.0
go: extracting github.com/hashicorp/go-safetemp v1.0.0
go: downloading golang.org/x/net v0.0.0-20200301022130-244492dfa37a
go: extracting github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: extracting google.golang.org/grpc v1.26.0
go: extracting golang.org/x/net v0.0.0-20200301022130-244492dfa37a
go: downloading github.com/mitchellh/go-homedir v1.1.0
go: downloading github.com/json-iterator/go v1.1.8
go: downloading github.com/docker/docker v1.4.2-0.20190924003213-a8608b5b67c7
go: extracting github.com/json-iterator/go v1.1.8
go: downloading github.com/olekukonko/tablewriter v0.0.4
go: extracting github.com/olekukonko/tablewriter v0.0.4
go: downloading k8s.io/utils v0.0.0-20200229041039-0a110f9eb7ab
go: extracting k8s.io/utils v0.0.0-20200229041039-0a110f9eb7ab
go: extracting github.com/mitchellh/go-homedir v1.1.0
go: extracting google.golang.org/genproto v0.0.0-20200117163144-32f20d992d24
go: downloading github.com/opencontainers/go-digest v1.0.0-rc1
go: extracting github.com/docker/docker v1.4.2-0.20190924003213-a8608b5b67c7
go: extracting github.com/aws/aws-sdk-go v1.27.1
go: extracting k8s.io/kubernetes v1.17.3
go: extracting github.com/opencontainers/go-digest v1.0.0-rc1
go: extracting google.golang.org/api v0.9.0
go: downloading github.com/russross/blackfriday v1.5.3-0.20200218234912-41c5fccfd6f6
go: downloading github.com/cenkalti/backoff v2.2.1+incompatible
go: downloading golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45
go: downloading github.com/VividCortex/ewma v1.1.1
go: downloading github.com/golang/protobuf v1.3.2
go: downloading github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d
go: extracting github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d
go: extracting github.com/russross/blackfriday v1.5.3-0.20200218234912-41c5fccfd6f6
go: extracting github.com/VividCortex/ewma v1.1.1
go: downloading github.com/imdario/mergo v0.3.8
go: extracting github.com/cenkalti/backoff v2.2.1+incompatible
go: extracting golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45
go: downloading k8s.io/cluster-bootstrap v0.17.3
go: downloading github.com/spf13/cast v1.3.1
go: downloading github.com/fsnotify/fsnotify v1.4.7
go: downloading github.com/docker/docker-credential-helpers v0.6.3
go: downloading github.com/mattn/go-colorable v0.1.2
go: downloading github.com/docker/distribution v2.7.1+incompatible
go: downloading github.com/ulikunitz/xz v0.5.5
go: extracting github.com/golang/protobuf v1.3.2
go: downloading github.com/docker/go-connections v0.4.0
go: downloading k8s.io/kubelet v0.17.3
go: extracting github.com/mattn/go-colorable v0.1.2
go: extracting github.com/fsnotify/fsnotify v1.4.7
go: extracting github.com/imdario/mergo v0.3.8
go: extracting github.com/docker/docker-credential-helpers v0.6.3
go: downloading github.com/mattn/go-runewidth v0.0.7
go: extracting github.com/spf13/cast v1.3.1
go: extracting github.com/docker/distribution v2.7.1+incompatible
go: downloading gopkg.in/inf.v0 v0.9.1
go: downloading github.com/hashicorp/golang-lru v0.5.3
go: extracting github.com/ulikunitz/xz v0.5.5
go: extracting k8s.io/cluster-bootstrap v0.17.3
go: downloading github.com/google/go-cmp v0.3.2-0.20191028172631-481baca67f93
go: downloading github.com/pkg/browser v0.0.0-20160118053552-9302be274faa
go: downloading go.opencensus.io v0.22.0
go: downloading github.com/spf13/afero v1.2.2
go: extracting github.com/docker/go-connections v0.4.0
go: downloading github.com/cpuguy83/go-md2man v1.0.10
go: extracting k8s.io/kubelet v0.17.3
go: downloading github.com/containerd/containerd v1.3.1-0.20191213020239-082f7e3aed57
go: extracting gopkg.in/inf.v0 v0.9.1
go: extracting github.com/google/go-cmp v0.3.2-0.20191028172631-481baca67f93
go: extracting github.com/spf13/afero v1.2.2
go: downloading github.com/googleapis/gnostic v0.3.0
go: extracting github.com/hashicorp/golang-lru v0.5.3
go: downloading github.com/davecgh/go-spew v1.1.1
go: extracting go.opencensus.io v0.22.0
go: extracting github.com/pkg/browser v0.0.0-20160118053552-9302be274faa
go: extracting github.com/mattn/go-runewidth v0.0.7
go: downloading github.com/hashicorp/go-cleanhttp v0.5.0
go: downloading github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af
go: downloading k8s.io/kube-proxy v0.17.3
go: downloading gopkg.in/ini.v1 v1.51.1
go: downloading github.com/c4milo/gotoolkit v0.0.0-20170318115440-bcc06269efa9
go: downloading github.com/hashicorp/go-version v1.1.0
go: extracting github.com/cpuguy83/go-md2man v1.0.10
go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.0
go: extracting github.com/containerd/containerd v1.3.1-0.20191213020239-082f7e3aed57
go: downloading sigs.k8s.io/yaml v1.1.0
go: extracting github.com/davecgh/go-spew v1.1.1
go: extracting github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af
go: extracting github.com/googleapis/gnostic v0.3.0
go: extracting github.com/c4milo/gotoolkit v0.0.0-20170318115440-bcc06269efa9
go: extracting github.com/hashicorp/go-cleanhttp v0.5.0
go: extracting k8s.io/kube-proxy v0.17.3
go: extracting gopkg.in/ini.v1 v1.51.1
go: downloading sigs.k8s.io/sig-storage-lib-external-provisioner v4.0.0+incompatible
go: downloading k8s.io/component-base v0.17.3
go: extracting github.com/hashicorp/go-version v1.1.0
go: downloading github.com/intel-go/cpuid v0.0.0-20181003105527-1a4a6f06a1c6
go: downloading github.com/sirupsen/logrus v1.4.2
go: downloading github.com/docker/spdystream v0.0.0-20160310174837-449fdfce4d96
go: downloading github.com/opencontainers/image-spec v1.0.1
go: downloading github.com/google/uuid v1.1.1
go: extracting github.com/cpuguy83/go-md2man/v2 v2.0.0
go: downloading github.com/russross/blackfriday/v2 v2.0.1
go: extracting sigs.k8s.io/yaml v1.1.0
go: extracting github.com/docker/spdystream v0.0.0-20160310174837-449fdfce4d96
go: extracting github.com/sirupsen/logrus v1.4.2
go: extracting sigs.k8s.io/sig-storage-lib-external-provisioner v4.0.0+incompatible
go: extracting github.com/intel-go/cpuid v0.0.0-20181003105527-1a4a6f06a1c6
go: extracting k8s.io/component-base v0.17.3
go: extracting github.com/opencontainers/image-spec v1.0.1
go: extracting github.com/google/uuid v1.1.1
go: downloading github.com/prometheus/client_golang v1.1.0
go: downloading github.com/miekg/dns v1.1.4
go: downloading github.com/golang/groupcache v0.0.0-20190129154638-5b532d6fd5ef
go: downloading k8s.io/kube-openapi v0.0.0-20191107075043-30be4d16710a
go: extracting github.com/russross/blackfriday/v2 v2.0.1
go: downloading github.com/shurcooL/sanitized_anchor_name v1.0.0
go: extracting github.com/miekg/dns v1.1.4
go: extracting github.com/golang/groupcache v0.0.0-20190129154638-5b532d6fd5ef
go: extracting github.com/prometheus/client_golang v1.1.0
go: extracting k8s.io/kube-openapi v0.0.0-20191107075043-30be4d16710a
go: downloading github.com/prometheus/common v0.6.0
go: downloading github.com/prometheus/procfs v0.0.5
go: downloading github.com/beorn7/perks v1.0.1
go: downloading github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: extracting github.com/shurcooL/sanitized_anchor_name v1.0.0
go: extracting github.com/beorn7/perks v1.0.1
go: extracting github.com/prometheus/common v0.6.0
go: extracting github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
go: extracting github.com/prometheus/procfs v0.0.5
go: extracting github.com/matttproud/golang_protobuf_extensions v1.0.1
go: finding github.com/machine-drivers/machine v0.7.1-0.20200323212942-41eb826190d8
go: finding github.com/golang-collections/collections v0.0.0-20130729185459-604e922904d3
go: finding github.com/pkg/errors v0.9.1
go: finding github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: finding github.com/pkg/profile v0.0.0-20161223203901-3a8809bd8a80
go: finding github.com/docker/go-units v0.4.0
go: finding github.com/spf13/cobra v1.0.0
go: finding github.com/blang/semver v3.5.0+incompatible
go: finding github.com/olekukonko/tablewriter v0.0.4
go: finding k8s.io/client-go v0.17.3
go: finding github.com/hooklift/iso9660 v0.0.0-20170318115843-1cf07e5970d8
go: finding github.com/libvirt/libvirt-go v3.4.0+incompatible
go: finding github.com/spf13/viper v1.6.1
go: finding golang.org/x/crypto v0.0.0-20200302210943-78000ba7a073
go: finding github.com/mattn/go-runewidth v0.0.7
go: finding github.com/imdario/mergo v0.3.8
go: finding github.com/juju/clock v0.0.0-20190205081909-9c5c9712527c
go: finding k8s.io/api v0.17.3
go: finding github.com/mattn/go-isatty v0.0.11
go: finding github.com/spf13/pflag v1.0.5
go: finding github.com/docker/docker v1.4.2-0.20190924003213-a8608b5b67c7
go: finding k8s.io/apimachinery v0.17.3
go: finding github.com/juju/mutex v0.0.0-20180619145857-d21b13acf4bf
go: finding golang.org/x/sys v0.0.0-20200124204421-9fbb57f87de9
go: finding github.com/c4milo/gotoolkit v0.0.0-20170318115440-bcc06269efa9
go: finding github.com/google/go-containerregistry v0.0.0-20200131185320-aec8da010de2
go: finding github.com/juju/errors v0.0.0-20190806202954-0232dcc7464d
go: finding github.com/gogo/protobuf v1.3.1
go: finding github.com/google/uuid v1.1.1
go: finding github.com/docker/cli v0.0.0-20200303162255-7d407207c304
go: finding github.com/fsnotify/fsnotify v1.4.7
go: finding github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51
go: finding github.com/mitchellh/go-ps v0.0.0-20170309133038-4fdf99ab2936
go: finding github.com/pkg/browser v0.0.0-20160118053552-9302be274faa
go: finding github.com/hashicorp/hcl v1.0.0
go: finding github.com/shirou/gopsutil v2.18.12+incompatible
go: finding cloud.google.com/go v0.45.1
go: finding github.com/cloudfoundry-attic/jibber_jabber v0.0.0-20151120183258-bcc4c8345a21
go: finding github.com/magiconair/properties v1.8.1
go: finding github.com/google/gofuzz v1.0.0
go: finding github.com/machine-drivers/docker-machine-driver-vmware v0.1.1
go: finding github.com/mitchellh/mapstructure v1.1.2
go: finding sigs.k8s.io/sig-storage-lib-external-provisioner v4.0.0+incompatible
go: finding golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e
go: finding k8s.io/klog v1.0.0
go: finding golang.org/x/text v0.3.2
go: finding github.com/pelletier/go-toml v1.6.0
go: finding github.com/otiai10/copy v1.0.2
go: finding github.com/cpuguy83/go-md2man/v2 v2.0.0
go: finding github.com/docker/distribution v2.7.1+incompatible
go: finding github.com/cenkalti/backoff v2.2.1+incompatible
go: finding github.com/golang/protobuf v1.3.2
go: finding github.com/docker/docker-credential-helpers v0.6.3
go: finding gopkg.in/inf.v0 v0.9.1
go: finding github.com/spf13/afero v1.2.2
go: finding github.com/googleapis/gnostic v0.3.0
go: finding github.com/spf13/cast v1.3.1
go: finding golang.org/x/net v0.0.0-20200301022130-244492dfa37a
go: finding github.com/russross/blackfriday/v2 v2.0.1
go: finding github.com/prometheus/client_golang v1.1.0
go: finding github.com/googleapis/gax-go/v2 v2.0.5
go: finding github.com/intel-go/cpuid v0.0.0-20181003105527-1a4a6f06a1c6
go: finding github.com/spf13/jwalterweatherman v1.1.0
go: finding k8s.io/kubernetes v1.17.3
go: finding gopkg.in/yaml.v2 v2.2.8
go: finding github.com/cheggaaa/pb/v3 v3.0.1
go: finding google.golang.org/genproto v0.0.0-20200117163144-32f20d992d24
go: finding golang.org/x/time v0.0.0-20190308202827-9d24e82272b4
go: finding github.com/opencontainers/go-digest v1.0.0-rc1
go: finding google.golang.org/api v0.9.0
go: finding google.golang.org/grpc v1.26.0
go: finding github.com/subosito/gotenv v1.2.0
go: finding github.com/shurcooL/sanitized_anchor_name v1.0.0
go: finding github.com/afbjorklund/go-getter v1.4.1-0.20190910175809-eb9f6c26742c
go: finding go.opencensus.io v0.22.0
go: finding gopkg.in/ini.v1 v1.51.1
go: finding github.com/VividCortex/ewma v1.1.1
go: finding k8s.io/component-base v0.17.3
go: finding github.com/docker/go-connections v0.4.0
go: finding k8s.io/kubectl v0.17.3
go: finding github.com/json-iterator/go v1.1.8
go: finding github.com/fatih/color v1.7.0
go: finding github.com/beorn7/perks v1.0.1
go: finding github.com/modern-go/reflect2 v1.0.1
go: finding github.com/mattn/go-colorable v0.1.2
go: finding github.com/opencontainers/image-spec v1.0.1
go: finding github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: finding golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45
go: finding github.com/hashicorp/golang-lru v0.5.3
go: finding github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: finding github.com/MakeNowJust/heredoc v0.0.0-20170808103936-bb23615498cd
go: finding github.com/prometheus/common v0.6.0
go: finding github.com/sayboras/dockerclient v0.0.0-20191231050035-015626177a97
go: finding github.com/russross/blackfriday v1.5.3-0.20200218234912-41c5fccfd6f6
go: finding github.com/containerd/containerd v1.3.1-0.20191213020239-082f7e3aed57
go: finding github.com/davecgh/go-spew v1.1.1
go: finding github.com/sirupsen/logrus v1.4.2
go: finding github.com/prometheus/procfs v0.0.5
go: finding k8s.io/cluster-bootstrap v0.17.3
go: finding github.com/matttproud/golang_protobuf_extensions v1.0.1
go: finding sigs.k8s.io/yaml v1.1.0
go: finding k8s.io/utils v0.0.0-20200229041039-0a110f9eb7ab
go: finding github.com/mitchellh/go-wordwrap v1.0.0
go: finding github.com/aws/aws-sdk-go v1.27.1
go: finding github.com/golang/groupcache v0.0.0-20190129154638-5b532d6fd5ef
go: finding k8s.io/kube-proxy v0.17.3
go: finding k8s.io/kubelet v0.17.3
go: finding github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d
go: finding github.com/hashicorp/go-cleanhttp v0.5.0
go: finding github.com/miekg/dns v1.1.4
go: finding github.com/hashicorp/go-safetemp v1.0.0
go: finding github.com/hashicorp/go-version v1.1.0
go: finding github.com/mitchellh/go-homedir v1.1.0
go: finding github.com/mitchellh/go-testing-interface v1.0.0
go: finding github.com/ulikunitz/xz v0.5.5
go: finding k8s.io/kube-openapi v0.0.0-20191107075043-30be4d16710a
go: finding github.com/google/go-cmp v0.3.2-0.20191028172631-481baca67f93
go: finding github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af
go: finding github.com/docker/spdystream v0.0.0-20160310174837-449fdfce4d96
go: finding github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2
go: downloading github.com/evanphx/json-patch v4.5.0+incompatible
go: downloading github.com/pmezard/go-difflib v1.0.0
go: extracting github.com/pmezard/go-difflib v1.0.0
go: extracting github.com/evanphx/json-patch v4.5.0+incompatible
go: finding github.com/pmezard/go-difflib v1.0.0
go: finding github.com/evanphx/json-patch v4.5.0+incompatible
ok  	k8s.io/minikube/cmd/minikube/cmd	6.268s	coverage: 20.7% of statements
ok  	k8s.io/minikube/cmd/minikube/cmd/config	0.151s	coverage: 21.6% of statements
E0521 22:47:22.412081   11651 out.go:102] [unset errFile]: * 
E0521 22:47:22.412303   11651 out.go:102] [unset errFile]: X Error getting primary control plane: remove /tmp/profile366505817/.minikube/profiles/config.json: no such file or directory
E0521 22:47:22.412320   11651 out.go:102] [unset errFile]: * 
E0521 22:47:22.412330   11651 out.go:102] [unset errFile]: * minikube is exiting due to an error. If the above message is not useful, open an issue:
E0521 22:47:22.412341   11651 out.go:102] [unset errFile]:   - https://github.com/kubernetes/minikube/issues/new/choose
FAIL	k8s.io/minikube/pkg/addons	0.029s
ok  	k8s.io/minikube/pkg/drivers	0.007s	coverage: 19.6% of statements
ok  	k8s.io/minikube/pkg/drivers/hyperkit	0.024s	coverage: 77.3% of statements
ok  	k8s.io/minikube/pkg/drivers/kvm	0.024s	coverage: 2.3% of statements
ok  	k8s.io/minikube/pkg/minikube/bootstrapper	0.603s	coverage: 55.7% of statements
ok  	k8s.io/minikube/pkg/minikube/bootstrapper/bsutil	0.054s	coverage: 63.2% of statements
ok  	k8s.io/minikube/pkg/minikube/bootstrapper/bsutil/ktmpl	0.014s	coverage: 100.0% of statements
ok  	k8s.io/minikube/pkg/minikube/bootstrapper/images	0.006s	coverage: 88.9% of statements
ok  	k8s.io/minikube/pkg/minikube/cluster	0.073s	coverage: 18.4% of statements
ok  	k8s.io/minikube/pkg/minikube/command	0.011s	coverage: 3.8% of statements
ok  	k8s.io/minikube/pkg/minikube/config	0.174s	coverage: 67.3% of statements
ok  	k8s.io/minikube/pkg/minikube/cruntime	0.040s	coverage: 40.6% of statements
ok  	k8s.io/minikube/pkg/minikube/docker	0.011s	coverage: 20.8% of statements
ok  	k8s.io/minikube/pkg/minikube/driver	0.012s	coverage: 37.3% of statements
ok  	k8s.io/minikube/pkg/minikube/extract	0.007s	coverage: 58.8% of statements
ok  	k8s.io/minikube/pkg/minikube/kubeconfig	0.024s	coverage: 77.0% of statements
ok  	k8s.io/minikube/pkg/minikube/localpath	0.009s	coverage: 51.9% of statements
ok  	k8s.io/minikube/pkg/minikube/logs	0.022s	coverage: 1.3% of statements
ok  	k8s.io/minikube/pkg/minikube/machine	1.337s	coverage: 38.6% of statements
ok  	k8s.io/minikube/pkg/minikube/notify	0.011s	coverage: 92.9% of statements
ok  	k8s.io/minikube/pkg/minikube/out	0.041s	coverage: 67.4% of statements
ok  	k8s.io/minikube/pkg/minikube/perf	0.268s	coverage: 54.7% of statements
ok  	k8s.io/minikube/pkg/minikube/problem	0.009s	coverage: 91.7% of statements
ok  	k8s.io/minikube/pkg/minikube/proxy	0.009s	coverage: 77.0% of statements
ok  	k8s.io/minikube/pkg/minikube/registry	0.011s	coverage: 75.5% of statements
ok  	k8s.io/minikube/pkg/minikube/service	0.080s	coverage: 84.2% of statements
ok  	k8s.io/minikube/pkg/minikube/storageclass	0.017s	coverage: 100.0% of statements
ok  	k8s.io/minikube/pkg/minikube/sysinit	0.022s	coverage: 6.5% of statements
ok  	k8s.io/minikube/pkg/minikube/translate	0.011s	coverage: 10.5% of statements
ok  	k8s.io/minikube/pkg/minikube/tunnel	1.981s	coverage: 64.2% of statements
ok  	k8s.io/minikube/pkg/util	1.025s	coverage: 77.9% of statements
ok  	k8s.io/minikube/pkg/util/lock	0.005s	coverage: 18.2% of statements
ok  	k8s.io/minikube/pkg/util/retry	0.002s	coverage: 0.0% of statements
FAIL
Makefile:271: recipe for target 'test' failed
make: *** [test] Error 32
travis_time:end:0007c62e:start=1590101115323692117,finish=1590101293426127416,duration=178102435299,event=script

TravisBuddy Request Identifier: c8e27860-9bc1-11ea-9438-0b74518d636c

minikube-pr-bot · 2020-05-27T17:15:45Z

kvm2 Driver
Times for minikube: [63.117514961 65.201182877 63.572960112000004]
Average time for minikube: 63.96388598333334

Times for Minikube (PR 7973): [63.565084184 64.37706506 61.454181332999994]
Average time for Minikube (PR 7973): 63.132110192333336

Averages Time Per Log

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 7973) |
+--------------------------------+-----------+--------------------+
| * minikube v1.10.1 on Debian   |  0.073902 |           0.062450 |
|                           9.11 |           |                    |
| * Using the kvm2 driver based  |  0.019043 |           0.020092 |
| on existing profile            |           |                    |
| * Starting control plane node  |  0.003189 |           0.004924 |
| minikube in cluster minikube   |           |                    |
| * Creating kvm2 VM (CPUs=2,    | 38.702831 |          39.452330 |
| Memory=3700MB, Disk=20000MB)   |           |                    |
| ...                            |           |                    |
| * Preparing Kubernetes v1.18.2 | 23.282093 |          21.366014 |
| on Docker 19.03.8 ...          |           |                    |
| * Verifying Kubernetes         |  1.515320 |           1.219348 |
| components...                  |           |                    |
| * Enabled addons:              |  0.279359 |           0.923628 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.083012 |           0.079306 |
| configured to use "minikube"   |           |                    |
|                                |  0.005136 |           0.004018 |
+--------------------------------+-----------+--------------------+

docker Driver
Times for minikube: [34.704150771 26.571003006 26.217317919000003]
Average time for minikube: 29.164157231999997

Times for Minikube (PR 7973): [27.939966897 25.753531435 26.397611032999997]
Average time for Minikube (PR 7973): 26.697036455

Averages Time Per Log

+----------------------------------------+-----------+--------------------+
|                  LOG                   | MINIKUBE  | MINIKUBE (PR 7973) |
+----------------------------------------+-----------+--------------------+
| * minikube v1.10.1 on Debian           |  0.077573 |           0.079754 |
|                                   9.11 |           |                    |
| * Using the docker driver              |  0.002308 |           0.002386 |
| based on existing profile              |           |                    |
| * Starting control plane node          |  0.059274 |           0.062086 |
| minikube in cluster minikube           |           |                    |
| * Creating docker container            |  7.742260 |           7.653979 |
| (CPUs=2, Memory=3700MB) ...            |           |                    |
| * Stopping "minikube" in               |  1.146963 |                    |
| docker ...                             |           |                    |
| * Deleting "minikube" in               |  6.348378 |                    |
| docker ...                             |           |                    |
| * Creating docker container            |           |                    |
| (CPUs=2, Memory=3700MB) ...            |           |                    |
| * Preparing Kubernetes v1.18.2         |  0.117664 |           0.119216 |
| on Docker 19.03.2 ...                  |           |                    |
|   -                                    | 17.582281 |          17.778241 |
| kubeadm.pod-network-cidr=10.244.0.0/16 |           |                    |
| * Verifying Kubernetes                 |  0.922111 |           0.908454 |
| components...                          |           |                    |
| * Enabled addons:                      |  0.090840 |           0.023638 |
| default-storageclass,                  |           |                    |
| storage-provisioner                    |           |                    |
| * Done! kubectl is now                 |  0.065324 |           0.062401 |
| configured to use "minikube"           |           |                    |
|                                        |  0.006076 |           0.006880 |
+----------------------------------------+-----------+--------------------+

minikube-pr-bot · 2020-05-27T17:35:35Z

kvm2 Driver
Times for minikube: [63.849858081 63.549235636 64.025932913]
Average time for minikube: 63.80834221

Times for Minikube (PR 7973): [67.83212226299999 64.745072134 60.655826620999996]
Average time for Minikube (PR 7973): 64.41100700599999

Averages Time Per Log

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 7973) |
+--------------------------------+-----------+--------------------+
| * minikube v1.10.1 on Debian   |  0.062422 |           0.061119 |
|                           9.11 |           |                    |
| * Using the kvm2 driver based  |  0.023101 |           0.022543 |
| on existing profile            |           |                    |
| * Starting control plane node  |  0.007984 |           0.003220 |
| minikube in cluster minikube   |           |                    |
| * Creating kvm2 VM (CPUs=2,    | 40.530634 |          40.149195 |
| Memory=3700MB, Disk=20000MB)   |           |                    |
| ...                            |           |                    |
| * Preparing Kubernetes v1.18.2 | 20.982141 |          22.298239 |
| on Docker 19.03.8 ...          |           |                    |
| * Verifying Kubernetes         |  1.298937 |           1.294764 |
| components...                  |           |                    |
| * Enabled addons:              |  0.816772 |           0.506760 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.081994 |           0.070403 |
| configured to use "minikube"   |           |                    |
|                                |  0.004357 |           0.004766 |
+--------------------------------+-----------+--------------------+

docker Driver
Times for minikube: [26.071697603000004 26.906706971000002 27.072375382999997]
Average time for minikube: 26.683593319

Times for Minikube (PR 7973): [27.507146273 29.135927897 26.893359191]
Average time for Minikube (PR 7973): 27.845477786999997

Averages Time Per Log

+----------------------------------------+-----------+--------------------+
|                  LOG                   | MINIKUBE  | MINIKUBE (PR 7973) |
+----------------------------------------+-----------+--------------------+
| * minikube v1.10.1 on Debian           |  0.082395 |           0.078281 |
|                                   9.11 |           |                    |
| * Using the docker driver              |  0.002637 |           0.002501 |
| based on existing profile              |           |                    |
| * Starting control plane node          |  0.060774 |           0.057500 |
| minikube in cluster minikube           |           |                    |
| * Creating docker container            |  7.927636 |           7.581545 |
| (CPUs=2, Memory=3700MB) ...            |           |                    |
| * Preparing Kubernetes v1.18.2         |  0.120100 |           0.119011 |
| on Docker 19.03.2 ...                  |           |                    |
|   -                                    | 17.254051 |          18.720986 |
| kubeadm.pod-network-cidr=10.244.0.0/16 |           |                    |
| * Verifying Kubernetes                 |  1.157611 |           1.163437 |
| components...                          |           |                    |
| * Enabled addons:                      |  0.002675 |           0.055415 |
| default-storageclass,                  |           |                    |
| storage-provisioner                    |           |                    |
| * Done! kubectl is now                 |  0.071529 |           0.063061 |
| configured to use "minikube"           |           |                    |
|                                        |  0.004186 |           0.003740 |
+----------------------------------------+-----------+--------------------+

minikube-pr-bot · 2020-05-27T17:55:05Z

kvm2 Driver
Times for minikube: [61.906767448000004 63.994333752 64.086602524]
Average time for minikube: 63.329234574666664

Times for Minikube (PR 7973): [63.127890451 63.711317521 62.37630870900001]
Average time for Minikube (PR 7973): 63.071838893666666

Averages Time Per Log

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 7973) |
+--------------------------------+-----------+--------------------+
| * minikube v1.10.1 on Debian   |  0.061618 |           0.059156 |
|                           9.11 |           |                    |
| * Using the kvm2 driver based  |  0.020048 |           0.021549 |
| on existing profile            |           |                    |
| * Starting control plane node  |  0.003237 |           0.003431 |
| minikube in cluster minikube   |           |                    |
| * Creating kvm2 VM (CPUs=2,    | 39.147736 |          39.374968 |
| Memory=3700MB, Disk=20000MB)   |           |                    |
| ...                            |           |                    |
| * Preparing Kubernetes v1.18.2 | 21.911065 |          21.408251 |
| on Docker 19.03.8 ...          |           |                    |
| * Verifying Kubernetes         |  1.369347 |           1.329823 |
| components...                  |           |                    |
| * Enabled addons:              |  0.733242 |           0.797667 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.079485 |           0.072491 |
| configured to use "minikube"   |           |                    |
|                                |  0.003455 |           0.004503 |
+--------------------------------+-----------+--------------------+

docker Driver
Times for minikube: [26.986865468 25.400740808 27.643489768000006]
Average time for minikube: 26.67703201466667

Times for Minikube (PR 7973): [26.913629169 25.327199245000003 27.182615238]
Average time for Minikube (PR 7973): 26.474481217333334

Averages Time Per Log

+----------------------------------------+-----------+--------------------+
|                  LOG                   | MINIKUBE  | MINIKUBE (PR 7973) |
+----------------------------------------+-----------+--------------------+
| * minikube v1.10.1 on Debian           |  0.078423 |           0.075995 |
|                                   9.11 |           |                    |
| * Using the docker driver              |  0.002435 |           0.002547 |
| based on existing profile              |           |                    |
| * Starting control plane node          |  0.059347 |           0.058832 |
| minikube in cluster minikube           |           |                    |
| * Creating docker container            |  7.829081 |           7.501385 |
| (CPUs=2, Memory=3700MB) ...            |           |                    |
| * Preparing Kubernetes v1.18.2         |  0.118281 |           0.117384 |
| on Docker 19.03.2 ...                  |           |                    |
|   -                                    | 17.317172 |          17.292918 |
| kubeadm.pod-network-cidr=10.244.0.0/16 |           |                    |
| * Verifying Kubernetes                 |  1.141859 |           1.246626 |
| components...                          |           |                    |
| * Enabled addons:                      |  0.061120 |           0.110469 |
| default-storageclass,                  |           |                    |
| storage-provisioner                    |           |                    |
| * Done! kubectl is now                 |  0.061588 |           0.063503 |
| configured to use "minikube"           |           |                    |
|                                        |  0.007725 |           0.004821 |
+----------------------------------------+-----------+--------------------+

minikube-pr-bot · 2020-05-27T22:56:20Z

kvm2 Driver
Times for minikube: [61.401729407 65.389526386 62.569778144000004]
Average time for minikube: 63.12034464566667

Times for Minikube (PR 7973): [63.55283334000001 65.123136077 61.765616205]
Average time for Minikube (PR 7973): 63.48052854066668

Averages Time Per Log

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 7973) |
+--------------------------------+-----------+--------------------+
| * minikube v1.10.1 on Debian   |  0.060339 |           0.058432 |
|                           9.11 |           |                    |
| * Using the kvm2 driver based  |  0.019036 |           0.019885 |
| on existing profile            |           |                    |
| * Starting control plane node  |  0.003382 |           0.002964 |
| minikube in cluster minikube   |           |                    |
| * Creating kvm2 VM (CPUs=2,    | 40.248226 |          40.182617 |
| Memory=3700MB, Disk=20000MB)   |           |                    |
| ...                            |           |                    |
| * Preparing Kubernetes v1.18.2 | 20.878134 |          21.105053 |
| on Docker 19.03.8 ...          |           |                    |
| * Verifying Kubernetes         |  1.419281 |           1.654241 |
| components...                  |           |                    |
| * Enabled addons:              |  0.412409 |           0.383934 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.076235 |           0.069069 |
| configured to use "minikube"   |           |                    |
|                                |  0.003302 |           0.004334 |
+--------------------------------+-----------+--------------------+

docker Driver
Times for minikube: [25.497116199999997 27.169056992999998 26.291884529999997]
Average time for minikube: 26.319352574333333

Times for Minikube (PR 7973): [27.081522105 27.790656118999998 27.011139165]
Average time for Minikube (PR 7973): 27.29443912966666

Averages Time Per Log

+----------------------------------------+-----------+--------------------+
|                  LOG                   | MINIKUBE  | MINIKUBE (PR 7973) |
+----------------------------------------+-----------+--------------------+
| * minikube v1.10.1 on Debian           |  0.071763 |           0.077955 |
|                                   9.11 |           |                    |
| * Using the docker driver              |  0.002427 |           0.002560 |
| based on existing profile              |           |                    |
| * Starting control plane node          |  0.056860 |           0.057745 |
| minikube in cluster minikube           |           |                    |
| * Creating docker container            |  7.591928 |           7.819146 |
| (CPUs=2, Memory=3700MB) ...            |           |                    |
| * Preparing Kubernetes v1.18.2         |  0.114491 |           0.122178 |
| on Docker 19.03.2 ...                  |           |                    |
|   -                                    | 17.159841 |          17.392559 |
| kubeadm.pod-network-cidr=10.244.0.0/16 |           |                    |
| * Verifying Kubernetes                 |  1.149529 |           1.742168 |
| components...                          |           |                    |
| * Enabled addons:                      |  0.097603 |           0.002879 |
| default-storageclass,                  |           |                    |
| storage-provisioner                    |           |                    |
| * Done! kubectl is now                 |  0.070087 |           0.072475 |
| configured to use "minikube"           |           |                    |
|                                        |  0.004824 |           0.004774 |
+----------------------------------------+-----------+--------------------+

sharifelgamal added 4 commits April 30, 2020 16:02

initial commit for restartWorker

94a9c03

more changes

76349e1

Merge branch 'master' of github.com:kubernetes/minikube into restart

4cb2d0f

only warn on fresh start

38ec707

k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels May 2, 2020

k8s-ci-robot requested review from afbjorklund and josedonizetti May 2, 2020 01:06

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 2, 2020

sharifelgamal added 4 commits May 4, 2020 09:26

lint

daaafd5

re-enable skipped multi node test

258be2c

Merge branch 'master' of github.com:kubernetes/minikube into restart

5589f01

Merge branch 'master' of github.com:kubernetes/minikube into restart

bd2c4d8

sharifelgamal added 8 commits May 4, 2020 15:57

Merge branch 'master' of github.com:kubernetes/minikube into restart

e3b7476

check kubelet status explicitly

2ec2665

let's change a bunch of stuff

4947106

Merge branch 'better-restart' of github.com:tstromberg/minikube into …

65c5d57

…restart

keepContext

358f167

Merge branch 'master' of github.com:kubernetes/minikube into restart

5e98113

save config

ad9b4c1

Merge branch 'master' of github.com:kubernetes/minikube into restart

6bb8371

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 10, 2020

sharifelgamal added 2 commits May 11, 2020 11:50

pass everything by reference since we actively change node IP

4886030

Merge branch 'master' of github.com:kubernetes/minikube into restart

d4639c5

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 11, 2020

it works

335379d

sharifelgamal changed the title ~~WIP: Make sure multinode clusters can survive restarts~~ Make sure multinode clusters can survive restarts May 12, 2020

k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label May 20, 2020

skip start/stop on github actions

95584ea

fix test

023b2dc

add postmortem logs

c412237

sharifelgamal added 2 commits May 21, 2020 14:54

Merge branch 'postmortem' of github.com:sharifelgamal/minikube into r…

bde6624

…estart

Merge branch 'master' of github.com:kubernetes/minikube into restart

c8723ce

Merge branch 'master' of github.com:kubernetes/minikube into restart

e225968

sharifelgamal added 3 commits May 27, 2020 09:58

it works! and simpler

b516783

Merge branch 'master' of github.com:kubernetes/minikube into restart

d075334

revert changes to node_add

101f2da

sharifelgamal added 2 commits May 27, 2020 10:17

let's simplify kubeadm again

a7e483a

skip restart test on github actions

606a307

sharifelgamal added 2 commits May 27, 2020 10:44

swap test order to make it work in github actions

1945b5f

comments

cebba29

Merge branch 'master' of github.com:kubernetes/minikube into restart

94ce379

sharifelgamal merged commit 4f06139 into kubernetes:master May 28, 2020

fmotrifork mentioned this pull request Jun 2, 2020

minikube 1.11.0 fishworks/fish-food#764

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make sure multinode clusters can survive restarts #7973

Make sure multinode clusters can survive restarts #7973

sharifelgamal commented May 2, 2020

k8s-ci-robot commented May 2, 2020

codecov-io commented May 4, 2020 •

edited

Loading

sharifelgamal commented May 20, 2020

minikube-pr-bot commented May 20, 2020

minikube-pr-bot commented May 20, 2020

minikube-pr-bot commented May 21, 2020

medyagh commented May 21, 2020

minikube-pr-bot commented May 21, 2020

minikube-pr-bot commented May 21, 2020

minikube-pr-bot commented May 21, 2020

TravisBuddy commented May 22, 2020

minikube-pr-bot commented May 27, 2020

minikube-pr-bot commented May 27, 2020

minikube-pr-bot commented May 27, 2020

minikube-pr-bot commented May 27, 2020

Make sure multinode clusters can survive restarts #7973

Make sure multinode clusters can survive restarts #7973

Conversation

sharifelgamal commented May 2, 2020

k8s-ci-robot commented May 2, 2020

codecov-io commented May 4, 2020 • edited Loading

Codecov Report

sharifelgamal commented May 20, 2020

minikube-pr-bot commented May 20, 2020

minikube-pr-bot commented May 20, 2020

minikube-pr-bot commented May 21, 2020

medyagh commented May 21, 2020

minikube-pr-bot commented May 21, 2020

minikube-pr-bot commented May 21, 2020

minikube-pr-bot commented May 21, 2020

TravisBuddy commented May 22, 2020

Travis tests have failed

1st Build

TravisBuddy Request Identifier: c8e27860-9bc1-11ea-9438-0b74518d636c

minikube-pr-bot commented May 27, 2020

minikube-pr-bot commented May 27, 2020

minikube-pr-bot commented May 27, 2020

minikube-pr-bot commented May 27, 2020

codecov-io commented May 4, 2020 •

edited

Loading