Updating minikube version

[liranmauda]

• Update minikube version to 1.8.2
• Update kubernetes version to 1.17.3
• Installing conntrack
• Added MINIKUBE_DEBUG="--alsologtostderr --v=5" commented out by default
• removed the "--extra-config=apiserver.authorization-mode=RBAC" flag as it no longer needed in kubernetes 1.17 and up

[liranmauda]

Currently failing due to kubernetes/minikube#7058

[guymguym]

@liranmauda Are you sure this is the same issue from minikube? In our case we use minikube config set vm-driver none and in the issue it was virtualbox.

Perhaps with vm-driver none the issue is that minikube doesn't support ubuntu xenial out of the box and requires some prior setup. Here is where we configure the distro:
https://github.com/noobaa/noobaa-operator/blob/master/.travis.yml#L1-L2

I would look for issues with ubuntu xenial and driver none.

[guymguym]

Here are the minikube docs for driver none:
https://minikube.sigs.k8s.io/docs/drivers/none/

[guymguym]

Check out this example:
https://github.com/lilic/travis-minikube/blob/master/.travis.yml

[liranmauda]

in the example above, they use kubectl 1.18.1, let's try that.

[guymguym]

Still failing but I see this info - seems like the only error is that you need to install conntrack

W0506 11:14:28.324671    5132 exit.go:101] Error starting cluster: init failed. output: "

-- stdout --
[init] Using Kubernetes version: v1.18.1
[preflight] Running pre-flight checks

-- /stdout --

** stderr ** 
W0506 11:14:27.469164    5265 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
    [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
    [WARNING FileExisting-ebtables]: ebtables not found in system path
    [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase preflight: [preflight] Some fatal errors occurred:
    [ERROR FileExisting-conntrack]: conntrack not found in system path
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher

** /stderr **

": /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.18.1:$PATH kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification": exit status 1

[guymguym]

@liranmauda add sudo apt-get -y install conntrack like we see here

[liranmauda]

@guymguym already did, it failed.
lets try change the distro to ubuntu bionic ?

[guymguym]

Can you add --v=5 like they suggest?

[guymguym]

Also worth trying minikube version 1.8.1 like in the minikube-travis example

[guymguym]

Check this out - 11 days ago - kubernetes/minikube#7905

[guymguym]

@liranmauda OK ignore previous link, but I think this one might be the issue:
kubernetes/minikube#6061

Here is the solution from it:

Notice this warning:
W1211 16:40:12.204626 4562 manifests.go:214] the default kube-apiserver authorization-mode is "Node,RBAC"; using "RBAC"

If you really want to specify particular modes, please use --extra-config=apiserver.authorization-mode=Node,RBAC with Kubernetes v1.17.x and newer.

[guymguym]

@liranmauda How about trying to remove that extra-config flag altogether? I don't think we need it on new kubernetes (not sure, but worth trying)

[liranmauda]

@liranmauda How about trying to remove that extra-config flag altogether? I don't think we need it on new kubernetes (not sure, but worth trying)

ho, i thought so but first pushed with --extra-config=apiserver.authorization-mode=Node,RBAC

[guymguym]

@liranmauda The tests are still running but minikube start finally worked!

[guymguym]

I think it will work the same if you remove this flag so I would do that.

Also remember to revert the dist back to xenial, set the minikube/kubernetes versions you want to use, and move these flags MINIKUBE_DEBUG="--alsologtostderr --v=5" that will be commented out by default.

[liranmauda]

It is looping on

+ main@.travis/install-minikube.sh:49 kubectl get pods --all-namespaces

NAMESPACE     NAME                                                                      READY   STATUS             RESTARTS   AGE

kube-system   coredns-66bff467f8-s6w9v                                                  0/1     CrashLoopBackOff   6          7m11s

kube-system   coredns-66bff467f8-tzdpn                                                  0/1     CrashLoopBackOff   6          7m11s

kube-system   etcd-travis-job-7f17877f-c6d3-455a-8a2b-3b151eff4bd4                      1/1     Running            0          7m18s

kube-system   kube-apiserver-travis-job-7f17877f-c6d3-455a-8a2b-3b151eff4bd4            1/1     Running            0          7m18s

kube-system   kube-controller-manager-travis-job-7f17877f-c6d3-455a-8a2b-3b151eff4bd4   1/1     Running            0          7m18s

kube-system   kube-proxy-fnm4b                                                          1/1     Running            0          7m11s

kube-system   kube-scheduler-travis-job-7f17877f-c6d3-455a-8a2b-3b151eff4bd4            1/1     Running            0          7m18s

kube-system   storage-provisioner                                                       1/1     Running            0          7m17s

[guymguym]

@liranmauda It waits for kube-addon-manager. I think this check is unneeded (it's a check we copied from somewhere which is obviously outdated):

noobaa-operator/.travis/install-minikube.sh

Lines 48 to 49 in 6553511

	# waiting for kube-addon-manager to be ready
	JSONPATH='{range .items[*]}{@.metadata.name}:{range @.status.conditions[*]}{@.type}={@.status};{end}{end}'; until kubectl -n kube-system get pods -lcomponent=kube-addon-manager -o jsonpath="$JSONPATH" 2>&1 | grep -q "Ready=True"; do sleep 1;echo "waiting for kube-addon-manager to be available"; kubectl get pods --all-namespaces; done

[guymguym]

You can update the checks after minikube start to match these:
https://github.com/lilic/travis-minikube/blob/08db8c0e32217a6f42a4de2ae6b86b1ec638526f/.travis.yml#L28-L36

[guymguym]

The two last steps in that script are irrelevant to us, because they try to deploy a redis pod, so that's not needed. But the first two are relevant:

# Following is just to demo that the kubernetes cluster works.
- kubectl cluster-info
# Wait for kube-dns to be ready.
- JSONPATH='{range .items[*]}{@.metadata.name}:{range @.status.conditions[*]}{@.type}={@.status};{end}{end}'; until kubectl -n kube-system get pods -lk8s-app=kube-dns -o jsonpath="$JSONPATH" 2>&1 | grep -q "Ready=True"; do sleep 1;echo "waiting for kube-dns to be available"; kubectl get pods --all-namespaces; done

[guymguym]

failed on MINIKUBE_DEBUG unbound variable - you need to set it to empty by default, and no need to export it.

[liranmauda]

is this relevant?

# Create example Redis deployment on Kubernetes.
- kubectl run travis-example --image=redis --labels="app=travis-example"

[guymguym]

Not relevant - I tried to write about it in this previous comment

[liranmauda]

Nice the tests started :)

[guymguym]

@liranmauda The failure is in test-cli-flow, is it related to the new version?