Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deploy_minikube.sh: Adding support for Centos8 #6073

Merged
merged 1 commit into from
Jul 6, 2020
Merged

deploy_minikube.sh: Adding support for Centos8 #6073

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 43 additions & 10 deletions .travis/deploy_minikube.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,34 +14,67 @@ set -x
# NOTE: This script was originally copied from the Cojaeger-operator build
# https://github.com/jaegertracing/jaeger-operator/blob/master/.travis/setupMinikube.sh

# socat is needed for port forwarding
sudo apt-get update && sudo apt-get install socat && sudo apt-get install conntrack

export MINIKUBE_VERSION=v1.8.2
export KUBERNETES_VERSION=v1.17.3

sudo mount --make-rshared /
sudo mount --make-rshared /proc
sudo mount --make-rshared /sys
source /etc/os-release

if [ "${ID}" == "ubuntu" ]
then
# socat is needed for port forwarding
sudo apt-get update && sudo apt-get install socat && sudo apt-get install conntrack
sudo mount --make-rshared /
sudo mount --make-rshared /proc
sudo mount --make-rshared /sys
elif [ "${ID}" == "centos" ]
then
dnf install -y sudo
sudo dnf -y update && sudo dnf -y install socat conntrack
dnf install -y dnf-plugins-core
sudo dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo dnf install -y docker-ce docker-ce-cli containerd.io --nobest
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why docker-ce and not podman and/or buildah?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

originally it was on deploying Ubuntu (as it for travis runs).
We can change to podman when running on Centos

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the look of it, minikube support for podman is still WIP.
We can see 2 things:

  1. Using the podman driver is experimental
  2. even though the driver is podman, the run-time is docker and changing it to cri-o still looks for docker (Fedora 31 vm-driver=podman fail to start trying to start docker service kubernetes/minikube#6795).

We are using MINIKUBE_VERSION=v1.8.2 and KUBERNETES_VERSION=v1.17.3 due to kubernetes/minikube#7828

running with podman results in:

+ main@./1.sh:70 cat /root/.minikube/config/config.json
{
    "WantNoneDriverWarning": false,
    "WantUpdateNotification": false,
    "container-runtime": "cri-o",
    "driver": "podman",
    "vm-driver": "none"
}+ main@./1.sh:72 minikube version
minikube version: v1.8.2
commit: eb13446e786c9ef70cb0a9f85a633194e62396a1
+ main@./1.sh:74 minikube start --kubernetes-version=v1.17.3
😄  minikube v1.8.2 on Centos 8.1.1911
    ▪ MINIKUBE_VERSION=v1.8.2
✨  Using the podman (experimental) driver based on user configuration
E0705 15:30:53.797867   10377 cache.go:106] Error downloading kic artifacts:  error loading image: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

once podman support in minikube will be better, and we will be able to upgrade the minikube version we can switch to podman on Centos8.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have opened a new issue to track this: #6075

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have seen similar problems with minikube+podman, it does not really work well yet.

#disable SELinux
SELinux_status=$(sestatus | grep "SELinux status" | awk -F ":" '{print $2}' | xargs )
if [ "${SELinux_status}" == "enabled" ]
then
sudo setenforce 0
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oy :(

Why?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, kubeadm on CentOS needs SELinux to be disabled:

kubernetes/minikube#6014 (comment)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does that happen with podman as well?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I mean, somehow, magically it is working in RHEL, no?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should look into it one we can switch into podman (see previous comment #6073 (comment))

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some locations where minikube stores executables, and these try to access configuration files. SElinux prevents that. I am not sure if the location that minikube uses is dynamic, or predictable. In the 2nd case, creating the directories in advance and setting appropriate labels might work.

This obviously need some more research. Moving to Permissive mode is acceptable for the moment.

fi
else
echo "${ID} is not supported for ${0}, Exiting."
exit 1
fi

curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/${KUBERNETES_VERSION}/bin/linux/amd64/kubectl && \
chmod +x kubectl && \
sudo mv kubectl /usr/local/bin/
#moving kubectl to /usr/bin/ and not /usr/local/bin/ because on Centos it fails to sudo from /usr/local/bin/
# it fails to sudo from /usr/local/bin/ even if it is the ${PATH}
sudo mv kubectl /usr/bin/

curl -Lo minikube https://storage.googleapis.com/minikube/releases/${MINIKUBE_VERSION}/minikube-linux-amd64 && \
chmod +x minikube && \
sudo mv minikube /usr/local/bin/minikube
#moving minikube to /usr/bin/ and not /usr/local/bin/ because on Centos it fails to sudo from /usr/local/bin/
# it fails to sudo from /usr/local/bin/ even if it is the ${PATH}
sudo mv minikube /usr/bin/minikube

mkdir "${HOME}"/.kube || true
touch "${HOME}"/.kube/config

# minikube config
minikube config set WantUpdateNotification false
minikube config set WantNoneDriverWarning false
minikube config set vm-driver none

cat ~/.minikube/config/config.json

minikube version
sudo minikube start --kubernetes-version=$KUBERNETES_VERSION #--insecure-registry="${LOCAL_IP}:5000" #TODO Remove insecure
sudo chown -R travis: /home/travis/.minikube/
#sudo minikube start --kubernetes-version=$KUBERNETES_VERSION #--insecure-registry="${LOCAL_IP}:5000" #TODO Remove insecure
sudo minikube start --kubernetes-version=$KUBERNETES_VERSION

current_user=$(whoami)
if [ "${current_user}" != "root" ]
then
sudo chown -R ${current_user}: /home/${current_user}/.minikube/
fi

minikube update-context

Expand Down