Skip to content

minio vm‐broker run operator tenant on k8s

Jump to bottom
Allan Roger Reid edited this page Dec 4, 2024 · 6 revisions

This walkthru shows how to setup a minio operator using k8s

Note: First, create an instance using steps in https://github.com/allanrogerr/public/wiki/minio-vm%E2%80%90broker-backup-instance

From the Instance > Summary page, obtain the ssh command, access the instance, set linger and reboot

e.g.

ssh -p 30086 ubuntu@1.2.3.4
loginctl enable-linger ubuntu
sudo reboot -h now

Install kubectl

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" && \
curl -LO "https://dl.k8s.io/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl.sha256" && \
echo "$(cat kubectl.sha256)  kubectl" | sha256sum --check && \
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl && \
kubectl version --client && \
kubectl version --client --output=yaml

Install kind

[ $(uname -m) = x86_64 ] && curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.20.0/kind-linux-amd64 && \
chmod +x ./kind && \
sudo mv ./kind /usr/local/bin/kind

Install podman

sudo apt-get update && \
sudo apt-get -y install podman

Ubuntu version 20.04:

echo "deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_20.04/ /" | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list
curl -L "https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_20.04/Release.key" | sudo apt-key add -
sudo apt update
sudo apt install podman

Symlink podman to replace docker commands

sudo ln -s /usr/bin/podman /usr/bin/docker

Create basic kind cluster template

vi kind-config.yaml

# four node (two workers) cluster config
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
  - role: control-plane
  - role: worker
  - role: worker
  - role: worker
  - role: worker

Delete and recreate kind cluster

kind delete cluster

kind create cluster --config kind-config.yaml

Apply operator

kubectl apply -k github.com/minio/operator/

You may need to set ephemeral storage to 0Mi if the minio-operator pods stay in Pending

image 
kubectl -n minio-operator edit deployments/minio-operator

spec:
  template:
    spec:
      containers:
      - args:
        resources:
          requests:
            ephemeral-storage: 0Mi

Access minio operator on k8s

Create and echo secret for jwt

kubectl apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: console-sa-secret
  namespace: minio-operator
  annotations:
    kubernetes.io/service-account.name: console-sa
type: kubernetes.io/service-account-token
EOF


SA_TOKEN=$(kubectl -n minio-operator  get secret console-sa-secret -o jsonpath="{.data.token}" | base64 --decode)
echo $SA_TOKEN

Obtain the ip4 address using ip a

Then port forward ip a | grep inet

    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
    inet 10.212.97.114/24 metric 100 brd 10.212.97.255 scope global dynamic eth0
    inet6 fd42:4487:a176:5c8f:216:3eff:fe77:892c/64 scope global mngtmpaddr 
    inet6 fe80::216:3eff:fe77:892c/64 scope link 

kubectl port-forward svc/console -n minio-operator 9091:9090 --address 10.212.97.114

Forwarding from 10.212.97.114:9091 -> 9090
Handling connection for 9091

Use jwt to access operator (see above echo $SA_TOKEN)

https://net-8657.lab.min.dev:9091/login

Spin up a tenant

To prevent this sort of error do the following steps: Error: Drive http://test-pool-0-3.test-hl.tenant-lite.svc.cluster.local:9000/export is part of root drive, will not be used (*errors.errorString) 1.- Add env variable

kubectl -n tenant-lite edit tenant/test

Add:

spec:
  env:
    - name: MINIO_CI_CD
      value: "on"
image
Clone this wiki locally