Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MetalLB and Traefik not getting installed in spec.k0s.config section #635

Open
wicadmin opened this issue Jan 15, 2024 · 5 comments
Open

MetalLB and Traefik not getting installed in spec.k0s.config section #635

wicadmin opened this issue Jan 15, 2024 · 5 comments

Comments

@wicadmin
Copy link

wicadmin commented Jan 15, 2024 •
edited
Loading

Hi, the extensions to install MetalLB and Traefik is being ignored. The cluster is getting built but these items are not getting configured.

apiVersion: k0sctl.k0sproject.io/v1beta1
kind: Cluster
metadata:
  name: k0s-cluster
spec:
  hosts:
  - ssh:
      address: 192.168.1.42
      user: kube
      port: 22
      keyPath: ~/.ssh/id_rsa
    role: controller
  - ssh:
      address: 192.168.1.47
      user: kube
      port: 22
      keyPath: ~/.ssh/id_rsa
    role: worker
  - ssh:
      address: 192.168.1.48
      user: kube
      port: 22
      keyPath: ~/.ssh/id_rsa
    role: worker
  - ssh:
      address: 192.168.1.49
      user: kube
      port: 22
      keyPath: ~/.ssh/id_rsa
    role: worker
  k0s:
    version: null
    versionChannel: stable
    dynamicConfig: false
    config:
      apiVersion: k0s.k0sproject.io/v1beta1
      kind: ClusterConfig
      metadata:
        name: k0s
      spec:
        telemetry:
          enabled: true
        extensions:
          helm:
            repositories:
            - name: traefik
              url: https://helm.traefik.io/traefik
            - name: bitnami
              url: https://charts.bitnami.com/bitnami
            charts:
            - name: traefik
              chartname: traefik/traefik
              version: "9.11.0"
              namespace: default
            - name: metallb
              chartname: bitnami/metallb
              version: "1.0.1"
              namespace: default
              values: |2
                configInline:
                  address-pools:
                  - name: generic-cluster-pool
                    protocol: layer2
                    addresses:
                    - 192.168.1.235-192.168.1.235


⣿⣿⡇⠀⠀⢀⣴⣾⣿⠟⠁⢸⣿⣿⣿⣿⣿⣿⣿⡿⠛⠁⠀⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀█████████ █████████ ███
⠀⣿⣿⡇⣠⣶⣿⡿⠋⠀⠀⠀⢸⣿⡇⠀⠀⠀⣠⠀⠀⢀⣠⡆⢸⣿⣿⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀███          ███    ███
⠀⣿⣿⣿⣿⣟⠋⠀⠀⠀⠀⠀⢸⣿⡇⠀⢰⣾⣿⠀⠀⣿⣿⡇⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀███          ███    ███
⠀⣿⣿⡏⠻⣿⣷⣤⡀⠀⠀⠀⠸⠛⠁⠀⠸⠋⠁⠀⠀⣿⣿⡇⠈⠉⠉⠉⠉⠉⠉⠉⠉⢹⣿⣿⠀███          ███    ███
⠀⣿⣿⡇⠀⠀⠙⢿⣿⣦⣀⠀⠀⠀⣠⣶⣶⣶⣶⣶⣶⣿⣿⡇⢰⣶⣶⣶⣶⣶⣶⣶⣶⣾⣿⣿⠀█████████    ███    ██████████
k0sctl v0.17.4 Copyright 2023, k0sctl authors.
Anonymized telemetry of usage will be sent to the authors.
By continuing to use k0sctl you agree to these terms:
https://k0sproject.io/licenses/eula
INFO ==> Running phase: Set k0s version  
INFO Looking up latest stable k0s version         
INFO Using k0s version v1.28.5+k0s.0              
INFO ==> Running phase: Connect to hosts 
INFO [ssh] 192.168.1.49:22: connected           
INFO [ssh] 192.168.1.42:22: connected           
INFO [ssh] 192.168.1.48:22: connected           
INFO [ssh] 192.168.1.47:22: connected           
INFO ==> Running phase: Detect host operating systems 
INFO [ssh] 192.168.1.49:22: is running Ubuntu 22.04.3 LTS 
INFO [ssh] 192.168.1.48:22: is running Ubuntu 22.04.3 LTS 
INFO [ssh] 192.168.1.42:22: is running Ubuntu 22.04.3 LTS 
INFO [ssh] 192.168.1.47:22: is running Ubuntu 22.04.3 LTS 
INFO ==> Running phase: Acquire exclusive host lock 
INFO ==> Running phase: Prepare hosts    
INFO ==> Running phase: Gather host facts 
INFO [ssh] 192.168.1.47:22: using k0s-worker1 as hostname 
INFO [ssh] 192.168.1.48:22: using k0s-worker2 as hostname 
INFO [ssh] 192.168.1.49:22: using k0s-worker3 as hostname 
INFO [ssh] 192.168.1.42:22: using k0s-control as hostname 
INFO [ssh] 192.168.1.47:22: discovered ens33 as private interface 
INFO [ssh] 192.168.1.48:22: discovered ens33 as private interface 
INFO [ssh] 192.168.1.42:22: discovered ens33 as private interface 
INFO [ssh] 192.168.1.49:22: discovered ens33 as private interface 
INFO ==> Running phase: Validate hosts   
INFO ==> Running phase: Gather k0s facts 
INFO ==> Running phase: Validate facts   
INFO [ssh] 192.168.1.42:22: validating configuration 
INFO ==> Running phase: Configure k0s    
INFO [ssh] 192.168.1.42:22: installing new configuration 
INFO ==> Running phase: Initialize the k0s cluster 
INFO [ssh] 192.168.1.42:22: installing k0s controller 
INFO [ssh] 192.168.1.42:22: waiting for the k0s service to start 
INFO [ssh] 192.168.1.42:22: waiting for kubernetes api to respond 
INFO ==> Running phase: Install workers  
INFO [ssh] 192.168.1.47:22: validating api connection to https://192.168.1.42:6443 
INFO [ssh] 192.168.1.48:22: validating api connection to https://192.168.1.42:6443 
INFO [ssh] 192.168.1.49:22: validating api connection to https://192.168.1.42:6443 
INFO [ssh] 192.168.1.42:22: generating a join token for worker 1 
INFO [ssh] 192.168.1.42:22: generating a join token for worker 2 
INFO [ssh] 192.168.1.42:22: generating a join token for worker 3 
INFO [ssh] 192.168.1.47:22: writing join token  
INFO [ssh] 192.168.1.48:22: writing join token  
INFO [ssh] 192.168.1.49:22: writing join token  
INFO [ssh] 192.168.1.48:22: installing k0s worker 
INFO [ssh] 192.168.1.47:22: installing k0s worker 
INFO [ssh] 192.168.1.49:22: installing k0s worker 
INFO [ssh] 192.168.1.48:22: starting service    
INFO [ssh] 192.168.1.47:22: starting service    
INFO [ssh] 192.168.1.49:22: starting service    
INFO [ssh] 192.168.1.47:22: waiting for node to become ready 
INFO [ssh] 192.168.1.48:22: waiting for node to become ready 
INFO [ssh] 192.168.1.49:22: waiting for node to become ready 
INFO ==> Running phase: Release exclusive host lock 
INFO ==> Running phase: Disconnect from hosts 
INFO ==> Finished in 1m45s

└─[$]> kubectl get all -A
NAMESPACE     NAME                                  READY   STATUS    RESTARTS   AGE
kube-system   pod/coredns-85df575cdb-jb77s          0/1     Running   0          73s
kube-system   pod/coredns-85df575cdb-w8pqx          0/1     Running   0          83s
kube-system   pod/konnectivity-agent-76hgn          1/1     Running   0          76s
kube-system   pod/konnectivity-agent-ljf5x          1/1     Running   0          81s
kube-system   pod/konnectivity-agent-vdn6c          1/1     Running   0          81s
kube-system   pod/kube-proxy-4v8k8                  1/1     Running   0          81s
kube-system   pod/kube-proxy-cnj8n                  1/1     Running   0          81s
kube-system   pod/kube-proxy-knwd8                  1/1     Running   0          76s
kube-system   pod/kube-router-9wlld                 1/1     Running   0          76s
kube-system   pod/kube-router-qhks9                 1/1     Running   0          81s
kube-system   pod/kube-router-spsj8                 1/1     Running   0          81s
kube-system   pod/metrics-server-7556957bb7-6nfvk   0/1     Running   0          83s

NAMESPACE     NAME                     TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                  AGE
default       service/kubernetes       ClusterIP   10.96.0.1        <none>        443/TCP                  108s
kube-system   service/kube-dns         ClusterIP   10.96.0.10       <none>        53/UDP,53/TCP,9153/TCP   98s
kube-system   service/metrics-server   ClusterIP   10.101.157.132   <none>        443/TCP                  94s

NAMESPACE     NAME                                DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
kube-system   daemonset.apps/konnectivity-agent   3         3         3       3            3           kubernetes.io/os=linux   97s
kube-system   daemonset.apps/kube-proxy           3         3         3       3            3           kubernetes.io/os=linux   98s
kube-system   daemonset.apps/kube-router          3         3         3       3            3           <none>                   98s

NAMESPACE     NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
kube-system   deployment.apps/coredns          0/2     2            0           98s
kube-system   deployment.apps/metrics-server   0/1     1            0           94s

NAMESPACE     NAME                                        DESIRED   CURRENT   READY   AGE
kube-system   replicaset.apps/coredns-85df575cdb          2         2         0       84s
kube-system   replicaset.apps/metrics-server-7556957bb7   1         1         0       84s
@twz123
Copy link
Member

twz123 commented Jan 15, 2024

As far as I can tell, the chart configs are broken. Can you verify?

$ kubectl -n kube-system get charts k0s-addon-chart-traefik k0s-addon-chart-metallb -o custom-columns=NAME:.metadata.name,ERROR:.status.error
NAME                      ERROR
k0s-addon-chart-traefik   can't install loadedChart `traefik`: failed to install CRD crds/ingressroute.yaml: resource mapping not found for name: "ingressroutes.traefik.containo.us" namespace: "" from "": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
ensure CRDs are installed first
k0s-addon-chart-metallb   can't locate chart `bitnami/metallb-1.0.1`: chart "metallb" matching 1.0.1 not found in bitnami index. (try 'helm repo update'): no chart version found for metallb-1.0.1

There are some forum threads describing how to install Traefik/MetalLB:

@till
Copy link

till commented Jan 15, 2024

Bitnami frequently retires charts from their main repo. There's another file that contains all. I'll look for it.

@wicadmin
Copy link
Author

As far as I can tell, the chart configs are broken. Can you verify?

$ kubectl -n kube-system get charts k0s-addon-chart-traefik k0s-addon-chart-metallb -o custom-columns=NAME:.metadata.name,ERROR:.status.error
NAME                      ERROR
k0s-addon-chart-traefik   can't install loadedChart `traefik`: failed to install CRD crds/ingressroute.yaml: resource mapping not found for name: "ingressroutes.traefik.containo.us" namespace: "" from "": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
ensure CRDs are installed first
k0s-addon-chart-metallb   can't locate chart `bitnami/metallb-1.0.1`: chart "metallb" matching 1.0.1 not found in bitnami index. (try 'helm repo update'): no chart version found for metallb-1.0.1

There are some forum threads describing how to install Traefik/MetalLB:

I was looking to get this done via k0sctl. For years I've written how to setup MetalLB and nginx ingress with kubeadm build. I was hoping k0s and k0sctl streamlined this but now it seems need to go back and do these as separate tasks vs. part of cluster build.

@wicadmin
Copy link
Author

Seems using the calico provider solves the issues.

@till
Copy link

till commented Jan 17, 2024 •
edited
Loading

For completeness, we use the following helm "repo":

helm repo add bitnami https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami

That contains all the "retired" charts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@till @twz123 @wicadmin