[Release-1.29] - Expose K3s metrics via supervisor port

[brandond]

Backport fix for Expose K3s metrics via supervisor port

• Expose K3s metrics via supervisor port #9951

[aganesh-suse]

Validated on release-1.29 branch with commit c1df575

Environment Details

Infrastructure

• Cloud
• Hosted

Node(s) CPU architecture, OS, and Version:

$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.2 LTS"

$ uname -m
x86_64

Cluster Configuration:

HA: 3 server/ 1 agent

Config.yaml:

Server Config:

token: xxxx
cluster-init: true
write-kubeconfig-mode: "0644"
node-external-ip: 1.1.1.1
node-label:
- k3s-upgrade=server
debug: true

supervisor-metrics: true
enable-pprof: true

Agent Config:

token: xxxx
server: https://1.1.1.1:6443
node-external-ip: 1.1.1.2
node-label:
- k3s-upgrade=agent
debug: true

enable-pprof: true

Testing Steps

1. Copy config.yaml

$ sudo mkdir -p /etc/rancher/k3s && sudo cp config.yaml /etc/rancher/k3s

2. Install k3s

curl -sfL https://get.k3s.io | sudo INSTALL_K3S_COMMIT='c1df57555d146696678bfad439d585d6f842f002' sh -s - server

3. Verify Cluster Status:

kubectl get nodes -o wide
kubectl get pods -A

4. Query /metrics and /debug/pprof

Validation Results:

• k3s version used for validation:

 $ k3s -v 
k3s version v1.29.6+k3s-c1df5755 (c1df5755)
go version go1.21.11

The following queries work on both server and agent nodes. Not providing output due to size.

 $ sudo curl -vks --cert /var/lib/rancher/k3s/server/tls/client-admin.crt --key /var/lib/rancher/k3s/server/tls/client-admin.key https://<server|agent ip>:6443/metrics 
  $ sudo curl -vks --cert /var/lib/rancher/k3s/server/tls/client-admin.crt --key /var/lib/rancher/k3s/server/tls/client-admin.key https://<server|agent ip>:6443/debug/pprof