File system error:Metrics not available at the moment

[xiaozhao123254]

[Nokel81]

Is the kubernetes cluster using containerD or some other runtime?

[austinnichols101]

I have the same message on a GKE autopilot cluster (containerD runtime)

[ryanthedev]

I have the same issue running on AKS/containerD.

[cbluth]

i have a similar symptom, i cannot view metrics in lens with an account on the cluster using a rolebinding in a specific namespace, using eks and deploying the lens-metric namespace via the buttons in lens (from a different account):

[Nokel81]

@cbluth If you open up your devtools, do you see any logs about permsissions?

[marcbachmann]

Regarding the missing container_fs_usage_bytes metric, that's coming soon in cadvisor.
A PR got merged: google/cadvisor#2936

[cbluth]

@Nokel81 @xiaozhao123254

i have a working template here:

# role
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: lens-metrics
  name: read-only-lens-metrics
rules:
- apiGroups:
  - ""
  - "apps"
  resources:
  - pods
  - events
  - services
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - "services/proxy"
  verbs:
  - create

# role binding
--- 
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  annotations:
  namespace: lens-metrics
  name: vendor-read-metrics
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: read-only-lens-metrics
subjects:
- kind: Group
  name: ${SOMEGROUP1}
  apiGroup: rbac.authorization.k8s.io
- kind: Group
  name: ${SOMEGROUP2}
  apiGroup: rbac.authorization.k8s.io

[lisenet]

I've this very problem on a baremetal K8s cluster provisioned by Kubeadm. Prometheus is up and running without issues.