Fix Two Broken Grafana Dashboards

[kenafoster]

Context

Since upgrading Grafana in Nebari 2024.6.1, two of the dashboards that were built using Angular no longer work and need to be ported to a new front-end framework .
Per discussion thread, Angular deprecation not the cause

Nebari team supports the dashboards included by default in the "Main" folder. Testing confirms that these two are not functional:

• JupyterHub Dashboard
• Usage Report

Value and/or benefit

These dashboards expose a lot of valuable information to users and admins that is difficult to manually retrieve

Anything else?

No response

[Adam-D-Lewis]

Thanks for reporting. I've confirmed that the "Main/JupyterHub Dashboard" isn't working.
You can still see how much memory your pod is using in "Supplementary/Kubernetes / Compute Resources / Pod" dashboard despite seeing the "This panel requires Angular (deprecated) warning."

From what I'm seeing, the issue is unrelated to the Angular deprecation warning. It's just a warning for now, but will become deprecated in Grafana 11 and 12 (https://grafana.com/docs/grafana/latest/developers/angular_deprecation/#when-will-angular-plugins-stop-working). Nebari uses Grafana 10 at the moment. Many of the dashboards with the warning come from https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack and it seems to have been addressed for those dashboards (prometheus-community/helm-charts#4531)

It'd be helpful to understand which dashboards aren't working as you expect @kenafoster. I suspect the Jupyterhub dashboards broke in the recent Jupyterhub version upgrade. When I added the jupyterhub dashboards originally, they came from https://github.com/jupyterhub/grafana-dashboards. I did need to make some adjustments since we were on an older version of jupyterhub at the time. Re-adding the unmodified or lightly modified dashboards from that repo would likely fix the issue.

Is this something you'd be interested in working on @kenafoster?

[kenafoster]

Hi @Adam-D-Lewis -

Which of the dashboards that Nebari ships with should be considered in scope? I have so far only been working with those in the Main folder since those appear customized while the rest (which are either named "Loki / *" or in a "Supplementary" folder) I have ignored since I'm guessing those are pre-installed and/or we don't maintain them. But I'm new to this whole area of Nebari so let me know.

Regarding the "Main" folder dashboards... I've found two issues. JupyterHub you've already diagnosed. The other that doesn't appear to be working is "Usage Report".

For "Usage Report" it looks like a fixable bug. I did just a tiny bit of digging and it looks like $hub variable in queries like the snippet below isn't set correctly. When I manually replace $hub with the namespace for my Nebari deployment, the dashboard works.

kube_pod_labels{
  label_app="jupyterhub",
  label_component="singleuser-server",
  namespace=~"$hub",
  job="kube-state-metrics"
}

Yes I can take on both of these fixes. Let me know if any of the other dashboards should be considered in scope and I'll cut tickets.

[Adam-D-Lewis]

Thanks @kenafoster 🙂

Yeah, the ones in the Main folder should be the only ones you need to look at. The others are maintained elsewhere.

[kenafoster]

@dcmcand this should still be open. #2671 fixed the Usage Report dashboard but not the JupyterHub Dashboard

I'll cut a PR to fix the latter soon