Things that I find useful, and that help in my day-to-day tasks -- a 'toolbelt'. I'm mostly writing these things here because I set them up, then a week later they need maintenance, but I completely forgot how I wired everything together. Making it public in case it's useful to anyone who comes across it.
Started getting interested in Grafana and Prometheus, so decided to use Project 1 from this list to learn use it to visualize metrics on my local machine. When I got to the bit about adding the Node Exporter Metrics Dashboard to Grafana, I actually typed the ID to this much cooler dashboard instead. Instead of getting the more basic view from Project 1, I got this much more detailed view instead:
I thought to myself this is worth keeping, so I moved the runnable files from my /training directory to my /projects/custom-apps one. This included both the Node Exporter and the running version of Prometheus that I got from following this guide.
Then I created two systemd service units, one to start node exporter, and the other to start prometheus itself. I already had grafana running as a service (installed from apt), so now all of them can be monitored with systemctl.
If this stuff ever breaks, check out the service units in /etc/systemd/system called node-exporter-for-prometheus.service and prometheus-for-node-exporter.service. If everything is ok there, check out what's happening inside ~/projects/custom-apps/prometheus-for-node-exporter.
Grafana runs on localhost:3000 by default, Node Exporter runs on localhost:9100 and Prometheus on localhost:9090.