Skip to content

Latest commit

 

History

History
195 lines (159 loc) · 6.71 KB

README.md

File metadata and controls

195 lines (159 loc) · 6.71 KB
Raw

Disk Usage Prometheus Exporter

Build Status codecov Go Report Card Maintainability CodeScene Code Health

Provides detailed info about disk usage of the selected filesystem path.

Uses gdu under the hood for the disk usage analysis.

Demo Grafana dashboard

https://grafana.milde.cz/d/0TfJhs_Mz/disk-usage (credentials: grafana / gdu)

Usage

Usage:
  disk_usage_exporter [flags]

Flags:
  -p, --analyzed-path string              Path where to analyze disk usage (default "/")
      --basic-auth-users stringToString   Basic Auth users and their passwords as bcypt hashes (default [])
  -b, --bind-address string               Address to bind to (default "0.0.0.0:9995")
  -c, --config string                     config file (default is $HOME/.disk_usage_exporter.yaml)
  -l, --dir-level int                     Directory nesting level to show (0 = only selected dir) (default 2)
  -L, --follow-symlinks                   Follow symlinks for files, i.e. show the size of the file to which symlink points to (symlinks to directories are not followed)
  -h, --help                              help for disk_usage_exporter
  -i, --ignore-dirs strings               Absolute paths to ignore (separated by comma) (default [/proc,/dev,/sys,/run,/var/cache/rsnapshot])
  -m, --mode string                       Expose method - either 'file' or 'http' (default "http")
      --multi-paths stringToString        Multiple paths where to analyze disk usage, in format /path1=level1,/path2=level2,... (default [])
  -f, --output-file string                Target file to store metrics in (default "./disk-usage-exporter.prom")

Either one path can be specified using --analyzed-path and --dir-level flags or multiple can be set using --multi-paths flag:

disk_usage_exporter --multi-paths=/home=2,/var=3

Example output

# HELP node_disk_usage_bytes Disk usage of the directory/file
# TYPE node_disk_usage_bytes gauge
node_disk_usage_bytes{path="/var/cache"} 2.1766144e+09
node_disk_usage_bytes{path="/var/db"} 20480
node_disk_usage_bytes{path="/var/dpkg"} 8192
node_disk_usage_bytes{path="/var/empty"} 4096
node_disk_usage_bytes{path="/var/games"} 4096
node_disk_usage_bytes{path="/var/lib"} 7.554709504e+09
node_disk_usage_bytes{path="/var/local"} 4096
node_disk_usage_bytes{path="/var/lock"} 0
node_disk_usage_bytes{path="/var/log"} 4.247068672e+09
node_disk_usage_bytes{path="/var/mail"} 0
node_disk_usage_bytes{path="/var/named"} 4096
node_disk_usage_bytes{path="/var/opt"} 4096
node_disk_usage_bytes{path="/var/run"} 0
node_disk_usage_bytes{path="/var/snap"} 1.11694848e+10
node_disk_usage_bytes{path="/var/spool"} 16384
node_disk_usage_bytes{path="/var/tmp"} 475136
# HELP node_disk_usage_level_1_bytes Disk usage of the directory/file level 1
# TYPE node_disk_usage_level_1_bytes gauge
node_disk_usage_level_1_bytes{path="/bin"} 0
node_disk_usage_level_1_bytes{path="/boot"} 1.29736704e+08
node_disk_usage_level_1_bytes{path="/etc"} 1.3090816e+07
node_disk_usage_level_1_bytes{path="/home"} 8.7081373696e+10
node_disk_usage_level_1_bytes{path="/lib"} 0
node_disk_usage_level_1_bytes{path="/lib64"} 0
node_disk_usage_level_1_bytes{path="/lost+found"} 4096
node_disk_usage_level_1_bytes{path="/mnt"} 4096
node_disk_usage_level_1_bytes{path="/opt"} 2.979229696e+09
node_disk_usage_level_1_bytes{path="/root"} 4096
node_disk_usage_level_1_bytes{path="/sbin"} 0
node_disk_usage_level_1_bytes{path="/snap"} 0
node_disk_usage_level_1_bytes{path="/srv"} 4.988928e+06
node_disk_usage_level_1_bytes{path="/tmp"} 1.3713408e+07
node_disk_usage_level_1_bytes{path="/usr"} 1.8109427712e+10
node_disk_usage_level_1_bytes{path="/var"} 2.5156793856e+10

Example Prometheus queries

Disk usage of /var directory:

sum(node_disk_usage_bytes{path=~"/var.*"})

Example config files

~/.disk_usage_exporter.yaml:

analyzed-path: /
bind-address: 0.0.0.0:9995
dir-level: 2
ignore-dirs:
- /proc
- /dev
- /sys
- /run

~/.disk_usage_exporter.yaml:

analyzed-path: /
mode: file
output-file: ./disk-usage-exporter.prom
dir-level: 2
ignore-dirs:
- /proc
- /dev
- /sys
- /run

~/.disk_usage_exporter.yaml:

multi-paths:
  /home: 2
  /var: 3
  /tmp: 1
bind-address: 0.0.0.0:9995
dir-level: 2
ignore-dirs:
- /proc
- /dev
- /sys
- /run
basic-auth-users:
  prom: $2b$12$MzUQjmLxPRM9WW6OI4ZzwuZHB7ubHiiSnngJxIufgZms27nw.5ZAq

Prometheus scrape config

Disk usage analysis can be resource heavy. Set the scrape_interval and scrape_timeout according to the size of analyzed path.

scrape_configs:
  - job_name: 'disk-usage'
    scrape_interval: 5m
    scrape_timeout: 20s
    static_configs:
    - targets: ['localhost:9995']

Dump to file

The official node-exporter allows to specify a folder which contains additional metric files through a textfile collection mechanism. In order to make use of this, one has to set up node-exporter according to the documentation and set the output-file of this exporter to any name ending in .prom within said folder (and of course also mode to file).

A common use case for this is when the calculation of metrics takes particularly long and therefore can only be done once in a while. To automate the periodic update of the output file, simply set up a cronjob.

Basic Auth

You can enable HTTP Basic authorization by setting the --basic-auth-users flag (password is "test"):

disk_usage_exporter --basic-auth-users='admin=$2b$12$hNf2lSsxfm0.i4a.1kVpSOVyBCfIB51VRjgBUyv6kdnyTlgWj81Ay'

or by setting the key in config:

basic-auth-users:
  admin: $2b$12$hNf2lSsxfm0.i4a.1kVpSOVyBCfIB51VRjgBUyv6kdnyTlgWj81Ay

The password needs to be hashed by bcrypt in both cases.

Example systemd unit file

[Unit]
Description=Prometheus disk usage exporter
Documentation=https://github.com/dundee/disk_usage_exporter

[Service]
Restart=always
User=prometheus
ExecStart=/usr/bin/disk_usage_exporter $ARGS
ExecReload=/bin/kill -HUP $MAINPID
TimeoutStopSec=20s
SendSIGKILL=no

[Install]
WantedBy=multi-user.target