metricbeat: provide means to exclude non-user file systems for fsstat.

[slalomnut]

Unable to exclude file systems (e.g. (cgroup|tmpfs|rootfs|devpts|pstore|configfs|debugfs|mqueue|hugetlbfs|nfsd|sunrpc|binfmt_misc) ) using filters or processors.
This makes it difficult, if not impossible, to track actual used and free disk space. A feature that would be very nice for infrastructure reporting.

[ruflin]

@slalomnut Can you share some more details on why it was not possible to exclude these?

[slalomnut]

I've tried both processors and filters but not excluding the fact my regex in the metricbeat.yml is incorrect.

 filters:
    - drop_event.when.regexp.mount_point: '^/(sys|cgroup|proc|dev|etc|host|run)($|/)'
    - drop_event.when.regexp.device_name: '^(tmpfs|sunrpc|rootfs)($|/)'
processors:
 - drop_event:
      when:
        regexp:
          system.fsstat:
            equals: '^(cgroup|tmpfs|rootfs|devpts|pstore|configfs|debugfs|mqueue|hugetlbfs|nfsd|sunrpc|/etc/auto.home|binfmt_misc)'

The above returns a value of 34 for fsstat.count. If I test the processor and run df -a |egrep -v '^(cgroup|tmpfs|rootfs|devpts|pstore|configfs|debugfs|mqueue|hugetlbfs|nfsd|sunrpc|/etc/auto.home|binfmt_misc)' |wc -l on the same system, I get 13.

[slalomnut]

See here: https://discuss.elastic.co/t/system-fsstat-filtering-processors/69029/3

[ruflin]

Ok, got it. This a feature request to do some filtering on which file systems should be taken into account for the calculations in the fsstat metricset. We could introduce something like:

fsstat.exclude: ["regexp1", "regexp2]

This would solve your problem, right?

For the moment I would recommend you to use filesystem metricset, only send the file systems you need and do the calculations on the elasticsearch side.

[andrewkroh]

I think this has been implemented with #6819.

There are a set of filesystem types that are filtered by default (all types marked as nodev in /proc/filesystems in Linux systems) and you can specify your own.

https://www.elastic.co/guide/en/beats/metricbeat/6.4/metricbeat-metricset-system-filesystem.html#_configuration_5