Duplicate Prometheus alerts

[baurmatt]

Due to deployment simplicity, we run this exporter in multiple clusters within the same projects. This leads to duplicate metrics and thus wrong/duplicate alerts.

All alerts are currently created with the following schema:

sum((syseleven_xxx_used / syseleven_xxx_total) * 100) by(region, project) > 90

I suggest to change this to:

sum((max(syseleven_xxx_used)by(__name__, project, region) / max(syseleven_xxx_total)by(__name__, project, region)) * 100) by(project, region) > 90

This will give us only the largest metric of a project and region, no matter how often the metric got collected.

[cstruck]

lgtm