Skip to content

Commit

Permalink
Fix alert to support multiple compactor pods running
Browse files Browse the repository at this point in the history
  • Loading branch information
QuentinBisson committed Sep 26, 2024
1 parent dd282cc commit a13601c
Show file tree
Hide file tree
Showing 3 changed files with 6 additions and 6 deletions.
4 changes: 2 additions & 2 deletions production/loki-mixin-compiled-ssd/alerts.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -52,10 +52,10 @@ groups:
# The "last successful run" metric is updated even if the compactor owns no tenants,
# so this alert correctly doesn't fire if compactor has nothing to do.
min (
time() - (loki_boltdb_shipper_compact_tables_operation_last_successful_run_timestamp_seconds{} > 0
time() - (loki_boltdb_shipper_compact_tables_operation_last_successful_run_timestamp_seconds{} > 0)
)
by (cluster, namespace)
> 60 * 60 * 24)
> 60 * 60 * 24
for: 1h
labels:
severity: critical
Expand Down
4 changes: 2 additions & 2 deletions production/loki-mixin-compiled/alerts.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -52,10 +52,10 @@ groups:
# The "last successful run" metric is updated even if the compactor owns no tenants,
# so this alert correctly doesn't fire if compactor has nothing to do.
min (
time() - (loki_boltdb_shipper_compact_tables_operation_last_successful_run_timestamp_seconds{} > 0
time() - (loki_boltdb_shipper_compact_tables_operation_last_successful_run_timestamp_seconds{} > 0)
)
by (cluster, namespace)
> 60 * 60 * 24)
> 60 * 60 * 24
for: 1h
labels:
severity: critical
Expand Down
4 changes: 2 additions & 2 deletions production/loki-mixin/alerts.libsonnet
Original file line number Diff line number Diff line change
Expand Up @@ -77,10 +77,10 @@
# The "last successful run" metric is updated even if the compactor owns no tenants,
# so this alert correctly doesn't fire if compactor has nothing to do.
min (
time() - (loki_boltdb_shipper_compact_tables_operation_last_successful_run_timestamp_seconds{} > 0
time() - (loki_boltdb_shipper_compact_tables_operation_last_successful_run_timestamp_seconds{} > 0)
)
by (%s, namespace)
> 60 * 60 * 24)
> 60 * 60 * 24
||| % $._config.per_cluster_label,
'for': '1h',
labels: {
Expand Down

0 comments on commit a13601c

Please sign in to comment.