Merge pull request cloudflare#458 from cloudflare/snooze

Allow snoozing checks
linode-obs · Nov 28, 2022 · ceca305 · ceca305
2 parents 7f75e8a + f771d0f
commit ceca305
Show file tree

Hide file tree

Showing 26 changed files with 705 additions and 5 deletions.
diff --git a/cmd/pint/tests/0111_snooze.txt b/cmd/pint/tests/0111_snooze.txt
@@ -0,0 +1,27 @@
+pint.ok -l debug --no-color lint rules
+! stdout .
+cmp stderr stderr.txt
+
+-- stderr.txt --
+level=info msg="Loading configuration file" path=.pint.hcl
+level=debug msg="File parsed" path=rules/0001.yml rules=1
+level=debug msg="Found recording rule" lines=2-3 path=rules/0001.yml record=sum-job
+level=debug msg="Check snoozed by comment" check=promql/aggregate(job:true) comment="snooze 2099-11-28T10:24:18Z promql/aggregate" snooze=promql/aggregate until=2099-11-28T10:24:18Z
+level=debug msg="Configured checks for rule" enabled=["promql/syntax","alerts/for","alerts/comparison","alerts/template","promql/fragile","promql/regexp"] path=rules/0001.yml rule=sum-job
+-- rules/0001.yml --
+# pint snooze 2099-11-28T10:24:18Z promql/aggregate
+- record: sum-job
+  expr: sum(foo)
+
+-- .pint.hcl --
+parser {
+  relaxed = [".*"]
+}
+rule {
+    match {
+      kind = "recording"
+    }
+    aggregate ".+" {
+        keep = [ "job" ]
+    }
+}
diff --git a/cmd/pint/tests/0112_expired_snooze.txt b/cmd/pint/tests/0112_expired_snooze.txt
@@ -0,0 +1,34 @@
+pint.error -l debug --no-color lint rules
+! stdout .
+cmp stderr stderr.txt
+
+-- stderr.txt --
+level=info msg="Loading configuration file" path=.pint.hcl
+level=debug msg="File parsed" path=rules/0001.yml rules=1
+level=debug msg="Found recording rule" lines=2-3 path=rules/0001.yml record=sum-job
+level=debug msg="Expired snooze" check=promql/aggregate(job:true) comment="snooze 2000-11-28T10:24:18Z promql/aggregate" snooze=promql/aggregate until=2000-11-28T10:24:18Z
+level=debug msg="Expired snooze" check=promql/aggregate(job:true) comment="snooze 2000-11-28T10:24:18Z promql/aggregate" snooze=promql/aggregate until=2000-11-28T10:24:18Z
+level=debug msg="Configured checks for rule" enabled=["promql/syntax","alerts/for","alerts/comparison","alerts/template","promql/fragile","promql/regexp","promql/aggregate(job:true)"] path=rules/0001.yml rule=sum-job
+rules/0001.yml:3: job label is required and should be preserved when aggregating "^.+$" rules, use by(job, ...) (promql/aggregate)
+ 3 |   expr: sum(foo)
+
+level=info msg="Problems found" Bug=1
+level=fatal msg="Fatal error" error="problems found"
+-- rules/0001.yml --
+# pint snooze 2000-11-28T10:24:18Z promql/aggregate
+- record: sum-job
+  expr: sum(foo)
+
+-- .pint.hcl --
+parser {
+  relaxed = [".*"]
+}
+rule {
+    match {
+      kind = "recording"
+    }
+    aggregate ".+" {
+        keep     = [ "job" ]
+        severity = "bug"
+    }
+}
diff --git a/docs/changelog.md b/docs/changelog.md
@@ -7,6 +7,22 @@
 - Use [uber-go/automaxprocs](https://github.com/uber-go/automaxprocs) to
   automatically set GOMAXPROCS to match Linux container CPU quota.
 - Added [labels/conflict](checks/labels/conflict.md) check.
+- If you want to disable invididual checks just for some time then you can now
+  snooze them instead of disabling forever.
+
+  The difference between `# pint disable ...` and `# pint snooze ...` comments is that
+  the snooze comment must include a timestamp. Selected check will be disabled *until*
+  that timestamp.
+  Timestamp must either use [RFC3339](https://www.rfc-editor.org/rfc/rfc3339) syntax
+  or `YYYY-MM-DD` (if you don't care about time and want to snooze until given date).
+  Examples:
+
+  ```yaml
+  # pint snooze 2023-01-12T10:00:00Z promql/series
+  # pint snooze 2023-01-12 promql/rate
+  - record: ...
+    expr: ...
+  ```
 
 ## v0.34.0
 

diff --git a/docs/checks/alerts/annotation.md b/docs/checks/alerts/annotation.md
@@ -143,3 +143,14 @@ annotation "dashboard" {
 Example comment disabling that rule:
 
 `# pint disable alerts/annotation(dashboard:https://grafana\.example\.com/.+:true)`
+
+## How to snooze it
+
+You can disable this check until given time by adding a comment to it. Example:
+
+`# pint snooze $TIMESTAMP alerts/annotation`
+
+Where `$TIMESTAMP` is either use [RFC3339](https://www.rfc-editor.org/rfc/rfc3339)
+formatted  or `YYYY-MM-DD`.
+Adding this comment will disable `alerts/annotation` *until* `$TIMESTAMP`, after that
+check will be re-enabled.
diff --git a/docs/checks/alerts/comparison.md b/docs/checks/alerts/comparison.md
@@ -49,3 +49,14 @@ a comment anywhere in that file. Example:
 Or you can disable it per rule by adding a comment to it. Example:
 
 `# pint disable alerts/comparison`
+
+## How to snooze it
+
+You can disable this check until given time by adding a comment to it. Example:
+
+`# pint snooze $TIMESTAMP alerts/comparison`
+
+Where `$TIMESTAMP` is either use [RFC3339](https://www.rfc-editor.org/rfc/rfc3339)
+formatted  or `YYYY-MM-DD`.
+Adding this comment will disable `alerts/comparison` *until* `$TIMESTAMP`, after that
+check will be re-enabled.
diff --git a/docs/checks/alerts/count.md b/docs/checks/alerts/count.md
@@ -84,3 +84,14 @@ Where `$prometheus` is the name of Prometheus server to disable.
 Example:
 
 `# pint disable alerts/count(prod)`
+
+## How to snooze it
+
+You can disable this check until given time by adding a comment to it. Example:
+
+`# pint snooze $TIMESTAMP alerts/count`
+
+Where `$TIMESTAMP` is either use [RFC3339](https://www.rfc-editor.org/rfc/rfc3339)
+formatted  or `YYYY-MM-DD`.
+Adding this comment will disable `alerts/count` *until* `$TIMESTAMP`, after that
+check will be re-enabled.
diff --git a/docs/checks/alerts/for.md b/docs/checks/alerts/for.md
@@ -35,3 +35,14 @@ a comment anywhere in that file. Example:
 Or you can disable it per rule by adding a comment to it. Example:
 
 `# pint disable alerts/for`
+
+## How to snooze it
+
+You can disable this check until given time by adding a comment to it. Example:
+
+`# pint snooze $TIMESTAMP alerts/for`
+
+Where `$TIMESTAMP` is either use [RFC3339](https://www.rfc-editor.org/rfc/rfc3339)
+formatted  or `YYYY-MM-DD`.
+Adding this comment will disable `alerts/for` *until* `$TIMESTAMP`, after that
+check will be re-enabled.
diff --git a/docs/checks/alerts/template.md b/docs/checks/alerts/template.md
@@ -52,3 +52,14 @@ a comment anywhere in that file. Example:
 Or you can disable it per rule by adding a comment to it. Example:
 
 `# pint disable alerts/template`
+
+## How to snooze it
+
+You can disable this check until given time by adding a comment to it. Example:
+
+`# pint snooze $TIMESTAMP alerts/template`
+
+Where `$TIMESTAMP` is either use [RFC3339](https://www.rfc-editor.org/rfc/rfc3339)
+formatted  or `YYYY-MM-DD`.
+Adding this comment will disable `alerts/template` *until* `$TIMESTAMP`, after that
+check will be re-enabled.
diff --git a/docs/checks/labels/conflict.md b/docs/checks/labels/conflict.md
@@ -75,3 +75,14 @@ Where `$prometheus` is the name of Prometheus server to disable.
 Example:
 
 `# pint disable labels/conflict(prod)`
+
+## How to snooze it
+
+You can disable this check until given time by adding a comment to it. Example:
+
+`# pint snooze $TIMESTAMP labels/conflict`
+
+Where `$TIMESTAMP` is either use [RFC3339](https://www.rfc-editor.org/rfc/rfc3339)
+formatted  or `YYYY-MM-DD`.
+Adding this comment will disable `labels/conflict` *until* `$TIMESTAMP`, after that
+check will be re-enabled.
diff --git a/docs/checks/promql/aggregate.md b/docs/checks/promql/aggregate.md
@@ -113,3 +113,14 @@ Example:
 Example:
 
 `# pint disable promql/aggregate(instance:true)`
+
+## How to snooze it
+
+You can disable this check until given time by adding a comment to it. Example:
+
+`# pint snooze $TIMESTAMP promql/aggregate`
+
+Where `$TIMESTAMP` is either use [RFC3339](https://www.rfc-editor.org/rfc/rfc3339)
+formatted  or `YYYY-MM-DD`.
+Adding this comment will disable `promql/aggregate` *until* `$TIMESTAMP`, after that
+check will be re-enabled.
diff --git a/docs/checks/promql/fragile.md b/docs/checks/promql/fragile.md
@@ -121,3 +121,14 @@ a comment anywhere in that file. Example:
 Or you can disable it per rule by adding a comment to it. Example:
 
 `# pint disable promql/fragile`
+
+## How to snooze it
+
+You can disable this check until given time by adding a comment to it. Example:
+
+`# pint snooze $TIMESTAMP promql/fragile`
+
+Where `$TIMESTAMP` is either use [RFC3339](https://www.rfc-editor.org/rfc/rfc3339)
+formatted  or `YYYY-MM-DD`.
+Adding this comment will disable `promql/fragile` *until* `$TIMESTAMP`, after that
+check will be re-enabled.
diff --git a/docs/checks/promql/range_query.md b/docs/checks/promql/range_query.md
@@ -91,4 +91,15 @@ Where `$prometheus` is the name of Prometheus server to disable.
 
 Example:
 
-`# pint disable promql/range_query(prod)`
+`# pint disable promql/range_query(prod)`
+
+## How to snooze it
+
+You can disable this check until given time by adding a comment to it. Example:
+
+`# pint snooze $TIMESTAMP promql/range_query`
+
+Where `$TIMESTAMP` is either use [RFC3339](https://www.rfc-editor.org/rfc/rfc3339)
+formatted  or `YYYY-MM-DD`.
+Adding this comment will disable `promql/range_query` *until* `$TIMESTAMP`, after that
+check will be re-enabled.
diff --git a/docs/checks/promql/rate.md b/docs/checks/promql/rate.md
@@ -101,4 +101,15 @@ Where `$prometheus` is the name of Prometheus server to disable.
 
 Example:
 
-`# pint disable promql/rate(prod)`
+`# pint disable promql/rate(prod)`
+
+## How to snooze it
+
+You can disable this check until given time by adding a comment to it. Example:
+
+`# pint snooze $TIMESTAMP promql/rate`
+
+Where `$TIMESTAMP` is either use [RFC3339](https://www.rfc-editor.org/rfc/rfc3339)
+formatted  or `YYYY-MM-DD`.
+Adding this comment will disable `promql/rate` *until* `$TIMESTAMP`, after that
+check will be re-enabled.
diff --git a/docs/checks/promql/regexp.md b/docs/checks/promql/regexp.md
@@ -61,3 +61,14 @@ a comment anywhere in that file. Example:
 Or you can disable it per rule by adding a comment to it. Example:
 
 `# pint disable promql/regexp`
+
+## How to snooze it
+
+You can disable this check until given time by adding a comment to it. Example:
+
+`# pint snooze $TIMESTAMP promql/regexp`
+
+Where `$TIMESTAMP` is either use [RFC3339](https://www.rfc-editor.org/rfc/rfc3339)
+formatted  or `YYYY-MM-DD`.
+Adding this comment will disable `promql/regexp` *until* `$TIMESTAMP`, after that
+check will be re-enabled.
diff --git a/docs/checks/promql/series.md b/docs/checks/promql/series.md
@@ -306,3 +306,14 @@ Example:
   # pint disable promql/series({job="dev"})
   expr: my_metric_name{job="dev", instance="a"} / other_metric_name{job="dev", instance="b"}
 ```
+
+## How to snooze it
+
+You can disable this check until given time by adding a comment to it. Example:
+
+`# pint snooze $TIMESTAMP promql/series`
+
+Where `$TIMESTAMP` is either use [RFC3339](https://www.rfc-editor.org/rfc/rfc3339)
+formatted  or `YYYY-MM-DD`.
+Adding this comment will disable `promql/series` *until* `$TIMESTAMP`, after that
+check will be re-enabled.
diff --git a/docs/checks/promql/syntax.md b/docs/checks/promql/syntax.md
@@ -35,3 +35,14 @@ a comment anywhere in that file. Example:
 Or you can disable it per rule by adding a comment to it. Example:
 
 `# pint disable promql/syntax`
+
+## How to snooze it
+
+You can disable this check until given time by adding a comment to it. Example:
+
+`# pint snooze $TIMESTAMP promql/syntax`
+
+Where `$TIMESTAMP` is either use [RFC3339](https://www.rfc-editor.org/rfc/rfc3339)
+formatted  or `YYYY-MM-DD`.
+Adding this comment will disable `promql/syntax` *until* `$TIMESTAMP`, after that
+check will be re-enabled.
diff --git a/docs/checks/promql/vector_matching.md b/docs/checks/promql/vector_matching.md
@@ -105,3 +105,14 @@ Where `$prometheus` is the name of Prometheus server to disable.
 Example:
 
 `# pint disable promql/vector_matching(prod)`
+
+## How to snooze it
+
+You can disable this check until given time by adding a comment to it. Example:
+
+`# pint snooze $TIMESTAMP promql/vector_matching`
+
+Where `$TIMESTAMP` is either use [RFC3339](https://www.rfc-editor.org/rfc/rfc3339)
+formatted  or `YYYY-MM-DD`.
+Adding this comment will disable `promql/vector_matching` *until* `$TIMESTAMP`, after that
+check will be re-enabled.
diff --git a/docs/checks/query/cost.md b/docs/checks/query/cost.md
@@ -109,3 +109,14 @@ Where `$prometheus` is the name of Prometheus server to disable.
 Example:
 
 `# pint disable query/cost(dev)`
+
+## How to snooze it
+
+You can disable this check until given time by adding a comment to it. Example:
+
+`# pint snooze $TIMESTAMP query/cost`
+
+Where `$TIMESTAMP` is either use [RFC3339](https://www.rfc-editor.org/rfc/rfc3339)
+formatted  or `YYYY-MM-DD`.
+Adding this comment will disable `query/cost` *until* `$TIMESTAMP`, after that
+check will be re-enabled.
diff --git a/docs/checks/rule/duplicate.md b/docs/checks/rule/duplicate.md
@@ -79,3 +79,14 @@ Where `$prometheus` is the name of Prometheus server to disable.
 Example:
 
 `# pint disable rule/duplicate(prod)`
+
+## How to snooze it
+
+You can disable this check until given time by adding a comment to it. Example:
+
+`# pint snooze $TIMESTAMP rule/duplicate`
+
+Where `$TIMESTAMP` is either use [RFC3339](https://www.rfc-editor.org/rfc/rfc3339)
+formatted  or `YYYY-MM-DD`.
+Adding this comment will disable `rule/duplicate` *until* `$TIMESTAMP`, after that
+check will be re-enabled.
diff --git a/docs/checks/rule/label.md b/docs/checks/rule/label.md
@@ -121,3 +121,14 @@ label "severity" {
 Example comment disabling that rule:
 
 `# pint disable rule/label(severity:true)`
+
+## How to snooze it
+
+You can disable this check until given time by adding a comment to it. Example:
+
+`# pint snooze $TIMESTAMP rule/label`
+
+Where `$TIMESTAMP` is either use [RFC3339](https://www.rfc-editor.org/rfc/rfc3339)
+formatted  or `YYYY-MM-DD`.
+Adding this comment will disable `rule/label` *until* `$TIMESTAMP`, after that
+check will be re-enabled.
diff --git a/docs/checks/rule/link.md b/docs/checks/rule/link.md
@@ -115,3 +115,14 @@ you can add a more specific comment.
 Example:
 
 `# pint disable rule/link(^https?://.+$)`
+
+## How to snooze it
+
+You can disable this check until given time by adding a comment to it. Example:
+
+`# pint snooze $TIMESTAMP rule/link`
+
+Where `$TIMESTAMP` is either use [RFC3339](https://www.rfc-editor.org/rfc/rfc3339)
+formatted  or `YYYY-MM-DD`.
+Adding this comment will disable `rule/link` *until* `$TIMESTAMP`, after that
+check will be re-enabled.
diff --git a/docs/checks/rule/reject.md b/docs/checks/rule/reject.md
@@ -123,3 +123,14 @@ Example:
 Example:
 
 `# pint disable rule/reject(val=~'^https?://.+$')`
+
+## How to snooze it
+
+You can disable this check until given time by adding a comment to it. Example:
+
+`# pint snooze $TIMESTAMP rule/reject`
+
+Where `$TIMESTAMP` is either use [RFC3339](https://www.rfc-editor.org/rfc/rfc3339)
+formatted  or `YYYY-MM-DD`.
+Adding this comment will disable `rule/reject` *until* `$TIMESTAMP`, after that
+check will be re-enabled.
diff --git a/docs/ignoring.md b/docs/ignoring.md
@@ -105,3 +105,22 @@ A single comment can only disable one check, so repeat it for every check you wi
 to disable.
 
 See each individual [check](checks/index.md) documentation for details.
+
+## Snoozing individual checks for specific rules
+
+If you want to disable invididual checks just for some time then you can snooze them
+instead of disabling forever.
+
+The difference between `# pint disable ...` and `# pint snooze ...` comments is that
+the snooze comment must include a timestamp. Selected check will be disabled *until*
+that timestamp.
+Timestamp must either use [RFC3339](https://www.rfc-editor.org/rfc/rfc3339) syntax
+or `YYYY-MM-DD` (if you don't care about time and want to snooze until given date).
+Examples:
+
+```yaml
+# pint snooze 2023-01-12T10:00:00Z promql/series
+# pint snooze 2023-01-12 promql/rate
+- record: ...
+  expr: ...
+```