Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Response Ops][Alerting] Should we send summary notifications when there are no alerts to report #155708

Closed
ymao1 opened this issue Apr 25, 2023 · 4 comments · Fixed by #156421
Closed

[Response Ops][Alerting] Should we send summary notifications when there are no alerts to report #155708

ymao1 opened this issue Apr 25, 2023 · 4 comments · Fixed by #156421
Assignees
@XavierM @ersin-erdal
Labels
Feature:Alerting Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams)

Comments

@ymao1
Copy link
Contributor

ymao1 commented Apr 25, 2023

The alerting framework started supporting summary actions in 8.8 and detection rules have migrated to using framework alert summaries with this PR. We noticed a discrepancy in behavior between the two summary features when there are no alerts to report. The framework summary will trigger a notification in this case (where the action message would indicate there are 0 alerts to report) while the detection rule summary would skip notifications.

Should we be matching the previous detection rule summary behavior?
@botelastic botelastic bot added the needs-team Issues missing a team label label Apr 25, 2023
@ymao1 ymao1 mentioned this issue Apr 25, 2023
Merged
1 task
@ymao1 ymao1 added Feature:Alerting Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) labels Apr 25, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/response-ops (Team:ResponseOps)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Apr 25, 2023
@ersin-erdal
Copy link
Contributor

IINM we discussed this when we develop the summary feature and decided to send notification for summary alerts on custom interval.
User may want to see that there were no alerts or only recovered alerts that were carried over from the previous execution as active.
We can change the behaviour but should we change it only for Sec. Rules or O11Y as well?

@XavierM
Copy link
Contributor

XavierM commented Apr 26, 2023 •
edited
Loading

I think we should change it because summary and conditional action are around notification fatigue and we want to avoid too many notification on our users.
What we can also do is asking our user what they want by asking them if they want notification when there is no data. if user request this behavior.

@mikecote mikecote self-assigned this Apr 27, 2023
@XavierM XavierM assigned XavierM and unassigned mikecote Apr 27, 2023
@XavierM
Copy link
Contributor

XavierM commented May 2, 2023

@shanisagiv1 and I have a discussion around this and we think it will be better to not send any notification if we do NOT find any alerts.

@ersin-erdal ersin-erdal self-assigned this May 2, 2023
@ersin-erdal ersin-erdal mentioned this issue May 2, 2023
Merged
ersin-erdal added a commit that referenced this issue May 3, 2023
@ersin-erdal
Don't trigger summary actions when there are no alerts to report (#15…
506806f 
…6421)

Resolves: #155708

Currently we always trigger summary actions on custom interval even if
there are no alerts to report.
This PR changes this behaviour to skip summary actions when there are no
alerts.

## To verify
Create a Security Rule with a summary action that is on custom interval
(`Summary of alerts` -> `Custom Frequency`)
Add an alerts filter to filter out all the alerts (e.g. by using host
name that doesn't exist)
Expect the summary action not to be triggered.
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue May 3, 2023
@ersin-erdal
Don't trigger summary actions when there are no alerts to report (ela…
7859834 
…stic#156421)

Resolves: elastic#155708

Currently we always trigger summary actions on custom interval even if
there are no alerts to report.
This PR changes this behaviour to skip summary actions when there are no
alerts.

## To verify
Create a Security Rule with a summary action that is on custom interval
(`Summary of alerts` -> `Custom Frequency`)
Add an alerts filter to filter out all the alerts (e.g. by using host
name that doesn't exist)
Expect the summary action not to be triggered.

(cherry picked from commit 506806f)
kibanamachine added a commit that referenced this issue May 3, 2023
@kibanamachine @ersin-erdal
[8.8] Don't trigger summary actions when there are no alerts to report (
c403354 
#156421) (#156617)

# Backport

This will backport the following commits from `main` to `8.8`:
- [Don't trigger summary actions when there are no alerts to report
(#156421)](#156421)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Ersin
Erdal","email":"92688503+ersin-erdal@users.noreply.github.com"},"sourceCommit":{"committedDate":"2023-05-03T19:15:01Z","message":"Don't
trigger summary actions when there are no alerts to report
(#156421)\n\nResolves: #155708\r\n\r\nCurrently we always trigger
summary actions on custom interval even if\r\nthere are no alerts to
report.\r\nThis PR changes this behaviour to skip summary actions when
there are no\r\nalerts.\r\n\r\n## To verify\r\nCreate a Security Rule
with a summary action that is on custom interval\r\n(`Summary of alerts`
-> `Custom Frequency`)\r\nAdd an alerts filter to filter out all the
alerts (e.g. by using host\r\nname that doesn't exist)\r\nExpect the
summary action not to be
triggered.","sha":"506806fee7f374205d98995a2754a0d296fbebb0","branchLabelMapping":{"^v8.9.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","Team:ResponseOps","v8.8.0","v8.9.0"],"number":156421,"url":"#156421
trigger summary actions when there are no alerts to report
(#156421)\n\nResolves: #155708\r\n\r\nCurrently we always trigger
summary actions on custom interval even if\r\nthere are no alerts to
report.\r\nThis PR changes this behaviour to skip summary actions when
there are no\r\nalerts.\r\n\r\n## To verify\r\nCreate a Security Rule
with a summary action that is on custom interval\r\n(`Summary of alerts`
-> `Custom Frequency`)\r\nAdd an alerts filter to filter out all the
alerts (e.g. by using host\r\nname that doesn't exist)\r\nExpect the
summary action not to be
triggered.","sha":"506806fee7f374205d98995a2754a0d296fbebb0"}},"sourceBranch":"main","suggestedTargetBranches":["8.8"],"targetPullRequestStates":[{"branch":"8.8","label":"v8.8.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.9.0","labelRegex":"^v8.9.0$","isSourceBranch":true,"state":"MERGED","url":"#156421
trigger summary actions when there are no alerts to report
(#156421)\n\nResolves: #155708\r\n\r\nCurrently we always trigger
summary actions on custom interval even if\r\nthere are no alerts to
report.\r\nThis PR changes this behaviour to skip summary actions when
there are no\r\nalerts.\r\n\r\n## To verify\r\nCreate a Security Rule
with a summary action that is on custom interval\r\n(`Summary of alerts`
-> `Custom Frequency`)\r\nAdd an alerts filter to filter out all the
alerts (e.g. by using host\r\nname that doesn't exist)\r\nExpect the
summary action not to be
triggered.","sha":"506806fee7f374205d98995a2754a0d296fbebb0"}}]}]
BACKPORT-->

Co-authored-by: Ersin Erdal <92688503+ersin-erdal@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@XavierM XavierM

@ersin-erdal ersin-erdal

Labels
Feature:Alerting Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams)
Projects
No open projects
AppEx: ResponseOps - Execution & Connectors
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Don't trigger summary actions when there are no alerts to report ersin-erdal/kibana
5 participants
@XavierM @mikecote @ymao1 @elasticmachine @ersin-erdal