Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cache and load maintenance windows only when they are needed #184324

Closed
3 tasks
mikecote opened this issue May 27, 2024 · 1 comment · Fixed by #192573
Closed
3 tasks

Cache and load maintenance windows only when they are needed #184324

mikecote opened this issue May 27, 2024 · 1 comment · Fixed by #192573
Assignees
@ymao1
Labels
Feature:Alerting Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams)

Comments

@mikecote
Copy link
Contributor

mikecote commented May 27, 2024 •
edited
Loading

Currently we load maintenance windows before every rule run. Since maintenance windows are used to silence notifications for the affected alerts, we should only load them when an alerting rule has alerts and include a cache mechanism so we don't have to load them as frequently.

Definition of Done

  • Maintenance windows are only loaded when an alerting rule finds alerts
  • Maintenance windows are cached so it's only fetched every so often when needed
  • tests
@mikecote mikecote added Feature:Alerting Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) labels May 27, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/response-ops (Team:ResponseOps)

@mikecote mikecote changed the title Delay loading maintenance windows during rule execution until they are needed Delay loading maintenance windows during rule execution until they are needed and cache them Jun 5, 2024
@mikecote mikecote changed the title Delay loading maintenance windows during rule execution until they are needed and cache them Cache and load maintenance windows only when they are needed Jun 5, 2024
@mikecote mikecote mentioned this issue Jul 3, 2024
Closed
@mikecote mikecote mentioned this issue Jul 12, 2024
Open
48 tasks
@ymao1 ymao1 mentioned this issue Sep 13, 2024
Merged
@ymao1 ymao1 closed this as completed in 93414a6 Sep 26, 2024
ymao1 added a commit to ymao1/kibana that referenced this issue Sep 26, 2024
@ymao1
[Response Ops][Alerting] Only load maintenance windows when there are…
260a307 
… alerts during rule execution and caching loaded maintenance windows (elastic#192573)

Resolves elastic#184324

## Summary

This PR moves the loading of maintenance windows further down in rule
execution so maintenance windows are only loaded when a rule execution
generates alerts. Also caches maintenance windows per space to reduce
the number of requests.

## To Verify

1. Add some logging to
x-pack/plugins/alerting/server/task_runner/maintenance_windows/maintenance_windows_service.ts
to indicate when windows are being fetched and when they're returning
from the cache.
2. Create and run some rules in different spaces with and without alerts
to see that the maintenance windows are only loaded when there are
alerts and that the windows are returned from the cache when the cache
has not expired.

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit 93414a6)
ymao1 referenced this issue Sep 26, 2024
@ymao1
[8.x] [Response Ops][Alerting] Only load maintenance windows when the…
3358772 
…re are alerts during rule execution and caching loaded maintenance windows (#192573) (#194191)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Response Ops][Alerting] Only load maintenance windows when there are
alerts during rule execution and caching loaded maintenance windows
(#192573)](#192573)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Ying
Mao","email":"ying.mao@elastic.co"},"sourceCommit":{"committedDate":"2024-09-26T12:59:36Z","message":"[Response
Ops][Alerting] Only load maintenance windows when there are alerts
during rule execution and caching loaded maintenance windows
(#192573)\n\nResolves
https://github.com/elastic/kibana/issues/184324\r\n\r\n##
Summary\r\n\r\nThis PR moves the loading of maintenance windows further
down in rule\r\nexecution so maintenance windows are only loaded when a
rule execution\r\ngenerates alerts. Also caches maintenance windows per
space to reduce\r\nthe number of requests.\r\n\r\n## To Verify\r\n\r\n1.
Add some logging
to\r\nx-pack/plugins/alerting/server/task_runner/maintenance_windows/maintenance_windows_service.ts\r\nto
indicate when windows are being fetched and when they're
returning\r\nfrom the cache.\r\n2. Create and run some rules in
different spaces with and without alerts\r\nto see that the maintenance
windows are only loaded when there are\r\nalerts and that the windows
are returned from the cache when the cache\r\nhas not
expired.\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine
<elasticmachine@users.noreply.github.com>\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"93414a672c2767b035110fa2d811cc040af57727","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Feature:Alerting","release_note:skip","Team:ResponseOps","v9.0.0","backport:prev-minor","ci:project-deploy-observability","Team:obs-ux-management","v8.16.0"],"number":192573,"url":"https://github.com/elastic/kibana/pull/192573","mergeCommit":{"message":"[Response
Ops][Alerting] Only load maintenance windows when there are alerts
during rule execution and caching loaded maintenance windows
(#192573)\n\nResolves
https://github.com/elastic/kibana/issues/184324\r\n\r\n##
Summary\r\n\r\nThis PR moves the loading of maintenance windows further
down in rule\r\nexecution so maintenance windows are only loaded when a
rule execution\r\ngenerates alerts. Also caches maintenance windows per
space to reduce\r\nthe number of requests.\r\n\r\n## To Verify\r\n\r\n1.
Add some logging
to\r\nx-pack/plugins/alerting/server/task_runner/maintenance_windows/maintenance_windows_service.ts\r\nto
indicate when windows are being fetched and when they're
returning\r\nfrom the cache.\r\n2. Create and run some rules in
different spaces with and without alerts\r\nto see that the maintenance
windows are only loaded when there are\r\nalerts and that the windows
are returned from the cache when the cache\r\nhas not
expired.\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine
<elasticmachine@users.noreply.github.com>\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"93414a672c2767b035110fa2d811cc040af57727"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/192573","number":192573,"mergeCommit":{"message":"[Response
Ops][Alerting] Only load maintenance windows when there are alerts
during rule execution and caching loaded maintenance windows
(#192573)\n\nResolves
https://github.com/elastic/kibana/issues/184324\r\n\r\n##
Summary\r\n\r\nThis PR moves the loading of maintenance windows further
down in rule\r\nexecution so maintenance windows are only loaded when a
rule execution\r\ngenerates alerts. Also caches maintenance windows per
space to reduce\r\nthe number of requests.\r\n\r\n## To Verify\r\n\r\n1.
Add some logging
to\r\nx-pack/plugins/alerting/server/task_runner/maintenance_windows/maintenance_windows_service.ts\r\nto
indicate when windows are being fetched and when they're
returning\r\nfrom the cache.\r\n2. Create and run some rules in
different spaces with and without alerts\r\nto see that the maintenance
windows are only loaded when there are\r\nalerts and that the windows
are returned from the cache when the cache\r\nhas not
expired.\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine
<elasticmachine@users.noreply.github.com>\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"93414a672c2767b035110fa2d811cc040af57727"}},{"branch":"8.x","label":"v8.16.0","labelRegex":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ymao1 ymao1

Labels
Feature:Alerting Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams)
Projects
None yet
Milestone
No milestone
3 participants
@mikecote @ymao1 @elasticmachine