[Alerting][Event log] Persisting duration information for active alerts in event log #101387

ymao1 · 2021-06-04T13:31:04Z

Resolves #93704

Summary

This PR calculates and persists duration information for alert (eg instance) events in the event log. When a new alert is triggered, a start time is recorded in the new-instance event, with an initial duration of 0. A uuid is also generated. While this alert remains active, each active-instance event log document will be written with the start time, the uuid and an updated duration in nanoseconds. When the alert recovers, the end time will be recorded in the recovered-instance document in the event log. The uuid will remain the same for the active span of an alert but will change if the alert recovers and then becomes active again.

Checklist

Delete any items that are not applicable to this PR.

Unit or functional tests were updated or added to match the most common scenarios

…-log/alert-duration

elasticmachine · 2021-06-07T17:01:29Z

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

pmuellr

LGTM, posted a few questions though.

I think we should probably open a new issue, given we now have this capability, that we can make the alert details page a little better by showing the REAL alert duration (for alerts that started after the 7.14 migration), rather than trying to calculate it based on previous records. I suspect that means the API we use in the event log will change as well, but we'd need to see.

x-pack/plugins/alerting/server/task_runner/task_runner.ts

pmuellr · 2021-06-08T17:03:43Z

x-pack/plugins/alerting/server/task_runner/task_runner.ts

+ // Inject start time into instance state of new instances
+ for (const id of newAlertIds) {
+ const state = currentAlerts[id].getState();
+ currentAlerts[id].replaceState({ ...state, start: currentTime, uuid: uuidv4() });


side thought, for @YulNaumenko - does this new UUID value for the alert id belong in one of the new rule. fields we're writing?

Yeah, makes sense to me. Need to check with @elastic/security-detections-response

I don't think it makes sense to write alert information into the rule object?

Good philosophical question. An alert is rule-related, so it seems fine to me to. As an example, the process. fields have a thread.id - kinda seems like the same kind of relationship. We might be able to get some historic context from the RFC that introduced this.

I guess the RAC field for this value is kibana.rac.alert.uuid right now? Per previous comment ^^^, I'm hesitant to add kibana.rac fields right now, especially if it's something that should be in rule. (or some other existing field). Maybe we need to get a new ECS RFC together to add such a field to rule.? Let's bring this up with the RAC alerts-as-data folk ...

x-pack/test/alerting_api_integration/spaces_only/tests/alerting/event_log.ts

…-log/alert-duration

ymao1 · 2021-06-08T19:29:02Z

I think we should probably open a new issue, given we now have this capability, that we can make the alert details page a little better by showing the REAL alert duration (for alerts that started after the 7.14 migration), rather than trying to calculate it based on previous records. I suspect that means the API we use in the event log will change as well, but we'd need to see.

Created this issue: #101662

YulNaumenko

LGTM

ymao1 · 2021-06-09T11:34:30Z

@elasticmachine merge upstream

…ibana into event-log/alert-duration

kibanamachine · 2021-06-09T15:08:03Z

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

💚 Build #130080 succeeded 947fea5
💚 Build #129926 succeeded 793dccd
💚 Build #129588 succeeded 10826c6
💔 Build #129558 failed 712ddf4
💚 Build #129264 succeeded b794c0a

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @ymao1

…ts in event log (elastic#101387) * WIP * Storing start, duration and end in alert state * Writing to event log * Updating unit tests * Adding unit tests * Fixing uuid in tests * Updating functional test * Adding functional test * Removing console logs * Fixing unit tests * PR fixes * Removing uuid from alert information Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

kibanamachine · 2021-06-09T15:11:54Z

💚 Backport successful

Status	Branch	Result
✅	7.x

This backport PR will be merged automatically after passing CI.

…ts in event log (#101387) (#101784) * WIP * Storing start, duration and end in alert state * Writing to event log * Updating unit tests * Adding unit tests * Fixing uuid in tests * Updating functional test * Adding functional test * Removing console logs * Fixing unit tests * PR fixes * Removing uuid from alert information Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: ymao1 <ying.mao@elastic.co>

* master: clarify which parts of TM are experimental (elastic#101757) Add sh scripts with _bulk_action route usage examples (elastic#101736) [Uptime] Only register route in side nav if uptime show capability is true (elastic#101709) Use KIBANA_DOCS in doc link service (elastic#101667) [Alerting][Event log] Persisting duration information for active alerts in event log (elastic#101387) Address design issues in Discover/Graph (elastic#101584) Optimize performance for document table (elastic#101715) Change file data visualizer links to point to new location in home application (elastic#101393) [Fleet] Tighten policy permissions, take II (elastic#97366) [ML] Add debounce to the severity control update (elastic#101581) [Fleet] Fix routing issues with `getPath` and `history.push` (elastic#101658) [APM] Add link-to/transaction route (elastic#101731) [Index Patterns] Runtime fields CRUD REST API (elastic#101164) [ILM] Refactor types and fix missing aria labels (elastic#101518) [Lens] New summary row feature for datatable (elastic#101075) Blocks save event filter with a white space name (elastic#101599) Improve security server types (elastic#101661) [APM] Replace side nav with tabs on Settings page (elastic#101460) [APM] Only register items in side nav if user has permissions to see app (elastic#101707) [Security solution][Endpoint] Add back button when to the event filters list (elastic#101280)

ymao1 added 9 commits June 2, 2021 11:27

WIP

7d77e26

Storing start, duration and end in alert state

af3b12d

Writing to event log

38a9d82

Updating unit tests

9434b3a

Adding unit tests

8411d8f

Fixing uuid in tests

d29519e

Updating functional test

c426afa

Adding functional test

6299489

Merge branch 'master' of https://github.com/elastic/kibana into event…

b794c0a

…-log/alert-duration

ymao1 changed the title ~~Event log/alert duration~~ [Alerting][Event log] Persisting duration information for active alerts in event log Jun 7, 2021

ymao1 added 2 commits June 7, 2021 12:59

Removing console logs

b2cc047

Merge branch 'master' of https://github.com/elastic/kibana into event…

712ddf4

…-log/alert-duration

ymao1 self-assigned this Jun 7, 2021

ymao1 added Feature:Alerting Feature:EventLog release_note:skip Skip the PR/issue when compiling release notes Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) v7.14.0 v8.0.0 labels Jun 7, 2021

ymao1 marked this pull request as ready for review June 7, 2021 17:01

ymao1 requested a review from a team as a code owner June 7, 2021 17:01

Fixing unit tests

10826c6

pmuellr approved these changes Jun 8, 2021

View reviewed changes

ymao1 added 2 commits June 8, 2021 14:59

PR fixes

1b6a4c0

Merge branch 'master' of https://github.com/elastic/kibana into event…

793dccd

…-log/alert-duration

ymao1 mentioned this pull request Jun 8, 2021

[Alerting UI] Use recorded start/duration/end times from event log for rule details page #101662

Open

YulNaumenko approved these changes Jun 8, 2021

View reviewed changes

ymao1 added the auto-backport Deprecated - use backport:version if exact versions are needed label Jun 9, 2021

kibanamachine and others added 3 commits June 9, 2021 07:34

Merge branch 'master' into event-log/alert-duration

947fea5

Removing uuid from alert information

dbe7598

Merge branch 'event-log/alert-duration' of https://github.com/ymao1/k…

6ae7ed3

…ibana into event-log/alert-duration

ymao1 mentioned this pull request Jun 9, 2021

[Alerting][Event Log] Consider adding uuid to active alert spans #101749

Closed

ymao1 merged commit a12ff5d into elastic:master Jun 9, 2021

kibanamachine mentioned this pull request Jun 9, 2021

[7.x] [Alerting][Event log] Persisting duration information for active alerts in event log (#101387) #101784

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Alerting][Event log] Persisting duration information for active alerts in event log #101387

[Alerting][Event log] Persisting duration information for active alerts in event log #101387

ymao1 commented Jun 4, 2021 •

edited

Loading

elasticmachine commented Jun 7, 2021

pmuellr left a comment

pmuellr Jun 8, 2021

YulNaumenko Jun 8, 2021

ymao1 Jun 8, 2021

pmuellr Jun 9, 2021

ymao1 commented Jun 8, 2021

YulNaumenko left a comment

ymao1 commented Jun 9, 2021

kibanamachine commented Jun 9, 2021

kibanamachine commented Jun 9, 2021

[Alerting][Event log] Persisting duration information for active alerts in event log #101387

[Alerting][Event log] Persisting duration information for active alerts in event log #101387

Conversation

ymao1 commented Jun 4, 2021 • edited Loading

Summary

Checklist

elasticmachine commented Jun 7, 2021

pmuellr left a comment

Choose a reason for hiding this comment

pmuellr Jun 8, 2021

Choose a reason for hiding this comment

YulNaumenko Jun 8, 2021

Choose a reason for hiding this comment

ymao1 Jun 8, 2021

Choose a reason for hiding this comment

pmuellr Jun 9, 2021

Choose a reason for hiding this comment

ymao1 commented Jun 8, 2021

YulNaumenko left a comment

Choose a reason for hiding this comment

ymao1 commented Jun 9, 2021

kibanamachine commented Jun 9, 2021

💚 Build Succeeded

Metrics [docs]

History

kibanamachine commented Jun 9, 2021

💚 Backport successful

ymao1 commented Jun 4, 2021 •

edited

Loading