[Cases] Case action #168369

cnasikas · 2023-10-09T14:45:51Z

Summary

Depends on: #166267, #170326, #169484, #173740, #173763, #178068, #178307, #178600, #180437

PRs:

Testing

Run Kibana with --run-examples if you want to use the "Always firing" rule.

Create a rule with a case action in observability and the stack. The security solution is not supported. You should not be able to assign a case action in a security solution rule.

Test the "Reopen closed cases" configuration.
Test the "Grouping by" configuration. Only one field is allowed. Not all fields are persisted in alerts. If you select a field not part of the alert the case action will create a case where the grouping value is set to unknow.
Test the "Time window" feature. You can comment out the validation to test for shorter times.
Verify that the case action is experimental.
Verify that based on the rule type the case is created in the correct solution.
Verify that you cannot create a rule with the case action on the basic license.
Verify that the execution of the case action fails if you do not have permission for cases. Pending work on the system actions framework level to not allow users to create rules with system actions where they do not have permission.
Stress test the case action by creating multiple rules.

Checklist

Delete any items that are not applicable to this PR.

Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios

For maintainers

This was checked for breaking API changes and was labeled appropriately

Release notes

Automatically create cases when an alert is triggered.

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

## Summary This PR is a continuation of the work for the Case action. This PR implements the basic logic of the case connector. Specifically: 1. Group the alerts based on the grouping provided by the user 2. Create the Oracle's SO IDs to fetch the records. If they do not exist they will get created and the counter will be set to 1. 3. Create the cases' SO IDs to fetch the Cases. If they do not exist they will get created. 4. Attach the alerts to the corresponding cases. Not in this PR: - Handle errors - Retries on errors - Reopen cases - Time window - Race conditions - Circuit breakers Depends on: #168370, #169484 ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

…te --fix'

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

## Summary Depends on: #171754 ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

## Summary This PR: 1. Creates the `CasesConnectorError` error 2. Separate the execution logic by moving the current logic to a new class called `CasesConnectorExecutor` 3. Let the `CasesConnector` class handle only the retry logic of the connector 4. Implements the [Full jitter backoff algorithm](https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/) which is used as the retry strategy of the connector Depends on: #172709 ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

## Summary This PR adds logging to the case action ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

pgayvallet

LGTM

x-pack/plugins/cases/server/saved_object_types/cases_rules.ts

x-pack/plugins/cases/server/connectors/cases/README.md

js-jankisalvi

Found a weird bug, created a log threshold rule in o11y, alert overview shows created case in o11y. But it created a case in stack management, because the consumer for log threshold ruleType is alerts.

obs-stack.case.action.conflict.mov

cnasikas · 2024-04-10T09:49:27Z

Found a weird bug, created a log threshold rule in o11y, alert overview shows created case in o11y. But it created a case in stack management, because the consumer for log threshold ruleType is alerts.

obs-stack.case.action.conflict.mov

Interesting bug. @XavierM recommended fallback to the producer of the rule if the consumer is a legacy one. Because it needs some slight changes in the system actions framework, I wonder if we can fix the bug on another PR.

x-pack/plugins/cases/server/connectors/cases/README.md

adcoelho · 2024-04-09T08:44:40Z

x-pack/plugins/cases/server/connectors/cases/cases_connector_executor.ts

+    }
+
+    this.logger.debug(
+      `[CasesConnector][CasesConnectorExecutor][attachAlertsToCases] Attaching alerts to ${casesUnderAlertLimit.length} cases that do not have reach the alert limit per case`,


nit:

Suggested change

`[CasesConnector][CasesConnectorExecutor][attachAlertsToCases] Attaching alerts to ${casesUnderAlertLimit.length} cases that do not have reach the alert limit per case`,

`[CasesConnector][CasesConnectorExecutor][attachAlertsToCases] Attaching alerts to ${casesUnderAlertLimit.length} cases that have not reached the alert limit per case`,

adcoelho · 2024-04-09T09:06:27Z

x-pack/plugins/cases/server/connectors/cases/cases_connector_executor.ts

+      : params.rule.name;
+
+    const groupingDescription = this.getGroupingDescription(grouping);
+    const description = `This case is auto-created by ${ruleName}. \n\n Grouping: ${groupingDescription}`;


nit:

Suggested change

const description = `This case is auto-created by ${ruleName}. \n\n Grouping: ${groupingDescription}`;

const description = `This case is created automatically by ${ruleName}. \n\n Grouping: ${groupingDescription}`;

Shani suggested some changes in the description too. I was thinking of doing it on another PR. Do you mind if we also address this in the next PR?

…fix'

cnasikas · 2024-04-11T07:51:02Z

Some updates:

Janki's bug: We will fall back to the producer of the rule. Needs some changes in the system actions framework to pass the producer to the case action. I would prefer to do it on another PR.
Per Shani's feedback: Remove years and months from the time window - Done in f73ec73 (#168369)
Do not allow users to create a rule with a case action if they do not have access to it. Blocked by [Actions] Authorize system action when create or editing rules #180437. Done in a7ddc63 (#168369)
Show only aggregatable fields. Done in 888b043 (#168369)
The user is "Unknown" when creating a new case from the case action. We need to make some changes in the cases client . It will be done on another PR.

kibana-ci · 2024-04-11T09:00:37Z

💚 Build Succeeded

Buildkite Build
Commit: f73ec73

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`cases`	717	752	+35

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/core-saved-objects-base-server-internal`	180	181	+1
`actions`	290	292	+2
`triggersActionsUi`	567	568	+1
total			+4

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`cases`	446.5KB	475.2KB	+28.7KB
`triggersActionsUi`	1.6MB	1.6MB	+174.0B
total			+28.9KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`cases`	148.9KB	153.0KB	+4.1KB

Unknown metric groups

API count

id	before	after	diff
`@kbn/core-saved-objects-base-server-internal`	223	224	+1
`actions`	296	298	+2
`triggersActionsUi`	593	594	+1
total			+4

async chunk count

id	before	after	diff
`cases`	26	27	+1

ESLint disabled line counts

id	before	after	diff
`cases`	58	61	+3

Total ESLint disabled count

id	before	after	diff
`cases`	74	77	+3

History

💔 Build #202589 failed a7ddc63
💚 Build #202514 succeeded 66012c2
💔 Build #202458 failed 1054f9a
💚 Build #202040 succeeded c6c8609
💔 Build #201878 failed de6d3c5

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @cnasikas

js-jankisalvi

What an amazing work!! 🤯 🎉 👏
Finally case action is here!! 📢 😃
Thank you for detailed scenarios, it made testing easier ❤️

Create a rule with a case action in observability and the stack. The security solution is not supported. You should not be able to assign a case action in a security solution rule. ✅

Test the "Reopen closed cases" configuration. ✅
Test the "Grouping by" configuration. Only one field is allowed. Not all fields are persisted in alerts. If you select a field not part of the alert the case action will create a case where the grouping value is set to unknow.

only single field is allowed, grouping value is unknown for non alert fields ✅ - we still need to prevent new field selection which is not in the list which @adcoelho mentioned, could be done in another PR.

Test the "Time window" feature. You can comment out the validation to test for shorter times. ✅
Verify that the case action is experimental. ✅

Verify that based on the rule type the case is created in the correct solution.

worked as expected, except the bug with log threshold rule with alerts as consumer

Verify that you cannot create a rule with the case action on the basic license. ✅
Verify that the execution of the case action fails if you do not have permission for cases. Pending work on the system actions framework level to not allow users to create rules with system actions where they do not have permission.

verified with none or read permission, in both scenario case action is disabled ✅

Stress test the case action by creating multiple rules. ✅

x-pack/plugins/cases/public/components/system_actions/cases/translations.ts

cnasikas · 2024-04-12T08:14:27Z

we still need to prevent new field selection which is not in the list which @adcoelho mentioned, could be done in another PR.

Thanks for that! I totally forgot it.

jcger

🚀

## Summary Creates a system connector that can call the observability ai assistant to execute actions on behalf of user. The connector is tagged as tech preview. The connector can be triggered when an alert fires. Connector can be configured with an initial message to the assistant which generates an answer and triggers potential actions on the assistant side. The current experimental scenario is to ask the assistant to generate a report of the alert that fired (by initially providing some context in the first message), recalling any information/potential resolutions of previous occurrences stored in the knowledge base and also including other active alerts that may be related. One last step that can be asked to the assistant is to trigger an action, currently only sending the report (or any other message) to a preconfigured slack webhook is supported. ## Testing _Note: when asked to send a message to another connector (in our case slack), we'll try to include a link to the generated conversation. It is only possible to generate this link if [server.publicBaseUrl](https://www.elastic.co/guide/en/kibana/current/settings.html#server-publicBaseUrl) is correctly set in kibana settings._ - Create a slack webhook connector - Get slack webhook. I can share one and invite you to the workspace, or if you want to create one: - create personal workspace at https://slack.com/signin#workspaces - create an app for that workspace at https://api.slack.com/apps - under Features > OAuth & Permissions > Scopes > Bot Token Scopes, add `incoming-webhook` permission - install the app - webhook url is available under Features > Incoming Webhooks - Create a rule that can be triggered with available documents and attach observability AI assistant connector. (I use `Error Count Threshold` and generate errors via `node scripts/synthtrace many_errors.ts --live`) - configure the connector with one genai connector and a message with instructions. Example: ``` High error count alert has triggered. Execute the following steps: - create a graph of the error count for the service impacted by the alert for the last 24h - to help troubleshoot recall past occurrences of this alarm, also any other active alerts. Generate a report with all the found informations and send it to slack connector as a single message. Also include the link to this conversation in the report ``` - Track alert status and verify connector was executed. You should get a slack notification sent by the assistant, and a new conversation will be stored TODO - unit/integration tests - see #168369 for reference implementation - documentation --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Dario Gieselaar <dario.gieselaar@elastic.co>

@adcoelho

## Summary In this PR: - Address @adcoelho comments regarding documentation. - Fix @js-jankisalvi bug about unsupported consumers (#168369 (review)). - Address @shanisagiv1 feedback regarding the title and the description. Specifically: - The title changed to "<rule_name> - Grouping by <grouping_by_value> (Auto-created)". - The description changed to "This case was created by the Case action in <rule_name_link>. The assigned alerts are grouped by <grouping_by_key>:<grouping_by_value>". - Add the grouping key as a tag. <img width="2289" alt="Screenshot 2024-04-13 at 4 41 36 PM" src="https://github.com/elastic/kibana/assets/7871006/63e17947-5f39-4437-820b-7c69f42bfbe3"> The issue about the "Unknown" user will be fixed in another PR. About @adcoelho bug: https://github.com/elastic/kibana/assets/7871006/c46aa7c4-9d1a-475b-9d07-6bdff3ef00c8 I think it is fine to leave it as it is because a) the value will not be saved even if they are added b) an error is being shown c) the only way to do it properly is to validate while the user is typing which is going to lead to bad UX. If you feel otherwise let me know. ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

Init the case action

d06df6d

cnasikas self-assigned this Oct 9, 2023

cnasikas added Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) Feature:Cases Cases feature release_note:feature Makes this part of the condensed release notes labels Oct 9, 2023

cnasikas and others added 25 commits October 16, 2023 11:29

Merge branch 'main' into case_action

0f63cd7

Merge branch 'main' into case_action

1b1caf0

Merge branch 'main' into case_action

1da8277

Merge branch 'main' into case_action

f6c1ce9

[Cases] Case action: Registration and Oracle (#168370)

d5bbd8e

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

Merge branch 'main' into case_action

a035154

Merge branch 'case_action' of github.com:elastic/kibana into case_action

bc79e00

Merge branch 'main' into case_action

d306266

Merge branch 'main' into case_action

f20aeaf

Merge branch 'main' into case_action

c8d3d30

Merge branch 'main' into case_action

76fd124

[CI] Auto-commit changed files from 'node scripts/check_mappings_upda…

5bd13c2

…te --fix'

[Cases] Case action: Time window (#171754)

7a2c5d9

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

Merge branch 'main' into case_action

c095b48

Merge branch 'main' into case_action

a1a9840

Merge branch 'main' into case_action

d37e952

Merge branch 'case_action' of github.com:elastic/kibana into case_action

855b94d

Merge branch 'main' into case_action

da09eea

Merge branch 'main' into case_action

ef60814

Merge branch 'case_action' of github.com:elastic/kibana into case_action

5a88a23

Merge branch 'main' into case_action

63874df

cnasikas added 2 commits April 8, 2024 10:18

Merge branch 'main' into case_action

67712a2

Attach ungrouped alerts

de6d3c5

pgayvallet approved these changes Apr 9, 2024

View reviewed changes

x-pack/plugins/cases/server/saved_object_types/cases_rules.ts Outdated Show resolved Hide resolved

adcoelho reviewed Apr 9, 2024

View reviewed changes

x-pack/plugins/cases/server/connectors/cases/README.md Outdated Show resolved Hide resolved

cnasikas added 2 commits April 9, 2024 11:39

Remove not relevant test

62d0284

Merge branch 'main' into case_action

c6c8609

js-jankisalvi reviewed Apr 9, 2024

View reviewed changes

cnasikas added 2 commits April 10, 2024 12:16

Show only aggregatable fields in the groupingBy

888b043

Merge branch 'main' into case_action

1054f9a

SiddharthMantri approved these changes Apr 10, 2024

View reviewed changes

PR feedback

66012c2

adcoelho reviewed Apr 10, 2024

View reviewed changes

cnasikas and others added 5 commits April 10, 2024 18:07

Merge branch 'main' into case_action

9d3b79e

Registers the required kibana privileges in the connector adapter

a7ddc63

Merge branch 'main' into case_action

5f0181d

[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…

607703d

…fix'

Remove months and years from the time window

f73ec73

js-jankisalvi approved these changes Apr 11, 2024

View reviewed changes

x-pack/plugins/cases/public/components/system_actions/cases/translations.ts Show resolved Hide resolved

jcger approved these changes Apr 12, 2024

View reviewed changes

cnasikas merged commit b735d8c into main Apr 12, 2024
36 checks passed

cnasikas deleted the case_action branch April 12, 2024 09:01

kibanamachine added the backport:skip This commit does not require backporting label Apr 12, 2024

cnasikas mentioned this pull request Apr 15, 2024

[Cases] Case action enhancements and fixes #180758

Merged

2 tasks

cnasikas mentioned this pull request Oct 19, 2024

[ResponseOps][Cases] Mark the case action as GA #196973

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Cases] Case action #168369

[Cases] Case action #168369

cnasikas commented Oct 9, 2023 •

edited

Loading

pgayvallet left a comment

js-jankisalvi left a comment •

edited

Loading

cnasikas commented Apr 10, 2024 •

edited

Loading

adcoelho Apr 9, 2024

adcoelho Apr 9, 2024

cnasikas Apr 11, 2024

cnasikas commented Apr 11, 2024

kibana-ci commented Apr 11, 2024

API count

async chunk count

ESLint disabled line counts

Total ESLint disabled count

js-jankisalvi left a comment

cnasikas commented Apr 12, 2024

jcger left a comment

	`[CasesConnector][CasesConnectorExecutor][attachAlertsToCases] Attaching alerts to ${casesUnderAlertLimit.length} cases that do not have reach the alert limit per case`,
	`[CasesConnector][CasesConnectorExecutor][attachAlertsToCases] Attaching alerts to ${casesUnderAlertLimit.length} cases that have not reached the alert limit per case`,

	const description = `This case is auto-created by ${ruleName}. \n\n Grouping: ${groupingDescription}`;
	const description = `This case is created automatically by ${ruleName}. \n\n Grouping: ${groupingDescription}`;

[Cases] Case action #168369

[Cases] Case action #168369

Conversation

cnasikas commented Oct 9, 2023 • edited Loading

Summary

Testing

Checklist

For maintainers

Release notes

pgayvallet left a comment

Choose a reason for hiding this comment

js-jankisalvi left a comment • edited Loading

Choose a reason for hiding this comment

cnasikas commented Apr 10, 2024 • edited Loading

adcoelho Apr 9, 2024

Choose a reason for hiding this comment

adcoelho Apr 9, 2024

Choose a reason for hiding this comment

cnasikas Apr 11, 2024

Choose a reason for hiding this comment

cnasikas commented Apr 11, 2024

kibana-ci commented Apr 11, 2024

💚 Build Succeeded

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Page load bundle

API count

async chunk count

ESLint disabled line counts

Total ESLint disabled count

History

js-jankisalvi left a comment

Choose a reason for hiding this comment

cnasikas commented Apr 12, 2024

jcger left a comment

Choose a reason for hiding this comment

cnasikas commented Oct 9, 2023 •

edited

Loading

js-jankisalvi left a comment •

edited

Loading

cnasikas commented Apr 10, 2024 •

edited

Loading