[Response Ops][Cases] Fetch alerts within observability #123883

jonathan-buttner · 2022-01-26T22:41:48Z

This PR adds functionality to cases within observability to retrieve the alerts data so the rules links can be built in the event that the case comments does not have the information.

If the alert can't be found it will still mark the rule as Unknown.

Example working in observability

Note

The alerts client returns the alert data in the form of collapsed fields "a.b.c" instead of it being nested. I removed the requirement for the useFetchAlertData to return Ecs because that will be misleading. Security solution has functionality that transforms the collapsed fields into nested fields by recursively iterating over the results. I'm open to doing that as well but we'd need to include those helper functions and I don't think it buys us much since we can just use lodash's get to access it as a collapsed field.

Example ES response

{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "max_score": null,
    "hits": [
      {
        "_index": ".internal.alerts-observability.metrics.alerts-default-000001",
        "_id": "662aba44-d24f-47f9-8933-0e0f81cae6df",
        "_seq_no": 696,
        "_primary_term": 1,
        "_score": null,
        "_source": {
          "kibana.alert.rule.category": "Metric threshold",
          "kibana.alert.rule.consumer": "infrastructure",
          "kibana.alert.rule.execution.uuid": "40cc6f59-1101-45b1-81df-fdb47f87fcc4",
          "kibana.alert.rule.name": "Jon rule",
          "kibana.alert.rule.producer": "infrastructure",
          "kibana.alert.rule.rule_type_id": "metrics.alert.threshold",
          "kibana.alert.rule.uuid": "325830a0-7eba-11ec-a3b9-350ed5c64d7e",
          "kibana.space_ids": [
            "default"
          ],
          "kibana.alert.rule.tags": [],
          "@timestamp": "2022-01-26T22:37:20.409Z",
          "kibana.alert.reason": "Document count has reported no data over the past 1m for *",
          "kibana.alert.duration.us": 26767233000,
          "kibana.alert.instance.id": "*",
          "kibana.alert.start": "2022-01-26T15:11:13.176Z",
          "kibana.alert.uuid": "662aba44-d24f-47f9-8933-0e0f81cae6df",
          "kibana.alert.status": "active",
          "kibana.alert.workflow_status": "open",
          "event.kind": "signal",
          "event.action": "active",
          "kibana.version": "8.1.0",
          "tags": []
        },
        "fields": {
          "kibana.alert.rule.rule_type_id": [
            "metrics.alert.threshold"
          ],
          "kibana.alert.workflow_status": [
            "open"
          ],
          "kibana.space_ids": [
            "default"
          ],
          "kibana.alert.rule.consumer": [
            "infrastructure"
          ]
        },
        "sort": [
          1643236640409
        ]
      }
    ]
  }
}

Testing

To get an alert to show up in observability I typically navigate to the Metrics section in observability. Then at the top where it says Alerts -> Create -> Metrics and then use a Document Count is above 0. Then all you need to do is add some data and the rule should be active and show up under the Alerts tab.

Once you have that, grab the alert ID out of the comment and make a postman request with this body:

POST api/cases/<case id>/comments

{
    "alertId": [<add a valid alertId here>],
    "index": [".internal.alerts-observability.metrics.alerts-default-000001"],
    "rule": { "id": null, "name": null },
    "type": "alert",
    "owner": "observability"
}

The case should show two alerts that both have the rule name link created. Following the link should take you to the appropriate rule.

Release Notes

This fixes an issue where the rule's link in Cases within Observability were displayed as Unknown rule even when we could retrieve the correct rule name.

Before:

After:

x-pack/plugins/observability/public/pages/cases/helpers.ts

jonathan-buttner · 2022-01-26T22:46:25Z

x-pack/plugins/observability/public/pages/cases/helpers.ts

          setAlert(parsedAlert);
-          setLoading(false);


I think we want to set loading to false when we're done, not only when we successfully get data?

jonathan-buttner · 2022-01-26T22:47:47Z

x-pack/plugins/observability/public/pages/cases/helpers.ts

      }
    };

-    if (!isEmpty(alertId) && loading === false && alert === null) {


I was running into rerender issues here as well when I added the abort signal, but removing these dependencies seemed to fix it. I don't think we want to rely on loading here because it'll change during the fetch.

you still want to check that is not loading or you can end up loading the same data twice.

jonathan-buttner · 2022-01-27T17:23:23Z

x-pack/plugins/cases/public/components/types.ts

@@ -6,3 +6,5 @@
 */

 export type { CaseActionConnector } from '../../common/ui/types';
+
+export type FetchAlertDataFunction = (alertIds: string[]) => [boolean, Record<string, unknown>];


Should this be in common/ui instead?

Yeap it makes more sense to me. Also, I think it is better if we renamed it to UseFetchAlertData so we can understand that the type is about a React hook.

elasticmachine · 2022-01-27T17:23:58Z

Pinging @elastic/response-ops (Team:ResponseOps)

elasticmachine · 2022-01-27T17:23:59Z

Pinging @elastic/response-ops-cases (Feature:Cases)

jonathan-buttner · 2022-01-27T19:23:35Z

@elasticmachine merge upstream

academo · 2022-01-28T15:11:57Z

x-pack/plugins/observability/public/pages/cases/helpers.ts

+  const [loading, setLoading] = useState(false);
+  const [alerts, setAlerts] = useState<Record<string, unknown> | undefined>(undefined);
+
+  const fetch = useCallback(


alternatively you can write this fetch function out of the hook, accept parameters and returns a promise, then where you use it you await the promise and call setAlerts there.

academo · 2022-01-28T15:16:39Z

x-pack/plugins/cases/public/components/user_actions/types.ts

@@ -52,7 +53,7 @@ export interface UserActionBuilderArgs {
  selectedOutlineCommentId: string;
  loadingCommentIds: string[];
  loadingAlertData: boolean;
-  alertData: Record<string, Ecs>;
+  alertData: Record<string, unknown>;


is unknown really the best type here? is there a closer real type we can use for this? like a Record or a string?

I'm open to other ideas, but I'm not quite sure what to do since security solution's implementation of this hook used to return an Ecs object with the fields broken done into nested objects like:

{ kibana: { alert: "some value", } }

Where as this function will return it as:

{ "kibana.alert": "some value" }

So the values could be object | string | string[] | number | number[] and potentially nested

By putting unknown I think we force the recipient to figure out what type the data is before they use it which might be safe.

Let's roll with this one and create an issue to work on a proper data type?

Will do, I'll create an issue

Issue: #124048

academo · 2022-01-28T15:18:19Z

x-pack/plugins/observability/public/pages/cases/helpers.ts

+import { BASE_RAC_ALERTS_API_PATH } from '../../../../rule_registry/common/constants';
+
+export const useFetchAlertData = (alertIds: string[]): [boolean, Record<string, unknown>] => {
+  const { http } = useKibana().services;


tip: you can also use useHttp() it returns the http object directly.

Are you referring to this one? https://github.com/elastic/kibana/blob/main/x-pack/plugins/cases/public/common/lib/kibana/hooks.ts#L32

Doesn't look like I can access that while inside of observability, should I move the useHttp hook into cases/common?

I do not think you should move it to cases/common so other plugins can use it. useHttp is a cases helper for internal use and useKibana is more appropriate in other plugins (unless they have their own helpers).

…link

…ana into cases-obs-rule-link

jonathan-buttner · 2022-01-30T16:35:09Z

@elasticmachine merge upstream

jonathan-buttner · 2022-01-31T02:52:24Z

@elasticmachine merge upstream

academo · 2022-02-03T08:31:12Z

x-pack/plugins/observability/public/pages/cases/use_fetch_alert_data.ts

+  });
+
+  useEffect(() => {
+    fetch();


this is a way to do it but it adds extra complications that are not necessary.

You can start the fetch immediately inside useDataFetcher and invoke it here, have it return a promise and await for that promise here.

jonathan-buttner · 2022-02-03T19:57:01Z

@elasticmachine merge upstream

cnasikas · 2022-02-07T08:56:58Z

@elasticmachine merge upstream

academo · 2022-02-07T15:10:19Z

🐑🐑🐑

angorayc

Tested locally and LGTM

jonathan-buttner · 2022-02-08T17:47:33Z

@elasticmachine merge upstream

jonathan-buttner · 2022-02-09T14:10:34Z

@elasticmachine merge upstream

kibana-ci · 2022-02-09T16:05:36Z

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`observability`	349	352	+3

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`cases`	294.8KB	294.8KB	+42.0B
`observability`	392.6KB	393.6KB	+1.0KB
total			+1.1KB

History

💚 Build #22587 succeeded 8a42ed8
💔 Build #22564 failed 2934a9a
💚 Build #22035 succeeded 953f058
💚 Build #21698 succeeded 7b9dffb
💛 Build #21638 was flaky 200b9b3
💚 Build #21444 succeeded ecc71a9

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit 30ed7bb)

kibanamachine · 2022-02-11T09:54:09Z

💔 Some backports could not be created

Status	Branch	Result
✅	8.1
❌	8.0	Backport failed because of merge conflicts

How to fix

Re-run the backport manually:

node scripts/backport --pr 123883

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

…25347) Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit 30ed7bb) Co-authored-by: Jonathan Buttner <56361221+jonathan-buttner@users.noreply.github.com>

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit 30ed7bb) # Conflicts: # x-pack/plugins/cases/README.md # x-pack/plugins/cases/public/components/app/types.ts # x-pack/plugins/cases/public/components/case_view/types.ts # x-pack/plugins/cases/public/components/user_actions/comment/alert.tsx # x-pack/plugins/cases/public/components/user_actions/index.test.tsx # x-pack/plugins/cases/public/components/user_actions/index.tsx # x-pack/plugins/cases/public/components/user_actions/types.ts # x-pack/plugins/observability/public/pages/cases/cases.tsx # x-pack/plugins/security_solution/public/cases/pages/use_fetch_alert_data.ts

… (#125370) Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Jonathan Buttner <56361221+jonathan-buttner@users.noreply.github.com>

Refetch loop with abort controller

71fff69

jonathan-buttner added release_note:fix Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) Feature:Cases Cases feature v8.1.0 v8.0.1 labels Jan 26, 2022

jonathan-buttner commented Jan 26, 2022

View reviewed changes

x-pack/plugins/observability/public/pages/cases/helpers.ts Outdated Show resolved Hide resolved

jonathan-buttner commented Jan 26, 2022

View reviewed changes

Adding memoization and fixing types

ae62352

jonathan-buttner commented Jan 27, 2022

View reviewed changes

jonathan-buttner requested review from cnasikas, peteharverson and academo January 27, 2022 17:23

jonathan-buttner marked this pull request as ready for review January 27, 2022 17:23

jonathan-buttner requested a review from a team as a code owner January 27, 2022 17:23

Merge branch 'main' into cases-obs-rule-link

129de80

academo reviewed Jan 28, 2022

View reviewed changes

jonathan-buttner added 2 commits January 28, 2022 11:47

Merge branch 'main' of github.com:elastic/kibana into cases-obs-rule-…

b285226

…link

Merge branch 'cases-obs-rule-link' of github.com:jonathan-buttner/kib…

160a8f3

…ana into cases-obs-rule-link

jonathan-buttner mentioned this pull request Jan 28, 2022

[Response Ops][Cases] Define a better type for useFetchAlertData hooks #124048

Open

Merge branch 'main' into cases-obs-rule-link

722eadc

kibanamachine requested a review from a team as a code owner January 30, 2022 16:35

academo reviewed Feb 3, 2022

View reviewed changes

Moving use effect into helper hook

200b9b3

Merge branch 'main' into cases-obs-rule-link

7b9dffb

Merge branch 'main' into cases-obs-rule-link

953f058

cnasikas added Feature:Grok Debugger Dev Tools Grok Debugger feature bug Fixes for quality problems that affect the customer experience and removed Feature:Grok Debugger Dev Tools Grok Debugger feature labels Feb 7, 2022

jonathan-buttner requested a review from academo February 7, 2022 15:09

academo approved these changes Feb 7, 2022

View reviewed changes

angorayc approved these changes Feb 8, 2022

View reviewed changes

jonathan-buttner added the v8.2.0 label Feb 8, 2022

kibanamachine and others added 2 commits February 8, 2022 12:47

Merge branch 'main' into cases-obs-rule-link

2934a9a

Fixing test

8a42ed8

Merge branch 'main' into cases-obs-rule-link

d10f375

cnasikas merged commit 30ed7bb into elastic:main Feb 11, 2022

cnasikas deleted the cases-obs-rule-link branch February 11, 2022 09:50

cnasikas added the auto-backport Deprecated - use backport:version if exact versions are needed label Feb 11, 2022

kibanamachine mentioned this pull request Feb 11, 2022

[8.1] [Response Ops][Cases] Fetch alerts within observability (#123883) #125347

Merged

cnasikas mentioned this pull request Feb 11, 2022

[8.0] [Response Ops][Cases] Fetch alerts within observability (#123883) #125370

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Response Ops][Cases] Fetch alerts within observability #123883

[Response Ops][Cases] Fetch alerts within observability #123883

jonathan-buttner commented Jan 26, 2022 •

edited

Loading

jonathan-buttner Jan 26, 2022

jonathan-buttner Jan 26, 2022

academo Jan 28, 2022

jonathan-buttner Jan 27, 2022

cnasikas Feb 1, 2022

elasticmachine commented Jan 27, 2022

elasticmachine commented Jan 27, 2022

jonathan-buttner commented Jan 27, 2022

academo Jan 28, 2022 •

edited

Loading

academo Jan 28, 2022

jonathan-buttner Jan 28, 2022

academo Jan 28, 2022

jonathan-buttner Jan 28, 2022

jonathan-buttner Jan 28, 2022

academo Jan 28, 2022

jonathan-buttner Jan 28, 2022 •

edited

Loading

cnasikas Jan 31, 2022

jonathan-buttner commented Jan 30, 2022

jonathan-buttner commented Jan 31, 2022

academo Feb 3, 2022

jonathan-buttner commented Feb 3, 2022

cnasikas commented Feb 7, 2022

academo commented Feb 7, 2022

angorayc left a comment

jonathan-buttner commented Feb 8, 2022

jonathan-buttner commented Feb 9, 2022

kibana-ci commented Feb 9, 2022

kibanamachine commented Feb 11, 2022

[Response Ops][Cases] Fetch alerts within observability #123883

[Response Ops][Cases] Fetch alerts within observability #123883

Conversation

jonathan-buttner commented Jan 26, 2022 • edited Loading

Note

Testing

Release Notes

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elasticmachine commented Jan 27, 2022

elasticmachine commented Jan 27, 2022

jonathan-buttner commented Jan 27, 2022

academo Jan 28, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonathan-buttner Jan 28, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonathan-buttner commented Jan 30, 2022

jonathan-buttner commented Jan 31, 2022

Choose a reason for hiding this comment

jonathan-buttner commented Feb 3, 2022

cnasikas commented Feb 7, 2022

academo commented Feb 7, 2022

angorayc left a comment

Choose a reason for hiding this comment

jonathan-buttner commented Feb 8, 2022

jonathan-buttner commented Feb 9, 2022

kibana-ci commented Feb 9, 2022

💚 Build Succeeded

Metrics [docs]

Module Count

Async chunks

History

kibanamachine commented Feb 11, 2022

💔 Some backports could not be created

How to fix

Questions ?

jonathan-buttner commented Jan 26, 2022 •

edited

Loading

academo Jan 28, 2022 •

edited

Loading

jonathan-buttner Jan 28, 2022 •

edited

Loading