From eb9de95267fba03a6f8fa6d14dfb2a3080236c79 Mon Sep 17 00:00:00 2001 From: "nastasha.solomon" Date: Tue, 7 May 2024 21:50:32 -0400 Subject: [PATCH 01/20] First draft --- docs/detections/rules-ui-create.asciidoc | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index bf8e964672..a0d3b7d109 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -311,7 +311,7 @@ NOTE: Rules that use aggregating queries might create duplicate alerts. This can [float] [[esql-non-agg-query]] ===== Non-aggregating query -Non-aggregating queries doesn't use `STATS...BY` functions and doesn't aggregate source event data. Alerts generated by an {esql} rule with a non-aggregating query only contain the fields returned by the query. +Non-aggregating queries don't use `STATS...BY` functions and don't aggregate source event data. Alerts generated by an ES|QL rule with a non-aggregating query contain the fields returned by the query and all fields in the source event document. Here is an example non-aggregating query: [source,esql] @@ -376,6 +376,8 @@ NOTE: The `max_signals` default value is 100. You can modify it using the <>. + [float] [[esql-rule-limitations]] ==== {esql} rule limitations @@ -383,7 +385,6 @@ NOTE: The `max_signals` default value is 100. You can modify it using the <>. -- If your {esql} query creates new fields that aren’t in the query’s source index, they can’t be added to the rule’s <>. [float] [[rule-ui-basic-params]] From 739359b781e4b038e44f8bde9045744fed7918cb Mon Sep 17 00:00:00 2001 From: "nastasha.solomon" Date: Wed, 8 May 2024 22:40:19 -0400 Subject: [PATCH 02/20] Additional information --- docs/detections/rules-ui-create.asciidoc | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index a0d3b7d109..c4ca9acc7a 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -289,7 +289,11 @@ TIP: Click the help icon (image:images/esql-ref-button.png[Click the ES|QL help [[esql-agg-query]] ===== Aggregating query -Aggregating queries use {ref}/esql-functions-operators.html#esql-agg-functions[`STATS...BY`] functions to aggregate source event data. Alerts generated by an {esql} rule with an aggregating query only contain the fields returned by the query. +Aggregating queries use {ref}/esql-functions-operators.html#esql-agg-functions[`STATS...BY`] functions to aggregate source event data. Alerts generated by a rule with an aggregating query only contain the fields that the {esql} query returns and new fields that the query creates. + +TIP: To improve the visibility of these fields while investigating alerts that the {esql} rule generates, add the fields to the rule's <>. + +NOTE: New fields are fields that don't exist in the query’s source index. Here is an example aggregating query: @@ -311,7 +315,11 @@ NOTE: Rules that use aggregating queries might create duplicate alerts. This can [float] [[esql-non-agg-query]] ===== Non-aggregating query -Non-aggregating queries don't use `STATS...BY` functions and don't aggregate source event data. Alerts generated by an ES|QL rule with a non-aggregating query contain the fields returned by the query and all fields in the source event document. +Non-aggregating queries don't use `STATS...BY` functions and don't aggregate source event data. Alerts generated by a non-aggregating query contain the fields the query returns, new fields the query creates, and all other fields in the source event document. + +TIP: To improve the visibility of these fields while investigating alerts that the {esql} rule generates, add the fields to the rule's <>. + +NOTE: New fields are fields that don't exist in the query’s source index. Here is an example non-aggregating query: [source,esql] @@ -376,15 +384,11 @@ NOTE: The `max_signals` default value is 100. You can modify it using the <>. - [float] [[esql-rule-limitations]] ==== {esql} rule limitations -The {esql} rule has the following limitations: - -- If your {esql} query creates new fields that aren’t part of the ECS schema, they won’t be mapped to the alerts index and you can't search or filter for them from the Alerts table. As a workaround, create <>. +If your {esql} query creates new fields that aren’t part of the ECS schema, they won’t be mapped to the alerts index and you can't search or filter for them from the Alerts table. As a workaround, create <>. [float] [[rule-ui-basic-params]] From b0da54d862875ed96f07b6524e9ad30ec6e0739f Mon Sep 17 00:00:00 2001 From: "nastasha.solomon" Date: Wed, 8 May 2024 22:48:47 -0400 Subject: [PATCH 03/20] Small edits --- docs/detections/rules-ui-create.asciidoc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index c4ca9acc7a..41968cb486 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -289,9 +289,9 @@ TIP: Click the help icon (image:images/esql-ref-button.png[Click the ES|QL help [[esql-agg-query]] ===== Aggregating query -Aggregating queries use {ref}/esql-functions-operators.html#esql-agg-functions[`STATS...BY`] functions to aggregate source event data. Alerts generated by a rule with an aggregating query only contain the fields that the {esql} query returns and new fields that the query creates. +Aggregating queries use {ref}/esql-functions-operators.html#esql-agg-functions[`STATS...BY`] functions to aggregate source event data. Alerts generated by a rule with an aggregating query only contain the source event fields that the {esql} query returns and new fields that the query creates. -TIP: To improve the visibility of these fields while investigating alerts that the {esql} rule generates, add the fields to the rule's <>. +TIP: To improve the visibility of these fields while investigating alerts that the {esql} rule generates, add them to the rule's <>. NOTE: New fields are fields that don't exist in the query’s source index. @@ -315,9 +315,9 @@ NOTE: Rules that use aggregating queries might create duplicate alerts. This can [float] [[esql-non-agg-query]] ===== Non-aggregating query -Non-aggregating queries don't use `STATS...BY` functions and don't aggregate source event data. Alerts generated by a non-aggregating query contain the fields the query returns, new fields the query creates, and all other fields in the source event document. +Non-aggregating queries don't use `STATS...BY` functions and don't aggregate source event data. Alerts generated by a non-aggregating query contain source event fields that the query returns, new fields the query creates, and all other fields in the source event document. -TIP: To improve the visibility of these fields while investigating alerts that the {esql} rule generates, add the fields to the rule's <>. +TIP: To improve the visibility of these fields while investigating alerts that the {esql} rule generates, add them to the rule's <>. NOTE: New fields are fields that don't exist in the query’s source index. From ab386a53c89d2672b5490a212243d7666c4d98d2 Mon Sep 17 00:00:00 2001 From: "nastasha.solomon" Date: Sat, 11 May 2024 16:49:18 -0400 Subject: [PATCH 04/20] Part of Vitalii's feedback --- docs/detections/rules-ui-create.asciidoc | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index 41968cb486..73e21fad88 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -289,9 +289,7 @@ TIP: Click the help icon (image:images/esql-ref-button.png[Click the ES|QL help [[esql-agg-query]] ===== Aggregating query -Aggregating queries use {ref}/esql-functions-operators.html#esql-agg-functions[`STATS...BY`] functions to aggregate source event data. Alerts generated by a rule with an aggregating query only contain the source event fields that the {esql} query returns and new fields that the query creates. - -TIP: To improve the visibility of these fields while investigating alerts that the {esql} rule generates, add them to the rule's <>. +Aggregating queries use {ref}/esql-functions-operators.html#esql-agg-functions[`STATS...BY`] functions to aggregate source event data. Alerts generated by a rule with an aggregating query only contain the source event fields that the {esql} query returns and new fields that the query creates. To improve the visibility of these fields while investigating alerts, add them to the rule's <>. NOTE: New fields are fields that don't exist in the query’s source index. @@ -315,11 +313,9 @@ NOTE: Rules that use aggregating queries might create duplicate alerts. This can [float] [[esql-non-agg-query]] ===== Non-aggregating query -Non-aggregating queries don't use `STATS...BY` functions and don't aggregate source event data. Alerts generated by a non-aggregating query contain source event fields that the query returns, new fields the query creates, and all other fields in the source event document. - -TIP: To improve the visibility of these fields while investigating alerts that the {esql} rule generates, add them to the rule's <>. +Non-aggregating queries don't use `STATS...BY` functions and don't aggregate source event data. Alerts generated by a non-aggregating query contain source event fields that the query returns, new fields the query creates, and all other fields in the source event document. To improve the visibility of these fields while investigating alerts that the {esql} rule generates, add them to the rule's <>. -NOTE: New fields are fields that don't exist in the query’s source index. +NOTE: New fields are fields that don't exist in the query’s source index. For example, if you use the {ref}/esql-commands.html#esql-eval[`EVAL`] command to append new columns with calculated values, the newly-added columns are considered new fields. Here is an example non-aggregating query: [source,esql] From 47437844cb79f6ebc6eb3cb601721f1031e8ab6c Mon Sep 17 00:00:00 2001 From: "nastasha.solomon" Date: Sun, 12 May 2024 14:30:16 -0400 Subject: [PATCH 05/20] Re-orged info --- docs/detections/rules-ui-create.asciidoc | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index 73e21fad88..5df391057a 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -289,9 +289,9 @@ TIP: Click the help icon (image:images/esql-ref-button.png[Click the ES|QL help [[esql-agg-query]] ===== Aggregating query -Aggregating queries use {ref}/esql-functions-operators.html#esql-agg-functions[`STATS...BY`] functions to aggregate source event data. Alerts generated by a rule with an aggregating query only contain the source event fields that the {esql} query returns and new fields that the query creates. To improve the visibility of these fields while investigating alerts, add them to the rule's <>. +Aggregating queries use {ref}/esql-functions-operators.html#esql-agg-functions[`STATS...BY`] functions to aggregate source event data. Alerts generated by a rule with an aggregating query only contain the source event fields that the {esql} query returns and new fields that the query creates. -NOTE: New fields are fields that don't exist in the query’s source index. +NOTE: New fields are fields that don't exist in the query’s source index. For example... Here is an example aggregating query: @@ -313,7 +313,7 @@ NOTE: Rules that use aggregating queries might create duplicate alerts. This can [float] [[esql-non-agg-query]] ===== Non-aggregating query -Non-aggregating queries don't use `STATS...BY` functions and don't aggregate source event data. Alerts generated by a non-aggregating query contain source event fields that the query returns, new fields the query creates, and all other fields in the source event document. To improve the visibility of these fields while investigating alerts that the {esql} rule generates, add them to the rule's <>. +Non-aggregating queries don't use `STATS...BY` functions and don't aggregate source event data. Alerts generated by a non-aggregating query contain source event fields that the query returns, new fields the query creates, and all other fields in the source event document. NOTE: New fields are fields that don't exist in the query’s source index. For example, if you use the {ref}/esql-commands.html#esql-eval[`EVAL`] command to append new columns with calculated values, the newly-added columns are considered new fields. @@ -384,7 +384,8 @@ NOTE: The `max_signals` default value is 100. You can modify it using the <>. +- If your {esql} query creates new fields that aren’t part of the ECS schema, they won’t be mapped to the alerts index and you can't search or filter for them from the Alerts table. As a workaround, create <>. +- New fields, which are fields that don't exist in the query’s source index, won't be displayed in the Highlighted fields section of the alert details flyout. To ensure they're visible while you're investigating alerts, add the new fields to the rule's <>. [float] [[rule-ui-basic-params]] From 69e7f007363229923a64c05b1b6860352be75c0d Mon Sep 17 00:00:00 2001 From: "nastasha.solomon" Date: Sun, 12 May 2024 14:50:36 -0400 Subject: [PATCH 06/20] Re-adding into sen --- docs/detections/rules-ui-create.asciidoc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index 5df391057a..b174241137 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -384,6 +384,8 @@ NOTE: The `max_signals` default value is 100. You can modify it using the <>. - New fields, which are fields that don't exist in the query’s source index, won't be displayed in the Highlighted fields section of the alert details flyout. To ensure they're visible while you're investigating alerts, add the new fields to the rule's <>. From 75b48187c184bfe48067b8dbedc49c763641264c Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Sun, 12 May 2024 15:06:20 -0400 Subject: [PATCH 07/20] Update docs/detections/rules-ui-create.asciidoc --- docs/detections/rules-ui-create.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index b174241137..0affe19035 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -387,7 +387,7 @@ NOTE: The `max_signals` default value is 100. You can modify it using the <>. -- New fields, which are fields that don't exist in the query’s source index, won't be displayed in the Highlighted fields section of the alert details flyout. To ensure they're visible while you're investigating alerts, add the new fields to the rule's <>. +- New fields created by your query won't be displayed in the Highlighted fields section of the alert details flyout. To ensure they're visible while you're investigating alerts, add the new fields to the rule's <>. [float] [[rule-ui-basic-params]] From 26de3a4bd298b68d2607ed4481aab82c8a9a2bcd Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Mon, 13 May 2024 08:49:57 -0400 Subject: [PATCH 08/20] Update docs/detections/rules-ui-create.asciidoc --- docs/detections/rules-ui-create.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index 0affe19035..d43a6a1800 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -291,7 +291,7 @@ TIP: Click the help icon (image:images/esql-ref-button.png[Click the ES|QL help Aggregating queries use {ref}/esql-functions-operators.html#esql-agg-functions[`STATS...BY`] functions to aggregate source event data. Alerts generated by a rule with an aggregating query only contain the source event fields that the {esql} query returns and new fields that the query creates. -NOTE: New fields are fields that don't exist in the query’s source index. For example... +NOTE: New fields are fields that don't exist in the query’s source index. For example, if you use the {ref}/esql-commands.html#esql-stats-by[`STATS...BY`] function to create a column with an aggregated value, the newly-added column is considered a new field. Here is an example aggregating query: From c64371a98f9a0d1ad2dfd723223b7699796b2a5c Mon Sep 17 00:00:00 2001 From: "nastasha.solomon" Date: Mon, 13 May 2024 20:49:03 -0400 Subject: [PATCH 09/20] Draft of Vitalii's input --- docs/detections/rules-ui-create.asciidoc | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index d43a6a1800..5e73d43048 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -382,12 +382,15 @@ NOTE: The `max_signals` default value is 100. You can modify it using the <>. -- If your {esql} query creates new fields that aren’t part of the ECS schema, they won’t be mapped to the alerts index and you can't search or filter for them from the Alerts table. As a workaround, create <>. -- New fields created by your query won't be displayed in the Highlighted fields section of the alert details flyout. To ensure they're visible while you're investigating alerts, add the new fields to the rule's <>. +[float] +[[esql-rule-limitations]] +==== Highlight fields returned by the {esql} rule query + +When configuring an {esql} rule's **<>**, you can specify any source event fields and new fields that the rule's aggregating or non-aggregating query returns. This can help ensure that returned fields are visible in the alert details flyout while you're investigating alerts. [float] [[rule-ui-basic-params]] From df9eabe267b5ee53dd65d39ef429b341c35018c6 Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Mon, 13 May 2024 21:24:00 -0400 Subject: [PATCH 10/20] Update docs/detections/rules-ui-create.asciidoc --- docs/detections/rules-ui-create.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index 5e73d43048..e8cee26f28 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -387,7 +387,7 @@ NOTE: The `max_signals` default value is 100. You can modify it using the <>. [float] -[[esql-rule-limitations]] +[[custom-highlighted-esql-fields]] ==== Highlight fields returned by the {esql} rule query When configuring an {esql} rule's **<>**, you can specify any source event fields and new fields that the rule's aggregating or non-aggregating query returns. This can help ensure that returned fields are visible in the alert details flyout while you're investigating alerts. From 584679014da017772361314df76fe68c8f3b9b50 Mon Sep 17 00:00:00 2001 From: "nastasha.solomon" Date: Tue, 14 May 2024 09:12:43 -0400 Subject: [PATCH 11/20] Expanding definition for new fields --- docs/detections/rules-ui-create.asciidoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index 5e73d43048..e4719e6f84 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -291,7 +291,7 @@ TIP: Click the help icon (image:images/esql-ref-button.png[Click the ES|QL help Aggregating queries use {ref}/esql-functions-operators.html#esql-agg-functions[`STATS...BY`] functions to aggregate source event data. Alerts generated by a rule with an aggregating query only contain the source event fields that the {esql} query returns and new fields that the query creates. -NOTE: New fields are fields that don't exist in the query’s source index. For example, if you use the {ref}/esql-commands.html#esql-stats-by[`STATS...BY`] function to create a column with an aggregated value, the newly-added column is considered a new field. +NOTE: New fields don't exist in the source index that the query is searching. Instead, they're created when the query runs, and you can access them in the details of any alerts that are generated. For example, if you use the {ref}/esql-commands.html#esql-stats-by[`STATS...BY`] function to create a column with aggregated values, the column is created when the query runs, and is added as a new field to any alerts that are generated by the rule. Here is an example aggregating query: @@ -315,7 +315,7 @@ NOTE: Rules that use aggregating queries might create duplicate alerts. This can ===== Non-aggregating query Non-aggregating queries don't use `STATS...BY` functions and don't aggregate source event data. Alerts generated by a non-aggregating query contain source event fields that the query returns, new fields the query creates, and all other fields in the source event document. -NOTE: New fields are fields that don't exist in the query’s source index. For example, if you use the {ref}/esql-commands.html#esql-eval[`EVAL`] command to append new columns with calculated values, the newly-added columns are considered new fields. +NOTE: New fields don't exist in the source index that the query is searching. Instead, they're created when the query runs, and you can access them in the details of any alerts that are generated. For example, if you use the {ref}/esql-commands.html#esql-eval[`EVAL`] command to append new columns with calculated values, the columns are created when the query runs, and are added as new fields to any alerts that are generated by the rule. Here is an example non-aggregating query: [source,esql] From 664f620e4ae1f10a099bba4f3e87e8174a3140ff Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Tue, 14 May 2024 10:08:41 -0400 Subject: [PATCH 12/20] Update docs/detections/rules-ui-create.asciidoc --- docs/detections/rules-ui-create.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index cb9aa292d2..9d50832564 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -390,7 +390,7 @@ If your {esql} query creates new fields that aren’t part of the ECS schema, th [[custom-highlighted-esql-fields]] ==== Highlight fields returned by the {esql} rule query -When configuring an {esql} rule's **<>**, you can specify any source event fields and new fields that the rule's aggregating or non-aggregating query returns. This can help ensure that returned fields are visible in the alert details flyout while you're investigating alerts. +When configuring an {esql} rule's **<>**, you can specify any fields that the rule's aggregating or non-aggregating query returns. This can help ensure that returned fields are visible in the alert details flyout while you're investigating alerts. [float] [[rule-ui-basic-params]] From 0fadccc7b9ee7d2f7d5bb748e18ec24eb3323d66 Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Tue, 14 May 2024 10:09:15 -0400 Subject: [PATCH 13/20] Update docs/detections/rules-ui-create.asciidoc --- docs/detections/rules-ui-create.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index 9d50832564..cdf588db3f 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -289,7 +289,7 @@ TIP: Click the help icon (image:images/esql-ref-button.png[Click the ES|QL help [[esql-agg-query]] ===== Aggregating query -Aggregating queries use {ref}/esql-functions-operators.html#esql-agg-functions[`STATS...BY`] functions to aggregate source event data. Alerts generated by a rule with an aggregating query only contain the source event fields that the {esql} query returns and new fields that the query creates. +Aggregating queries use {ref}/esql-functions-operators.html#esql-agg-functions[`STATS...BY`] functions to aggregate source event data. Alerts generated by a rule with an aggregating query only contain the fields that the {esql} query returns and new fields that the query creates. NOTE: New fields don't exist in the source index that the query is searching. Instead, they're created when the query runs, and you can access them in the details of any alerts that are generated. For example, if you use the {ref}/esql-commands.html#esql-stats-by[`STATS...BY`] function to create a column with aggregated values, the column is created when the query runs, and is added as a new field to any alerts that are generated by the rule. From 23d529a0e3986aafff56fce1676c925b8c279537 Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Tue, 14 May 2024 16:37:20 -0400 Subject: [PATCH 14/20] Update docs/detections/rules-ui-create.asciidoc Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com> --- docs/detections/rules-ui-create.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index 4e6d25eae4..512cbc7a7f 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -289,7 +289,7 @@ TIP: Click the help icon (image:images/esql-ref-button.png[Click the ES|QL help [[esql-agg-query]] ===== Aggregating query -Aggregating queries use {ref}/esql-functions-operators.html#esql-agg-functions[`STATS...BY`] functions to aggregate source event data. Alerts generated by a rule with an aggregating query only contain the fields that the {esql} query returns and new fields that the query creates. +Aggregating queries use {ref}/esql-functions-operators.html#esql-agg-functions[`STATS...BY`] functions to aggregate source event data. Alerts generated by a rule with an aggregating query only contain the fields that the {esql} query returns and any new fields that the query creates. NOTE: New fields don't exist in the source index that the query is searching. Instead, they're created when the query runs, and you can access them in the details of any alerts that are generated. For example, if you use the {ref}/esql-commands.html#esql-stats-by[`STATS...BY`] function to create a column with aggregated values, the column is created when the query runs, and is added as a new field to any alerts that are generated by the rule. From 47fb98ede2743ce4d0d8f69e1d94ab312ea6129a Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Tue, 14 May 2024 16:49:54 -0400 Subject: [PATCH 15/20] Update docs/detections/rules-ui-create.asciidoc Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com> --- docs/detections/rules-ui-create.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index 512cbc7a7f..5dcb8aa197 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -315,7 +315,7 @@ NOTE: Rules that use aggregating queries might create duplicate alerts. This can ===== Non-aggregating query Non-aggregating queries don't use `STATS...BY` functions and don't aggregate source event data. Alerts generated by a non-aggregating query contain source event fields that the query returns, new fields the query creates, and all other fields in the source event document. -NOTE: New fields don't exist in the source index that the query is searching. Instead, they're created when the query runs, and you can access them in the details of any alerts that are generated. For example, if you use the {ref}/esql-commands.html#esql-eval[`EVAL`] command to append new columns with calculated values, the columns are created when the query runs, and are added as new fields to any alerts that are generated by the rule. +NOTE: New fields don't exist in the source index that the query is searching. Instead, they're created when the query runs, and you can access them in the details of any alerts that are generated. For example, if you use the {ref}/esql-commands.html#esql-eval[`EVAL`] command to append new columns with calculated values, the columns are created when the query runs, and are added as new fields to any alerts generated by the rule. Here is an example non-aggregating query: [source,esql] From b52728ce6cc48717c46f8749ef7d9a7f7fca128e Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Tue, 14 May 2024 16:50:36 -0400 Subject: [PATCH 16/20] Update docs/detections/rules-ui-create.asciidoc Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com> --- docs/detections/rules-ui-create.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index 5dcb8aa197..847b083564 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -384,7 +384,7 @@ NOTE: The `max_signals` default value is 100. You can modify it using the <>. +If your {esql} query creates new fields that aren’t part of the ECS schema, they aren't mapped to the alerts index so you can't search for or filter them in the Alerts table. As a workaround, create <>. [float] [[custom-highlighted-esql-fields]] From fbdc6dc0f74471a826395443103bdddc94d98076 Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Tue, 14 May 2024 16:50:45 -0400 Subject: [PATCH 17/20] Update docs/detections/rules-ui-create.asciidoc Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com> --- docs/detections/rules-ui-create.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index 847b083564..0f48a7ff4a 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -390,7 +390,7 @@ If your {esql} query creates new fields that aren’t part of the ECS schema, th [[custom-highlighted-esql-fields]] ==== Highlight fields returned by the {esql} rule query -When configuring an {esql} rule's **<>**, you can specify any fields that the rule's aggregating or non-aggregating query returns. This can help ensure that returned fields are visible in the alert details flyout while you're investigating alerts. +When configuring an {esql} rule's **<>**, you can specify any fields that the rule's aggregating or non-aggregating query return. This can help ensure that returned fields are visible in the alert details flyout while you're investigating alerts. [float] [[rule-ui-basic-params]] From 218e0d3914c2bf3c02647483a32bcace5dba1aab Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Fri, 17 May 2024 16:08:56 -0400 Subject: [PATCH 18/20] Update docs/detections/rules-ui-create.asciidoc --- docs/detections/rules-ui-create.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index 0f48a7ff4a..80f0ae2c57 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -291,7 +291,7 @@ TIP: Click the help icon (image:images/esql-ref-button.png[Click the ES|QL help Aggregating queries use {ref}/esql-functions-operators.html#esql-agg-functions[`STATS...BY`] functions to aggregate source event data. Alerts generated by a rule with an aggregating query only contain the fields that the {esql} query returns and any new fields that the query creates. -NOTE: New fields don't exist in the source index that the query is searching. Instead, they're created when the query runs, and you can access them in the details of any alerts that are generated. For example, if you use the {ref}/esql-commands.html#esql-stats-by[`STATS...BY`] function to create a column with aggregated values, the column is created when the query runs, and is added as a new field to any alerts that are generated by the rule. +NOTE: A _new field_ is a field that doesn't exist in the query's source index and is instead created when the rule runs. You can access new fields in the details of any alerts that are generated by the rule. For example, if you use the {ref}/esql-commands.html#esql-stats-by[`STATS...BY`] function to create a column with aggregated values, the column is created when the rule runs, and is added as a new field to any alerts that are generated by the rule. Here is an example aggregating query: From d24c3d291252006642aca121cb980d927190cb55 Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Fri, 17 May 2024 16:13:03 -0400 Subject: [PATCH 19/20] Update docs/detections/rules-ui-create.asciidoc --- docs/detections/rules-ui-create.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index 80f0ae2c57..25c27535e1 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -315,7 +315,7 @@ NOTE: Rules that use aggregating queries might create duplicate alerts. This can ===== Non-aggregating query Non-aggregating queries don't use `STATS...BY` functions and don't aggregate source event data. Alerts generated by a non-aggregating query contain source event fields that the query returns, new fields the query creates, and all other fields in the source event document. -NOTE: New fields don't exist in the source index that the query is searching. Instead, they're created when the query runs, and you can access them in the details of any alerts that are generated. For example, if you use the {ref}/esql-commands.html#esql-eval[`EVAL`] command to append new columns with calculated values, the columns are created when the query runs, and are added as new fields to any alerts generated by the rule. +NOTE: A _new field_ is a field that doesn't exist in the query's source index and is instead created when the rule runs. You can access new fields in the details of any alerts that are generated by the rule. For example, if you use the {ref}/esql-commands.html#esql-eval[`EVAL`] command to append new columns with calculated values, the columns are created when the rule runs, and are added as new fields to any alerts generated by the rule. Here is an example non-aggregating query: [source,esql] From 814166bb68f1aab7a40f486f66e6948be9fe1df4 Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Fri, 17 May 2024 16:34:22 -0400 Subject: [PATCH 20/20] Update docs/detections/rules-ui-create.asciidoc --- docs/detections/rules-ui-create.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index 25c27535e1..77e97dcfab 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -291,7 +291,7 @@ TIP: Click the help icon (image:images/esql-ref-button.png[Click the ES|QL help Aggregating queries use {ref}/esql-functions-operators.html#esql-agg-functions[`STATS...BY`] functions to aggregate source event data. Alerts generated by a rule with an aggregating query only contain the fields that the {esql} query returns and any new fields that the query creates. -NOTE: A _new field_ is a field that doesn't exist in the query's source index and is instead created when the rule runs. You can access new fields in the details of any alerts that are generated by the rule. For example, if you use the {ref}/esql-commands.html#esql-stats-by[`STATS...BY`] function to create a column with aggregated values, the column is created when the rule runs, and is added as a new field to any alerts that are generated by the rule. +NOTE: A _new field_ is a field that doesn't exist in the query's source index and is instead created when the rule runs. You can access new fields in the details of any alerts that are generated by the rule. For example, if you use the `STATS...BY` function to create a column with aggregated values, the column is created when the rule runs and is added as a new field to any alerts that are generated by the rule. Here is an example aggregating query: