Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Known issue][Detection Engine][8.16.0 & 8.16.1] Exception item comment validation on newline chars #6280

Open
wants to merge 20 commits into
base: 8.x
Choose a base branch
from
Open
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
118 changes: 118 additions & 0 deletions docs/release-notes/8.16.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,65 @@
[[known-issue-8.16.1]]
==== Known issues

// tag::known-issue[201820]
[discrete]
.The **Exceptions** tab won't properly load if exceptions contain comments with newline characters (`\n`)
[%collapsible]
====
*Details* +
On December 5, 2024, it was discovered that the **Exceptions** tab won't load properly if any exceptions contain comments with newline characters (`\n`). This issue occurs when you upgrade to 8.16.0 or later.
nastasha-solomon marked this conversation as resolved.
Show resolved Hide resolved

*Workaround* +

For custom rules:

. From the **Rules** page, <<import-export-rules-ui,export>> the rule or rules with the affected exception lists.
. Modify the `.ndjson` file so `comments` no longer contain newline characters.
. Return to the **Rules** page and <<import-export-rules-ui,re-import>> the rules. Make sure to select the **Overwrite existing exception lists with conflicting "list_id"** option.

For prebuilt rules:

NOTE: If you only need to fix exceptions for the Elastic Endpoint rule, you can export and re-import its exception list from the **<<shared-exception-lists,Shared Exception Lists>>** page.

. Follow these steps to fetch the affected exception list ID or IDs that are associated with the rule:
.. Find the affected rule's ID (`id`). From the **Rules** page, open the details of a rule, go to the page URL, and copy the string at the end. For example, in the URL http://host.name/app/security/rules/id/167a5f6f-2148-4792-8226-b5e7a58ef46e, the string at the end (`167a5f6f-2148-4792-8226-b5e7a58ef46e`) is the `id`.
.. Specify the `id` when fetching the rule's details using the {api-kibana}/operation/operation-readrule[Retrieve a detection rule API]. Here is an example request that includes the `id`:
+
[source,console]
----
curl -H 'kbn-xsrf: true' -H 'elastic-api-version: 2023-10-31' http://localhost:5601/api/detection_engine/rules?id=167a5f6f-2148-4792-8226-b5e7a58ef46e -u elastic:changeme
----
+
.. The JSON response contains the `id`, `list_id`, and `namespace_type` values within the `exceptions_list` key (as shown below). You need these values when using the Exception list API to retrieve the affected exception list.
+
[source,console]
----
{
"id": "167a5f6f-2148-4792-8226-b5e7a58ef46e",
"exceptions_list": [
{
"id": "490525a2-eb66-4320-95b5-88bdd1302dc4",
"list_id": "f75aae6f-0229-413f-881d-81cb3abfbe2d",
"namespace_type": "single"
}
]
}
----
+
. Use the export exceptions API to retrieve the affected exception list. Insert the values for the `id`, `list_id`, and `namespace_type` parameters into the following API call:
+
[source,console]
----
curl -XPOST -H 'Authorization: ApiKey API_KEY_HERE' -H 'kbn-xsrf: true' -H 'elastic-api-version: 2023-10-31' 'KIBANA_URL/api/exception_lists/_export?list_id=f75aae6f-0229-413f-881d-81cb3abfbe2d&id=490525a2-eb66-4320-95b5-88bdd1302dc4&namespace_type=single' -o list.ndjson
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how we usually document curl requests, do we need to specify that they need to also include the space? If not, ignore.

Suggested change
curl -XPOST -H 'Authorization: ApiKey API_KEY_HERE' -H 'kbn-xsrf: true' -H 'elastic-api-version: 2023-10-31' 'KIBANA_URL/api/exception_lists/_export?list_id=f75aae6f-0229-413f-881d-81cb3abfbe2d&id=490525a2-eb66-4320-95b5-88bdd1302dc4&namespace_type=single' -o list.ndjson
curl -XPOST -H 'Authorization: ApiKey API_KEY_HERE' -H 'kbn-xsrf: true' -H 'elastic-api-version: 2023-10-31' '${KIBANA_URL}${SPACE_URL}/api/exception_lists/_export?list_id=f75aae6f-0229-413f-881d-81cb3abfbe2d&id=490525a2-eb66-4320-95b5-88bdd1302dc4&namespace_type=single' -o list.ndjson

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question, I'm not sure. Do users still need to specify the space if they're using the default one?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't believe so, think just if they need to hit a different space. Might be worth just noting somewhere though to make sure to include space if other than default?

----
+
. Modify the exception list's `.ndjson` file to ensure `comments[].comment` values don't contain newline characters (`\n`).
. Re-import the modified exception list using **Import exception lists** option on the **Shared Exception Lists** page (find the page in the navigation menu or by using the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]).
nastasha-solomon marked this conversation as resolved.
Show resolved Hide resolved
+
Note that the import will initially fail because the exception list already exists. After the initial failure, an option to overwrite the existing list will appear. Select the option, then resubmit the request to import the corrected exception list.
nastasha-solomon marked this conversation as resolved.
Show resolved Hide resolved
====
// end::known-issue[201820]

// tag::known-issue[]
[discrete]
.Duplicate alerts can be produced from manually running threshold rules
Expand Down Expand Up @@ -50,6 +109,65 @@ On November 12, 2024, it was discovered that manually running a custom query rul
[[known-issue-8.16.0]]
==== Known issues

// tag::known-issue[201820]
[discrete]
.The **Exceptions** tab won't properly load if exceptions contain comments with newline characters (`\n`)
[%collapsible]
====
*Details* +
On December 5, 2024, it was discovered that the **Exceptions** tab won't load properly if any exceptions contain comments with newline characters (`\n`). This issue occurs when you upgrade to 8.16.0 or later.
nastasha-solomon marked this conversation as resolved.
Show resolved Hide resolved

*Workaround* +

For custom rules:

. From the **Rules** page, <<import-export-rules-ui,export>> the rule or rules with the affected exception lists.
. Modify the `.ndjson` file so `comments` no longer contain newline characters.
. Return to the **Rules** page and <<import-export-rules-ui,re-import>> the rules. Make sure to select the **Overwrite existing exception lists with conflicting "list_id"** option.

For prebuilt rules:

NOTE: If you only need to fix Endpoint exceptions for the Elastic Endpoint rule, follow the above instructions for fixing custom rule exceptions.

. Follow these steps to fetch the affected exception list ID or IDs that are associated with the rule:
.. Find the affected rule's ID (`id`). From the **Rules** page, open the details of a rule, go to the page URL, and copy the string at the end. For example, in the URL http://host.name/app/security/rules/id/167a5f6f-2148-4792-8226-b5e7a58ef46e, the string at the end (`167a5f6f-2148-4792-8226-b5e7a58ef46e`) is the `id`.
.. Specify the `id` when fetching the rule's details using the {api-kibana}/operation/operation-readrule[Retrieve a detection rule API]. Here is an example request that includes the `id`:
+
[source,console]
----
curl -H 'Authorization: ApiKey API_KEY_HERE' -H 'kbn-xsrf: true' -H 'elastic-api-version: 2023-10-31' KIBANA_URL/api/detection_engine/rules?id=167a5f6f-2148-4792-8226-b5e7a58ef46e
----
+
.. The JSON response contains the `id`, `list_id`, and `namespace_type` values within the `exceptions_list` key (as shown below). You need these values when using the Exception list API to retrieve the affected exception list.
+
[source,console]
----
{
"id": "167a5f6f-2148-4792-8226-b5e7a58ef46e",
"exceptions_list": [
{
"id": "490525a2-eb66-4320-95b5-88bdd1302dc4",
"list_id": "f75aae6f-0229-413f-881d-81cb3abfbe2d",
"namespace_type": "single"
}
]
}
----
+
. Use the export exceptions API to retrieve the affected exception list. Insert the values for the `id`, `list_id`, and `namespace_type` parameters into the following API call:
+
[source,console]
----
curl -XPOST -u elastic:changeme -H 'kbn-xsrf: true' -H 'elastic-api-version: 2023-10-31' 'http://localhost:5601/api/exception_lists/_export?list_id=f75aae6f-0229-413f-881d-81cb3abfbe2d&id=490525a2-eb66-4320-95b5-88bdd1302dc4&namespace_type=single' -o list.ndjson
----
+
. Modify the exception list's `.ndjson` file to ensure `comments[].comment` values don't contain newline characters (`\n`).
. Re-import the modified exception list using **Import exception lists** option on the **Shared Exception Lists** page (find the page in the navigation menu or by using the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]).
+
Note that the import will initially fail because the exception list already exists. After the initial failure, an option to overwrite the existing list will appear. Select the option, then resubmit the request to import the corrected exception list.
====
// end::known-issue[201820]

// tag::known-issue[]
[discrete]
.Attempting to edit an Elastic AI Assistant Knowledge Base index results in an error
Expand Down
Loading