From 47bf63e264be4b8c4f4a7a81a432ac73841b0c44 Mon Sep 17 00:00:00 2001 From: natasha-moore-elastic Date: Thu, 16 Nov 2023 15:56:06 +0000 Subject: [PATCH 01/14] Adds new Advanced Behavioral Detections section --- .../advanced-behavioral-detections.asciidoc | 2 ++ ...dvanced-entity-analytics-overview.asciidoc | 26 ++++++------------ .../analyze-risk-score-data.asciidoc | 14 +++++----- .../behavioral-detection-use-cases.asciidoc | 2 ++ .../entity-risk-scoring.asciidoc | 15 ++++++++++ .../images/cloned-job-details.png | Bin .../images/filter-add-item.png | Bin .../images/ml-rule-threshold.png | Bin .../images/ml-ui.png | Bin .../images/rule-scope.png | Bin .../images/rules-table-error-icon.png | Bin .../images/rules-table-ml-job-error.png | Bin .../images/start-job-window.png | Bin .../machine-learning.asciidoc | 6 +--- .../prebuilt-ml-jobs.asciidoc | 4 +++ .../tune-anomaly-results.asciidoc | 12 ++++---- .../turn-on-risk-engine.asciidoc | 8 +++--- docs/detections/detections-index.asciidoc | 4 --- 18 files changed, 49 insertions(+), 44 deletions(-) create mode 100644 docs/advanced-entity-analytics/advanced-behavioral-detections.asciidoc create mode 100644 docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc create mode 100644 docs/advanced-entity-analytics/entity-risk-scoring.asciidoc rename docs/{detections/machine-learning => advanced-entity-analytics}/images/cloned-job-details.png (100%) rename docs/{detections/machine-learning => advanced-entity-analytics}/images/filter-add-item.png (100%) rename docs/{detections/machine-learning => advanced-entity-analytics}/images/ml-rule-threshold.png (100%) rename docs/{detections/machine-learning => advanced-entity-analytics}/images/ml-ui.png (100%) rename docs/{detections/machine-learning => advanced-entity-analytics}/images/rule-scope.png (100%) rename docs/{detections/machine-learning => advanced-entity-analytics}/images/rules-table-error-icon.png (100%) rename docs/{detections/machine-learning => advanced-entity-analytics}/images/rules-table-ml-job-error.png (100%) rename docs/{detections/machine-learning => advanced-entity-analytics}/images/start-job-window.png (100%) rename docs/{detections/machine-learning => advanced-entity-analytics}/machine-learning.asciidoc (95%) create mode 100644 docs/advanced-entity-analytics/prebuilt-ml-jobs.asciidoc rename docs/{detections/machine-learning => advanced-entity-analytics}/tune-anomaly-results.asciidoc (95%) diff --git a/docs/advanced-entity-analytics/advanced-behavioral-detections.asciidoc b/docs/advanced-entity-analytics/advanced-behavioral-detections.asciidoc new file mode 100644 index 0000000000..eff703ef05 --- /dev/null +++ b/docs/advanced-entity-analytics/advanced-behavioral-detections.asciidoc @@ -0,0 +1,2 @@ +[[advanced-behavioral-detections]] += Advanced Behavioral Detections \ No newline at end of file diff --git a/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc b/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc index 6ab4c4ba2b..7c16f6cec5 100644 --- a/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc +++ b/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc @@ -3,21 +3,11 @@ Advanced Entity Analytics generates a set of threat detection and risk analytics that allows you to expedite alert triage and hunt for new threats from within an entity's environment. This feature combines the power of the SIEM detection engine and Elastic's {ml} capabilities to identify unusual user behaviors and generate comprehensive risk analytics for hosts and users. -[discrete] -[[entity-risk-scoring]] -== Entity Risk Scoring - -beta::[] - -Entity Risk Scoring is an advanced {elastic-sec} analytics feature that helps security analysts detect changes in an entity's risk posture, hunt for new threats, and prioritize incident response. - -Entity Risk Scoring allows you to monitor the change in the risk posture of hosts and users from your environment. The risk scoring engine generates these advanced scoring analytics by factoring threats from its end-to-end XDR use cases, such as SIEM, cloud, and endpoint. - -The next-generation risk scoring engine provides greater scalability and performance. It leverages the Elastic SIEM detection engine to generate host and user risk scores from the last 30 days. - -It also generates risk scores on a recurring interval, and allows for easy onboarding and management. The engine is built to factor in risks from all {elastic-sec} use cases, and allows you to customize and control how and when risk is calculated. - -Learn how to <>. - -include::turn-on-risk-engine.asciidoc[] -include::analyze-risk-score-data.asciidoc[] \ No newline at end of file +include::entity-risk-scoring.asciidoc[leveloffset=+1] +include::turn-on-risk-engine.asciidoc[leveloffset=+2] +include::analyze-risk-score-data.asciidoc[leveloffset=+2] +include::advanced-behavioral-detections.asciidoc[leveloffset=+1] +include::machine-learning.asciidoc[leveloffset=+2] +include::prebuilt-ml-jobs.asciidoc[leveloffset=+3] +include::tune-anomaly-results.asciidoc[leveloffset=+3] +include::behavioral-detection-use-cases.asciidoc[leveloffset=+2] \ No newline at end of file diff --git a/docs/advanced-entity-analytics/analyze-risk-score-data.asciidoc b/docs/advanced-entity-analytics/analyze-risk-score-data.asciidoc index 440ab5d164..d4570835ca 100644 --- a/docs/advanced-entity-analytics/analyze-risk-score-data.asciidoc +++ b/docs/advanced-entity-analytics/analyze-risk-score-data.asciidoc @@ -1,5 +1,5 @@ [[analyze-risk-score-data]] -== View and analyze risk score data += View and analyze risk score data The {security-app} provides several options to monitor the change in the risk posture of hosts and users from your environment. Use the following places in the {security-app} to view and analyze risk score data: @@ -13,7 +13,7 @@ TIP: We recommend that you prioritize <> to iden [discrete] [[entity-analytics-dashboard]] -=== Entity Analytics dashboard +== Entity Analytics dashboard From the Entity Analytics dashboard, you can access entity key performance indicators (KPIs), risk scores, and levels. You can also click the number link in the **Alerts** column to investigate and analyze the alerts on the Alerts page. @@ -22,12 +22,12 @@ image::dashboards/images/entity-dashboard.png[Entity Analytics dashboard] [discrete] [[alert-triaging]] -=== Alert triaging +== Alert triaging You can prioritize alert triaging to analyze alerts associated with risky entities using the following features in the {security-app}. [discrete] [[alerts-page]] -==== Alerts page +=== Alerts page Use the Alerts table to investigate and analyze host and user risk levels and scores. We recommend adding the `user.risk.calculated_level` and `host.risk.calculated_level` columns to the Alerts table to easily display this data. To do this, select **Fields**, search for `user.risk` and `host.risk`, then select the appropriate fields from the list. Learn more about <>. @@ -41,7 +41,7 @@ image::images/filter-by-host-risk-level.png[Alerts filtered by high host risk le [discrete] [[alert-details-flyout]] -==== Alert details flyout +=== Alert details flyout To access risk score data in the alert details flyout, select **Insights** -> **Entities** on the **Overview** tab: @@ -50,7 +50,7 @@ image::images/alerts-flyout-rs.png[Risk scores in the Alerts flyout] [discrete] [[hosts-users-pages]] -==== Hosts and Users pages +=== Hosts and Users pages On the Hosts and Users pages, you can access the risk score data: @@ -66,7 +66,7 @@ image::images/hosts-hr-data.png[Host risk data on the Host risk tab of the Hosts [discrete] [[host-user-details-pages]] -==== Host and user details pages +=== Host and user details pages On the host details and user details pages, you can access the risk score data: diff --git a/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc b/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc new file mode 100644 index 0000000000..c872efba2d --- /dev/null +++ b/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc @@ -0,0 +1,2 @@ +[[behavioral-detection-use-cases]] += Behavioral detection use cases \ No newline at end of file diff --git a/docs/advanced-entity-analytics/entity-risk-scoring.asciidoc b/docs/advanced-entity-analytics/entity-risk-scoring.asciidoc new file mode 100644 index 0000000000..8ddcad2c9f --- /dev/null +++ b/docs/advanced-entity-analytics/entity-risk-scoring.asciidoc @@ -0,0 +1,15 @@ +[[entity-risk-scoring]] += Entity Risk Scoring + +beta::[] + +Entity Risk Scoring is an advanced {elastic-sec} analytics feature that helps security analysts detect changes in an entity's risk posture, hunt for new threats, and prioritize incident response. + +Entity Risk Scoring allows you to monitor the change in the risk posture of hosts and users from your environment. The risk scoring engine generates these advanced scoring analytics by factoring threats from its end-to-end XDR use cases, such as SIEM, cloud, and endpoint. + +The next-generation risk scoring engine provides greater scalability and performance. It leverages the Elastic SIEM detection engine to generate host and user risk scores from the last 30 days. + +It also generates risk scores on a recurring interval, and allows for easy onboarding and management. The engine is built to factor in risks from all {elastic-sec} use cases, and allows you to customize and control how and when risk is calculated. + +Learn how to <>. + diff --git a/docs/detections/machine-learning/images/cloned-job-details.png b/docs/advanced-entity-analytics/images/cloned-job-details.png similarity index 100% rename from docs/detections/machine-learning/images/cloned-job-details.png rename to docs/advanced-entity-analytics/images/cloned-job-details.png diff --git a/docs/detections/machine-learning/images/filter-add-item.png b/docs/advanced-entity-analytics/images/filter-add-item.png similarity index 100% rename from docs/detections/machine-learning/images/filter-add-item.png rename to docs/advanced-entity-analytics/images/filter-add-item.png diff --git a/docs/detections/machine-learning/images/ml-rule-threshold.png b/docs/advanced-entity-analytics/images/ml-rule-threshold.png similarity index 100% rename from docs/detections/machine-learning/images/ml-rule-threshold.png rename to docs/advanced-entity-analytics/images/ml-rule-threshold.png diff --git a/docs/detections/machine-learning/images/ml-ui.png b/docs/advanced-entity-analytics/images/ml-ui.png similarity index 100% rename from docs/detections/machine-learning/images/ml-ui.png rename to docs/advanced-entity-analytics/images/ml-ui.png diff --git a/docs/detections/machine-learning/images/rule-scope.png b/docs/advanced-entity-analytics/images/rule-scope.png similarity index 100% rename from docs/detections/machine-learning/images/rule-scope.png rename to docs/advanced-entity-analytics/images/rule-scope.png diff --git a/docs/detections/machine-learning/images/rules-table-error-icon.png b/docs/advanced-entity-analytics/images/rules-table-error-icon.png similarity index 100% rename from docs/detections/machine-learning/images/rules-table-error-icon.png rename to docs/advanced-entity-analytics/images/rules-table-error-icon.png diff --git a/docs/detections/machine-learning/images/rules-table-ml-job-error.png b/docs/advanced-entity-analytics/images/rules-table-ml-job-error.png similarity index 100% rename from docs/detections/machine-learning/images/rules-table-ml-job-error.png rename to docs/advanced-entity-analytics/images/rules-table-ml-job-error.png diff --git a/docs/detections/machine-learning/images/start-job-window.png b/docs/advanced-entity-analytics/images/start-job-window.png similarity index 100% rename from docs/detections/machine-learning/images/start-job-window.png rename to docs/advanced-entity-analytics/images/start-job-window.png diff --git a/docs/detections/machine-learning/machine-learning.asciidoc b/docs/advanced-entity-analytics/machine-learning.asciidoc similarity index 95% rename from docs/detections/machine-learning/machine-learning.asciidoc rename to docs/advanced-entity-analytics/machine-learning.asciidoc index 5e00ae0201..55059d5243 100644 --- a/docs/detections/machine-learning/machine-learning.asciidoc +++ b/docs/advanced-entity-analytics/machine-learning.asciidoc @@ -73,7 +73,7 @@ Or <> describes all available {ml} jobs and lists which ECS fields are required on your hosts when you are not using {beats} or the {agent} to ship your data. For information on tuning anomaly results to reduce the -number of false positives, see <>. +//number of false positives, see <>. NOTE: Machine learning jobs look back and analyze two weeks of historical data prior to the time they are enabled. After jobs are enabled, they continuously @@ -103,7 +103,3 @@ NOTE: To adjust the `score` threshold that determines which anomalies are shown, you can modify *{kib}* -> *{stack-manage-app}* -> *Advanced Settings* -> *`securitySolution:defaultAnomalyScore`*. -[[prebuilt-ml-jobs]] -== Prebuilt job reference - -include::{stack-docs-root}/docs/en/stack/ml/anomaly-detection/ootb-ml-jobs-siem.asciidoc[tag=siem-jobs] diff --git a/docs/advanced-entity-analytics/prebuilt-ml-jobs.asciidoc b/docs/advanced-entity-analytics/prebuilt-ml-jobs.asciidoc new file mode 100644 index 0000000000..f92d8e6611 --- /dev/null +++ b/docs/advanced-entity-analytics/prebuilt-ml-jobs.asciidoc @@ -0,0 +1,4 @@ +[[prebuilt-ml-jobs]] += Prebuilt job reference + +include::{stack-docs-root}/docs/en/stack/ml/anomaly-detection/ootb-ml-jobs-siem.asciidoc[tag=siem-jobs] \ No newline at end of file diff --git a/docs/detections/machine-learning/tune-anomaly-results.asciidoc b/docs/advanced-entity-analytics/tune-anomaly-results.asciidoc similarity index 95% rename from docs/detections/machine-learning/tune-anomaly-results.asciidoc rename to docs/advanced-entity-analytics/tune-anomaly-results.asciidoc index b922f8ef16..1c0d64a399 100644 --- a/docs/detections/machine-learning/tune-anomaly-results.asciidoc +++ b/docs/advanced-entity-analytics/tune-anomaly-results.asciidoc @@ -1,5 +1,5 @@ [[tuning-anomaly-results]] -== Optimizing anomaly results += Optimizing anomaly results To gain clearer insights into real threats, you can tune the anomaly results. The following procedures help to reduce the number of false positives: @@ -8,7 +8,7 @@ To gain clearer insights into real threats, you can tune the anomaly results. Th [float] [[rarely-used-processes]] -=== Filter out anomalies from rarely used applications and processes +== Filter out anomalies from rarely used applications and processes When anomalies include results from a known process that only runs occasionally, you can filter out the unwanted results. @@ -22,7 +22,7 @@ For example, to filter out results from a housekeeping process, named [float] [[create-fiter-list]] -==== Create a filter list +=== Create a filter list . Go to *Machine Learning* -> *Anomaly Detection* -> *Settings*. . Click *Filter Lists* and then *New*. @@ -42,7 +42,7 @@ The new filter appears in the Filter List and can be added to relevant jobs. [float] [[add-job-filter]] -==== Add the filter to the relevant job +=== Add the filter to the relevant job . Go to *Machine Learning* -> *Anomaly Detection* -> *Anomaly Explorer*. . Navigate to the job results for which the filter is required. If the job results @@ -70,7 +70,7 @@ before the filter was added are still displayed. [float] [[clone-job]] -==== Clone and rerun the job +=== Clone and rerun the job If you want to remove all the previously detected results for the process, you must clone and run the cloned job. @@ -108,7 +108,7 @@ After a while, results will start to appear on the *Anomaly Explorer* page. [float] [[define-rule-threshold]] -=== Define an anomaly threshold for a job +== Define an anomaly threshold for a job Certain jobs use a high-count function to look for unusual spikes in process events. For some processes, a burst of activity is a normal, such as diff --git a/docs/advanced-entity-analytics/turn-on-risk-engine.asciidoc b/docs/advanced-entity-analytics/turn-on-risk-engine.asciidoc index 5c098ccd16..dc8194f36b 100644 --- a/docs/advanced-entity-analytics/turn-on-risk-engine.asciidoc +++ b/docs/advanced-entity-analytics/turn-on-risk-engine.asciidoc @@ -1,5 +1,5 @@ [[turn-on-risk-engine]] -== Turn on the risk scoring engine += Turn on the risk scoring engine beta[] @@ -20,7 +20,7 @@ The latest risk scoring engine runs hourly to aggregate `Open` and `Acknowledged |============================================== [discrete] -=== Preview risky entities +== Preview risky entities You can preview risky entities before installing the latest risk engine. The preview shows the riskiest hosts and users found in the 1000 sampled entities during the time frame selected in the date picker. @@ -32,7 +32,7 @@ To preview risky entities, go to **Manage** -> **Entity Risk Score**: image::images/preview-risky-entities.png[Preview of risky entities] [discrete] -=== Turn on the latest risk engine +== Turn on the latest risk engine [NOTE] ====== @@ -50,7 +50,7 @@ image::images/turn-on-risk-engine.png[Turn on entity risk scoring] [discrete] [[upgrade-risk-engine]] -=== Upgrade to the latest risk engine +== Upgrade to the latest risk engine If you upgraded to 8.11 from an earlier {stack} version, and you have the original risk engine installed, you can upgrade to the latest risk engine. You will be prompted to upgrade in places where risk score data exists, such as: diff --git a/docs/detections/detections-index.asciidoc b/docs/detections/detections-index.asciidoc index d853ca4df8..557b99198c 100644 --- a/docs/detections/detections-index.asciidoc +++ b/docs/detections/detections-index.asciidoc @@ -1,7 +1,3 @@ -include::machine-learning/machine-learning.asciidoc[] - -include::machine-learning/tune-anomaly-results.asciidoc[] - include::detection-engine-intro.asciidoc[] include::about-rules.asciidoc[] From 6370d464391ff4612e715c800b5a40641414298b Mon Sep 17 00:00:00 2001 From: natasha-moore-elastic Date: Thu, 16 Nov 2023 17:36:03 +0000 Subject: [PATCH 02/14] Moves L4 pages to L3 --- .../advanced-entity-analytics-overview.asciidoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc b/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc index 7c16f6cec5..c8789a9ce0 100644 --- a/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc +++ b/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc @@ -8,6 +8,6 @@ include::turn-on-risk-engine.asciidoc[leveloffset=+2] include::analyze-risk-score-data.asciidoc[leveloffset=+2] include::advanced-behavioral-detections.asciidoc[leveloffset=+1] include::machine-learning.asciidoc[leveloffset=+2] -include::prebuilt-ml-jobs.asciidoc[leveloffset=+3] -include::tune-anomaly-results.asciidoc[leveloffset=+3] +include::prebuilt-ml-jobs.asciidoc[leveloffset=+2] +include::tune-anomaly-results.asciidoc[leveloffset=+2] include::behavioral-detection-use-cases.asciidoc[leveloffset=+2] \ No newline at end of file From 3f5b51e1c7572816a1229957366f607bde383ed1 Mon Sep 17 00:00:00 2001 From: natasha-moore-elastic Date: Tue, 21 Nov 2023 13:20:58 +0000 Subject: [PATCH 03/14] Moves ml-integrations under Behavioral detection use cases --- .../behavioral-detection-use-cases.asciidoc | 17 ++++++++++++++++- .../machine-learning.asciidoc | 15 +-------------- 2 files changed, 17 insertions(+), 15 deletions(-) diff --git a/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc b/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc index c872efba2d..3d8b62402b 100644 --- a/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc +++ b/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc @@ -1,2 +1,17 @@ [[behavioral-detection-use-cases]] -= Behavioral detection use cases \ No newline at end of file += Behavioral detection use cases + +Introduction to be added + +[float] +[[ml-integrations]] +=== Jobs in Advanced Analytics (UEBA) Elastic integrations + +You can also install {ml} jobs using https://docs.elastic.co/integrations[Elastic integrations]. Here are the Advanced Analytics integrations available for Security: + +* https://docs.elastic.co/integrations/ded[Data Exfiltration Detection] +* https://docs.elastic.co/integrations/dga[Domain Generation Algorithm Detection] +* https://docs.elastic.co/integrations/lmd[Lateral Movement Detection] +* https://docs.elastic.co/integrations/problemchild[Living off the Land Attack Detection] + +To learn more about {ml} jobs enabled by these integrations, refer to the https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html[Prebuilt jobs page]. diff --git a/docs/advanced-entity-analytics/machine-learning.asciidoc b/docs/advanced-entity-analytics/machine-learning.asciidoc index 55059d5243..199bab0432 100644 --- a/docs/advanced-entity-analytics/machine-learning.asciidoc +++ b/docs/advanced-entity-analytics/machine-learning.asciidoc @@ -1,6 +1,6 @@ [[machine-learning]] [role="xpack"] -= Anomaly detection with {ml} += Anomaly detection :frontmatter-description: Use the power of machine learning to detect outliers and suspicious events. :frontmatter-tags-products: [security] @@ -80,19 +80,6 @@ prior to the time they are enabled. After jobs are enabled, they continuously analyze incoming data. When jobs are stopped and restarted within the two-week time frame, previously analyzed data is not processed again. -[float] -[[ml-integrations]] -=== Jobs in Advanced Analytics (UEBA) Elastic integrations - -You can also install {ml} jobs using https://docs.elastic.co/integrations[Elastic integrations]. Here are the Advanced Analytics integrations available for Security: - -* https://docs.elastic.co/integrations/ded[Data Exfiltration Detection] -* https://docs.elastic.co/integrations/dga[Domain Generation Algorithm Detection] -* https://docs.elastic.co/integrations/lmd[Lateral Movement Detection] -* https://docs.elastic.co/integrations/problemchild[Living off the Land Attack Detection] - -To learn more about {ml} jobs enabled by these integrations, refer to the https://www.elastic.co/guide/en/security/current/prebuilt-ml-jobs.html[Prebuilt jobs page]. - [float] [[view-anomalies]] == View detected anomalies From b3fc7a38b1c3ec53fff854de2f6784bcb5ae9dc1 Mon Sep 17 00:00:00 2001 From: natasha-moore-elastic Date: Tue, 21 Nov 2023 13:31:45 +0000 Subject: [PATCH 04/14] Adds Advanced Behavioral Detections intro section --- .../advanced-behavioral-detections.asciidoc | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/docs/advanced-entity-analytics/advanced-behavioral-detections.asciidoc b/docs/advanced-entity-analytics/advanced-behavioral-detections.asciidoc index eff703ef05..dacc12a8aa 100644 --- a/docs/advanced-entity-analytics/advanced-behavioral-detections.asciidoc +++ b/docs/advanced-entity-analytics/advanced-behavioral-detections.asciidoc @@ -1,2 +1,9 @@ [[advanced-behavioral-detections]] -= Advanced Behavioral Detections \ No newline at end of file += Advanced Behavioral Detections + +Use Elastic's {ml} capabilities and advanced correlation, scoring, and visualization techniques to identify behavioral threats that are most likely to be involved in security incidents in your environment. + +Advanced Behavioral Detections provide two key capabilities: + +* <> +* <> From 1e4c255abb005c89eaef2dfbb2c1227d5580c09c Mon Sep 17 00:00:00 2001 From: natasha-moore-elastic Date: Tue, 21 Nov 2023 13:55:37 +0000 Subject: [PATCH 05/14] Adds links to AEA page --- .../advanced-entity-analytics-overview.asciidoc | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc b/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc index c8789a9ce0..b4fa061e79 100644 --- a/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc +++ b/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc @@ -3,6 +3,11 @@ Advanced Entity Analytics generates a set of threat detection and risk analytics that allows you to expedite alert triage and hunt for new threats from within an entity's environment. This feature combines the power of the SIEM detection engine and Elastic's {ml} capabilities to identify unusual user behaviors and generate comprehensive risk analytics for hosts and users. +Advanced Entity Analytics provides two key capabilities: + +* <> +* <> + include::entity-risk-scoring.asciidoc[leveloffset=+1] include::turn-on-risk-engine.asciidoc[leveloffset=+2] include::analyze-risk-score-data.asciidoc[leveloffset=+2] From ce012064055f88d6cc4aa0750eaeb9b648da0a1b Mon Sep 17 00:00:00 2001 From: natasha-moore-elastic Date: Tue, 21 Nov 2023 14:00:47 +0000 Subject: [PATCH 06/14] Uncomments reference that previously broke the build for no reason --- docs/advanced-entity-analytics/machine-learning.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/advanced-entity-analytics/machine-learning.asciidoc b/docs/advanced-entity-analytics/machine-learning.asciidoc index 199bab0432..6b4ae308d9 100644 --- a/docs/advanced-entity-analytics/machine-learning.asciidoc +++ b/docs/advanced-entity-analytics/machine-learning.asciidoc @@ -73,7 +73,7 @@ Or <> describes all available {ml} jobs and lists which ECS fields are required on your hosts when you are not using {beats} or the {agent} to ship your data. For information on tuning anomaly results to reduce the -//number of false positives, see <>. +number of false positives, see <>. NOTE: Machine learning jobs look back and analyze two weeks of historical data prior to the time they are enabled. After jobs are enabled, they continuously From 48982a002e12ddc99a3670b3800605e6d5ca561c Mon Sep 17 00:00:00 2001 From: natasha-moore-elastic Date: Tue, 21 Nov 2023 14:20:20 +0000 Subject: [PATCH 07/14] Replaces verbal reference to ml-integrations --- .../behavioral-detection-use-cases.asciidoc | 2 +- docs/advanced-entity-analytics/machine-learning.asciidoc | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc b/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc index 2bfde3aaaa..1f16c59ddf 100644 --- a/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc +++ b/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc @@ -7,7 +7,7 @@ Introduction to be added [[ml-integrations]] === Jobs in Advanced Analytics (UEBA) Elastic integrations -You can also install {ml} jobs using https://docs.elastic.co/integrations[Elastic integrations]. Here are the Advanced Analytics integrations available for Security: +You can install {ml} jobs using https://docs.elastic.co/integrations[Elastic integrations]. Here are the Advanced Analytics integrations available for Security: * {integrations-docs}/ded[Data Exfiltration Detection] * {integrations-docs}/dga[Domain Generation Algorithm Detection] diff --git a/docs/advanced-entity-analytics/machine-learning.asciidoc b/docs/advanced-entity-analytics/machine-learning.asciidoc index 6b4ae308d9..c43fe0c870 100644 --- a/docs/advanced-entity-analytics/machine-learning.asciidoc +++ b/docs/advanced-entity-analytics/machine-learning.asciidoc @@ -68,7 +68,7 @@ data's index patterns in *{kib}* -> *{stack-manage-app}* -> *Data Views*. Or -* You install one or more of the Advanced Analytics integrations (refer to the following section). +* You install one or more of the <>. <> describes all available {ml} jobs and lists which ECS fields are required on your hosts when you are not using {beats} or the {agent} From 6f9953a251179c6b22b6d5990d99978c88889fec Mon Sep 17 00:00:00 2001 From: natasha-moore-elastic Date: Tue, 21 Nov 2023 16:20:37 +0000 Subject: [PATCH 08/14] Removes frontmatter --- docs/advanced-entity-analytics/machine-learning.asciidoc | 5 ----- 1 file changed, 5 deletions(-) diff --git a/docs/advanced-entity-analytics/machine-learning.asciidoc b/docs/advanced-entity-analytics/machine-learning.asciidoc index c43fe0c870..7a3cb11fb1 100644 --- a/docs/advanced-entity-analytics/machine-learning.asciidoc +++ b/docs/advanced-entity-analytics/machine-learning.asciidoc @@ -2,11 +2,6 @@ [role="xpack"] = Anomaly detection -:frontmatter-description: Use the power of machine learning to detect outliers and suspicious events. -:frontmatter-tags-products: [security] -:frontmatter-tags-content-type: [overview] -:frontmatter-tags-user-goals: [manage] - {ml-docs}/ml-ad-overview.html[{ml-cap}] functionality is available when you have the appropriate subscription, are using a *{ess-trial}[cloud deployment]*, or are testing out a *Free Trial*. Refer to <> for more information. From e8ad0e386fbe8279e4c9616727a6130c2cd0503a Mon Sep 17 00:00:00 2001 From: natasha-moore-elastic Date: Wed, 29 Nov 2023 13:04:57 +0000 Subject: [PATCH 09/14] Behavioral detection updates --- .../advanced-entity-analytics-overview.asciidoc | 4 ++-- .../behavioral-detection-use-cases.asciidoc | 17 ++++++++++++++--- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc b/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc index b4fa061e79..4fa2c5b7ab 100644 --- a/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc +++ b/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc @@ -13,6 +13,6 @@ include::turn-on-risk-engine.asciidoc[leveloffset=+2] include::analyze-risk-score-data.asciidoc[leveloffset=+2] include::advanced-behavioral-detections.asciidoc[leveloffset=+1] include::machine-learning.asciidoc[leveloffset=+2] -include::prebuilt-ml-jobs.asciidoc[leveloffset=+2] include::tune-anomaly-results.asciidoc[leveloffset=+2] -include::behavioral-detection-use-cases.asciidoc[leveloffset=+2] \ No newline at end of file +include::behavioral-detection-use-cases.asciidoc[leveloffset=+2] +include::prebuilt-ml-jobs.asciidoc[leveloffset=+2] diff --git a/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc b/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc index 1f16c59ddf..834ee71623 100644 --- a/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc +++ b/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc @@ -1,13 +1,24 @@ [[behavioral-detection-use-cases]] = Behavioral detection use cases -Introduction to be added +Behavioral use cases detect potential insider and external threats based on observed user actions and host activities. These use cases employ a threat-centric approach to flag deviations or suspicious activities by analyzing patterns, anomalies, and context enrichment. + +Elastic builds behavioral detection use cases on its foundational SIEM detection capabilities, leveraging {ml} algorithms and AI to enable proactive threat detection and hunting. [float] [[ml-integrations]] -=== Jobs in Advanced Analytics (UEBA) Elastic integrations +=== Elastic {integrations} for behavioral detection use cases + +Elastic {integrations-docs}[{integrations}] provide a convenient way to enable behavioral detection use case packages. These integrations streamline the deployment of package components, such as data ingestion, transforms, rules, {ml} jobs, and scripts to your environment. + +.Requirements +[sidebar] +-- +* Elastic {integrations} require a https://www.elastic.co/pricing[Platinum subscription] or higher. +* To learn more about the requirements for using {ml} jobs, refer to <>. +-- -You can install {ml} jobs using https://docs.elastic.co/integrations[Elastic integrations]. Here are the Advanced Analytics integrations available for Security: +The following is a list of behavioral detection use cases and their integrations: * {integrations-docs}/ded[Data Exfiltration Detection] * {integrations-docs}/dga[Domain Generation Algorithm Detection] From a9f6d0bcbd22bc0e4a705e30a944ae0cb08cd850 Mon Sep 17 00:00:00 2001 From: natasha-moore-elastic <137783811+natasha-moore-elastic@users.noreply.github.com> Date: Thu, 30 Nov 2023 11:10:55 +0000 Subject: [PATCH 10/14] Apply suggestions from code review Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com> --- .../advanced-behavioral-detections.asciidoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/advanced-entity-analytics/advanced-behavioral-detections.asciidoc b/docs/advanced-entity-analytics/advanced-behavioral-detections.asciidoc index dacc12a8aa..e62cb8ce8f 100644 --- a/docs/advanced-entity-analytics/advanced-behavioral-detections.asciidoc +++ b/docs/advanced-entity-analytics/advanced-behavioral-detections.asciidoc @@ -1,9 +1,9 @@ [[advanced-behavioral-detections]] = Advanced Behavioral Detections -Use Elastic's {ml} capabilities and advanced correlation, scoring, and visualization techniques to identify behavioral threats that are most likely to be involved in security incidents in your environment. +Elastic's {ml} capabilities and advanced correlation, scoring, and visualization techniques can help you identify potential behavioral threats that may be associated with security incidents. -Advanced Behavioral Detections provide two key capabilities: +Advanced Behavioral Detections includes two key capabilities: * <> * <> From 50da8f5437ca3f9d084925f002791b49beac6063 Mon Sep 17 00:00:00 2001 From: natasha-moore-elastic Date: Thu, 30 Nov 2023 11:16:01 +0000 Subject: [PATCH 11/14] Apply suggestion from TW review --- .../behavioral-detection-use-cases.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc b/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc index 834ee71623..0fc3ed9cc0 100644 --- a/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc +++ b/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc @@ -18,7 +18,7 @@ Elastic {integrations-docs}[{integrations}] provide a convenient way to enable b * To learn more about the requirements for using {ml} jobs, refer to <>. -- -The following is a list of behavioral detection use cases and their integrations: +The following is a list of integrations for various behavioral detection use cases: * {integrations-docs}/ded[Data Exfiltration Detection] * {integrations-docs}/dga[Domain Generation Algorithm Detection] From d778f494573b70bfd6c4fcff00bf17eb2fe5314c Mon Sep 17 00:00:00 2001 From: natasha-moore-elastic Date: Mon, 4 Dec 2023 11:56:41 +0000 Subject: [PATCH 12/14] Applies review feedback --- .../behavioral-detection-use-cases.asciidoc | 10 +++++----- .../entity-risk-scoring.asciidoc | 4 +--- 2 files changed, 6 insertions(+), 8 deletions(-) diff --git a/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc b/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc index 0fc3ed9cc0..d11b02a114 100644 --- a/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc +++ b/docs/advanced-entity-analytics/behavioral-detection-use-cases.asciidoc @@ -1,24 +1,24 @@ [[behavioral-detection-use-cases]] = Behavioral detection use cases -Behavioral use cases detect potential insider and external threats based on observed user actions and host activities. These use cases employ a threat-centric approach to flag deviations or suspicious activities by analyzing patterns, anomalies, and context enrichment. +Behavioral detection identifies potential internal and external threats based on user and host activity. It employs a threat-centric approach to flag suspicious activity by analyzing patterns, anomalies, and context enrichment. -Elastic builds behavioral detection use cases on its foundational SIEM detection capabilities, leveraging {ml} algorithms and AI to enable proactive threat detection and hunting. +{elastic-sec} builds the behavioral detection feature on its foundational SIEM detection capabilities, leveraging {ml} algorithms to enable proactive threat detection and hunting. [float] [[ml-integrations]] === Elastic {integrations} for behavioral detection use cases -Elastic {integrations-docs}[{integrations}] provide a convenient way to enable behavioral detection use case packages. These integrations streamline the deployment of package components, such as data ingestion, transforms, rules, {ml} jobs, and scripts to your environment. +Behavioral detection integrations provide a convenient way to enable behavioral detection capabilities. They streamline the deployment of components that implement behavioral detection, such as data ingestion, transforms, rules, {ml} jobs, and scripts. .Requirements [sidebar] -- -* Elastic {integrations} require a https://www.elastic.co/pricing[Platinum subscription] or higher. +* Elastic integrations require a https://www.elastic.co/pricing[Platinum subscription] or higher. * To learn more about the requirements for using {ml} jobs, refer to <>. -- -The following is a list of integrations for various behavioral detection use cases: +Here's a list of integrations for various behavioral detection use cases: * {integrations-docs}/ded[Data Exfiltration Detection] * {integrations-docs}/dga[Domain Generation Algorithm Detection] diff --git a/docs/advanced-entity-analytics/entity-risk-scoring.asciidoc b/docs/advanced-entity-analytics/entity-risk-scoring.asciidoc index 8ddcad2c9f..ebf23e17f6 100644 --- a/docs/advanced-entity-analytics/entity-risk-scoring.asciidoc +++ b/docs/advanced-entity-analytics/entity-risk-scoring.asciidoc @@ -5,9 +5,7 @@ beta::[] Entity Risk Scoring is an advanced {elastic-sec} analytics feature that helps security analysts detect changes in an entity's risk posture, hunt for new threats, and prioritize incident response. -Entity Risk Scoring allows you to monitor the change in the risk posture of hosts and users from your environment. The risk scoring engine generates these advanced scoring analytics by factoring threats from its end-to-end XDR use cases, such as SIEM, cloud, and endpoint. - -The next-generation risk scoring engine provides greater scalability and performance. It leverages the Elastic SIEM detection engine to generate host and user risk scores from the last 30 days. +Entity Risk Scoring allows you to monitor risk score changes of hosts and users in your environment. When generating advanced scoring analytics, the risk scoring engine utilizes threats from its end-to-end XDR use cases, such as SIEM, cloud, and endpoint. It leverages the Elastic SIEM detection engine to generate host and user risk scores from the last 30 days. It also generates risk scores on a recurring interval, and allows for easy onboarding and management. The engine is built to factor in risks from all {elastic-sec} use cases, and allows you to customize and control how and when risk is calculated. From 23ca78ce7b8829b7315e4d4ac5395782f6a55f43 Mon Sep 17 00:00:00 2001 From: natasha-moore-elastic Date: Mon, 4 Dec 2023 12:05:02 +0000 Subject: [PATCH 13/14] Lowercase advanced behavioral detections --- .../advanced-behavioral-detections.asciidoc | 4 ++-- .../advanced-entity-analytics-overview.asciidoc | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/advanced-entity-analytics/advanced-behavioral-detections.asciidoc b/docs/advanced-entity-analytics/advanced-behavioral-detections.asciidoc index e62cb8ce8f..eb17be7fde 100644 --- a/docs/advanced-entity-analytics/advanced-behavioral-detections.asciidoc +++ b/docs/advanced-entity-analytics/advanced-behavioral-detections.asciidoc @@ -1,9 +1,9 @@ [[advanced-behavioral-detections]] -= Advanced Behavioral Detections += Advanced behavioral detections Elastic's {ml} capabilities and advanced correlation, scoring, and visualization techniques can help you identify potential behavioral threats that may be associated with security incidents. -Advanced Behavioral Detections includes two key capabilities: +Advanced behavioral detections includes two key capabilities: * <> * <> diff --git a/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc b/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc index 4fa2c5b7ab..03eab0a51b 100644 --- a/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc +++ b/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc @@ -6,7 +6,7 @@ Advanced Entity Analytics generates a set of threat detection and risk analytics Advanced Entity Analytics provides two key capabilities: * <> -* <> +* <> include::entity-risk-scoring.asciidoc[leveloffset=+1] include::turn-on-risk-engine.asciidoc[leveloffset=+2] From 1b59a356dfd1a82b393d7f8b592c8f2e35b1a7d3 Mon Sep 17 00:00:00 2001 From: natasha-moore-elastic Date: Mon, 4 Dec 2023 19:13:43 +0000 Subject: [PATCH 14/14] Lowercase entity risk scoring --- .../advanced-entity-analytics-overview.asciidoc | 2 +- docs/advanced-entity-analytics/entity-risk-scoring.asciidoc | 6 +++--- docs/advanced-entity-analytics/turn-on-risk-engine.asciidoc | 2 +- docs/getting-started/ers-req.asciidoc | 6 +++--- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc b/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc index 03eab0a51b..2a035c4a2a 100644 --- a/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc +++ b/docs/advanced-entity-analytics/advanced-entity-analytics-overview.asciidoc @@ -5,7 +5,7 @@ Advanced Entity Analytics generates a set of threat detection and risk analytics Advanced Entity Analytics provides two key capabilities: -* <> +* <> * <> include::entity-risk-scoring.asciidoc[leveloffset=+1] diff --git a/docs/advanced-entity-analytics/entity-risk-scoring.asciidoc b/docs/advanced-entity-analytics/entity-risk-scoring.asciidoc index ebf23e17f6..bbfc7ea1a0 100644 --- a/docs/advanced-entity-analytics/entity-risk-scoring.asciidoc +++ b/docs/advanced-entity-analytics/entity-risk-scoring.asciidoc @@ -1,11 +1,11 @@ [[entity-risk-scoring]] -= Entity Risk Scoring += Entity risk scoring beta::[] -Entity Risk Scoring is an advanced {elastic-sec} analytics feature that helps security analysts detect changes in an entity's risk posture, hunt for new threats, and prioritize incident response. +Entity risk scoring is an advanced {elastic-sec} analytics feature that helps security analysts detect changes in an entity's risk posture, hunt for new threats, and prioritize incident response. -Entity Risk Scoring allows you to monitor risk score changes of hosts and users in your environment. When generating advanced scoring analytics, the risk scoring engine utilizes threats from its end-to-end XDR use cases, such as SIEM, cloud, and endpoint. It leverages the Elastic SIEM detection engine to generate host and user risk scores from the last 30 days. +Entity risk scoring allows you to monitor risk score changes of hosts and users in your environment. When generating advanced scoring analytics, the risk scoring engine utilizes threats from its end-to-end XDR use cases, such as SIEM, cloud, and endpoint. It leverages the Elastic SIEM detection engine to generate host and user risk scores from the last 30 days. It also generates risk scores on a recurring interval, and allows for easy onboarding and management. The engine is built to factor in risks from all {elastic-sec} use cases, and allows you to customize and control how and when risk is calculated. diff --git a/docs/advanced-entity-analytics/turn-on-risk-engine.asciidoc b/docs/advanced-entity-analytics/turn-on-risk-engine.asciidoc index dc8194f36b..c90f984d8f 100644 --- a/docs/advanced-entity-analytics/turn-on-risk-engine.asciidoc +++ b/docs/advanced-entity-analytics/turn-on-risk-engine.asciidoc @@ -3,7 +3,7 @@ beta[] -IMPORTANT: To use Entity Risk Scoring, your role must have the appropriate privileges. For more information, refer to <>. +IMPORTANT: To use entity risk scoring, your role must have the appropriate privileges. For more information, refer to <>. The latest risk scoring engine runs hourly to aggregate `Open` and `Acknowledged` <> from the last 30 days, and assigns risk score to the host or user. It then aggregates the individual risk scores and normalizes them to a 0-100 range. The engine assigns a risk level by mapping the normalized risk score to one of these levels: diff --git a/docs/getting-started/ers-req.asciidoc b/docs/getting-started/ers-req.asciidoc index 5a4092a6d0..62bd992da0 100644 --- a/docs/getting-started/ers-req.asciidoc +++ b/docs/getting-started/ers-req.asciidoc @@ -1,9 +1,9 @@ [[ers-requirements]] -= Entity Risk Scoring prerequisites += Entity risk scoring prerequisites -To use Entity Risk Scoring, your role must have certain cluster, index, and {kib} privileges. This feature requires a https://www.elastic.co/pricing[Platinum subscription] or higher. +To use entity risk scoring, your role must have certain cluster, index, and {kib} privileges. This feature requires a https://www.elastic.co/pricing[Platinum subscription] or higher. -This page covers the requirements and guidelines for using the Entity Risk Scoring feature, as well as its known limitations. +This page covers the requirements and guidelines for using the entity risk scoring feature, as well as its known limitations. [discrete] == Privileges