Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RFC] Risk Score Extensions - Stage 2 #2276

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open

[RFC] Risk Score Extensions - Stage 2 #2276

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
3ba9f6a
Stage 2 Risk Extensions RFC
rylnd Sep 19, 2023
65447a6
Update rfcs/text/0042-risk-score-extensions.md
rylnd Sep 28, 2023
f2024e6
Merge branch 'main' into risk_score_extension_stage_2
rylnd Sep 28, 2023
6460bab
Merge branch 'main' into risk_score_extension_stage_2
ebeahan Oct 13, 2023
6fc0186
Replace Alert document with Risk Score document
rylnd Oct 26, 2023
9cd969d
Add a more realistic category_1_count value
rylnd Dec 8, 2023
323ed90
Add asset criticality fields
rylnd Dec 8, 2023
54c42b5
More clearly state that the category score is normalized
rylnd Dec 8, 2023
3a21061
Add more realistic fields/values for our score document
rylnd Dec 8, 2023
a00f454
Fix allowed values for criticality_level
rylnd Dec 8, 2023
7af698b
Swap Risk Categories 2 and 4
rylnd Dec 19, 2023
cd6e17c
Revert "Add asset criticality fields"
rylnd Jan 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 59 additions & 23 deletions rfcs/text/0042-risk-score-extensions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,10 +65,6 @@ Beyond the per-category explanations, these fields' purpose is to provide more i
* `notes`
* Miscellaneous text field intended to provide more details that cannot be presented in the other fields.

<!--
Stage 1: Describe at a high level how this change affects fields. Include new or updated yml field definitions for all of the essential fields in this draft. While not exhaustive, the fields documented here should be comprehensive enough to deeply evaluate the technical considerations of this change. The goal here is to validate the technical details for all essential fields and to provide a basis for adding experimental field definitions to the schema. Use GitHub code blocks with yml syntax formatting, and add them to the corresponding RFC folder.
-->

<!--
Stage 2: Add or update all remaining field definitions. The list should now be exhaustive. The goal here is to validate the technical details of all remaining fields and to provide a basis for releasing these field definitions as beta in the schema. Use GitHub code blocks with yml syntax formatting, and add them to the corresponding RFC folder.
-->
Expand All @@ -81,40 +77,79 @@ We intend to leverage these new fields as part of the new implementation of the

The new Risk Engine will initially use Detection Engine Alerts as inputs to its scoring mechanism. However, we intend also to allow ingestion from the other Risk Categories described here, provided that they conform to the appropriate schema. Said schema is outside of the scope of this RFC, but based on the current implementation all we will need are a `score` field and a `category` field in order to ingest any arbitrary document.

<!--
Stage 2: Included a real world example source document. Ideally this example comes from the source(s) identified in stage 1. If not, it should replace them. The goal here is to validate the utility of these field changes in the context of a real world example. Format with the source name as a ### header and the example document in a GitHub code block with json formatting, or if on the larger side, add them to the corresponding RFC folder.
-->
### Risk Score Document
The following is an example risk score generated from Detection Engine Alerts, corresponding to the entity `host.name: 'siem-kibana'`

```json
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not seeing the fields from https://github.com/elastic/ecs/blob/main/rfcs/text/0042/risk.yml included in the example alert included. Are those fields still relevant?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably my misunderstanding; I didn't quite understand what "source document" meant in this context, so this is an alert document from which a risk score document would be derived. Should this instead be a risk score document?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea is to capture one or more real-world examples of how these fields are used, ideally like you'd see in the _source field of a ES document.

Should this instead be a risk score document?

I'm not familiar with what's in a risk score document. If the risk score doc provides examples using the risk.category_*_score and risk.category_*_count fields as proposed, yes, I think that's helpful.

"id": "a4cf452c1e0375c3d4412cb550ad1783358468a3b3b777da4829d72c7d6fb74f",
"index": "risk-score.risk-score-latest-default",
"source": {
"@timestamp": "2021-03-10T14:51:05.766Z",
"host": {
"name": "siem-kibana",
"risk": {
"calculated_level": "Critical",
"calculated_score_norm": 90,
"id_field": "host.name",
"id_value": "siem-kibana",
"calculated_score": 150,
"category_1_score": 80,
"category_1_count": 4354,
"category_2_score": 10,
"category_2_count": 1,
"notes": [],
"inputs": [
{
"id": "62895f54816047b9bf82929a61a6c571f41de9c2361670f6ef0136360e006f58",
"index": ".internal.alerts-security.alerts-default-000001",
"description": "New Rule Test",
"category": "category_1",
"risk_score": 70,
"timestamp": "2023-08-14T09:08:18.664Z"
},
{
"id": "e5bf3da3c855486ac7b40fa1aa33e19cf1380e413b79ed76bddf728f8fec4462",
"index": ".internal.alerts-security.alerts-default-000001",
"description": "New Rule Test",
"category": "category_1",
"risk_score": 70,
"timestamp": "2023-08-14T09:08:18.664Z"
}
]
}
}
}
}
```

<!--
Stage 3: Add more real world example source documents so we have at least 2 total, but ideally 3. Format as described in stage 2.
-->

## Scope of impact

<!--
Stage 2: Identifies scope of impact of changes. Are breaking changes required? Should deprecation strategies be adopted? Will significant refactoring be involved? Break the impact down into:
* Ingestion mechanisms (e.g. beats/logstash)
* Usage mechanisms (e.g. Kibana applications, detections)
* ECS project (e.g. docs, tooling)
The goal here is to research and understand the impact of these changes on users in the community and development teams across Elastic. 2-5 sentences each.
-->
As these proposed fields are currently employed by the entity analytics risk engine, there are no first-party adoption/deprecation concerns. Additionally, since these fields are purely additive with respect to the existing risk fields, there are no migration/deprecation concerns.

* Ingestion mechanisms
* While not officially supported, any systems that are currently ingesting risk documents (as originally introduced in RFC 31) _may_ need to be updated to support these additional fields.
* Usage mechanisms (e.g. Kibana applications, detections)
* Kibana's usage of these fields is being developed by the Entity Analytics team, mainly leveraging them to build novel UI features for Risk Explainability.
* ECS project (e.g. docs, tooling)
* There are no novel field types/mechanisms presented here.

## Concerns

There are two broad concerns at this stage:
There are three broad concerns at this stage:

1. Category fields introducing a new "ordered" pattern
* Rather than having either an array of objects, or an explicit `nested` field type, both of which allow an arbitrary number of items, we're instead opting to add 10 explicit fields (five explicit categories, each with two fields) under the _assumption_ that we won't extend the number of categories further. We have a bit of wiggle room (i.e. six categories, 12 fields wouldn't be out of question), but this is not a scalable solution if we need a large number of categories. However, that is only a potential future issue, and we can likely reevaluate and address it if/when it arises.
* RESOLUTION: we've decided to stick with this approach, for now.
ebeahan marked this conversation as resolved.
Show resolved Hide resolved
2. Mapping of `inputs` as a simple `object`
* The biggest motivation for this choice is to avoid the performance/storage/syntax complexities that come with a `nested` mapping, but we also don't have any feature requirements that would currently necessitate `inputs` being `nested`.

<!--
Stage 1: Identify potential concerns, implementation challenges, or complexity. Spend some time on this. Play devil's advocate. Try to identify the sort of non-obvious challenges that tend to surface later. The goal here is to surface risks early, allow everyone the time to work through them, and ultimately document resolution for posterity's sake.
-->

<!--
Stage 2: Document new concerns or resolutions to previously listed concerns. It's not critical that all concerns have resolutions at this point, but it would be helpful if resolutions were taking shape for the most significant concerns.
-->
* RESOLUTION: we've decided to stick with the simple, non-`nested` mapping for now. This decision is reinforced by the fact that these fields are used for the _explanation_ of a risk score, and it is unlikely that they will be useful in searches.
3. Lack of user feedback
* Since these fields have been developed in an unreleased kibana feature, we have not had the opportunity to e.g. release a Technical Preview in order to garner user adoption and feedback. Lack of community engagement on these fields is likely the biggest risk, at this point.

<!--
Stage 3: Document resolutions for all existing concerns. Any new concerns should be documented along with their resolution. The goal here is to eliminate risk of churn and instability by ensuring all concerns have been addressed.
Expand All @@ -141,6 +176,7 @@ The following are the people that consulted on the contents of this RFC.

* Stage 0: https://github.com/elastic/ecs/pull/2232
* Stage 1: https://github.com/elastic/ecs/pull/2236
* Stage 2: https://github.com/elastic/ecs/pull/2276

<!--
* Stage 1: https://github.com/elastic/ecs/pull/NNN
Expand Down
26 changes: 13 additions & 13 deletions rfcs/text/0042/risk.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
type: float
example: 33.0
description: >
The contribution of Category 1 to the overall risk score (`calculated_score`).
The contribution of Category 1 to the overall normalized risk score (`calculated_score_norm`).

Risk Categories logically group risk inputs from various domain use cases. Category 1 includes Alerts, namely from Kibana's Detection Engine.
- name: category_1_count
Expand All @@ -47,25 +47,25 @@
- name: category_2_score
level: extended
type: float
example: 35.0
example: 55.0
description: >
The contribution of Category 2 to the overall risk score (`calculated_score`).
The contribution of Category 2 to the overall normalized risk score (`calculated_score_norm`).

Risk Categories logically group risk inputs from various domain use cases. Category 2 includes inputs from Posture Management.
Risk Categories logically group risk inputs from various domain use cases. Category 2 includes Entity Contexts.
- name: category_2_count
level: extended
type: long
example: 1921
example: 1308
description: >
The number of risk input documents that contributed to the Category 2 score.

Risk Categories logically group risk inputs from various domain use cases. Category 2 includes inputs from Posture Management.
Risk Categories logically group risk inputs from various domain use cases. Category 2 includes Entity Contexts.
- name: category_3_score
level: extended
type: float
example: 25.0
description: >
The contribution of Category 3 to the overall risk score (`calculated_score`).
The contribution of Category 3 to the overall normalized risk score (`calculated_score_norm`).

Risk Categories logically group risk inputs from various domain use cases. Category 3 includes inputs from Vulnerabilities.
- name: category_3_count
Expand All @@ -79,25 +79,25 @@
- name: category_4_score
level: extended
type: float
example: 55.0
example: 35.0
description: >
The contribution of Category 4 to the overall risk score (`calculated_score`).
The contribution of Category 4 to the overall normalized risk score (`calculated_score_norm`).

Risk Categories logically group risk inputs from various domain use cases. Category 4 includes Entity Contexts.
Risk Categories logically group risk inputs from various domain use cases. Category 4 includes inputs from Posture Management.
- name: category_4_count
level: extended
type: long
example: 1308
example: 1921
description: >
The number of risk input documents that contributed to the Category 4 score.

Risk Categories logically group risk inputs from various domain use cases. Category 4 includes Entity Contexts.
Risk Categories logically group risk inputs from various domain use cases. Category 4 includes inputs from Posture Management.
- name: category_5_score
level: extended
type: float
example: 75.0
description: >
The contribution of Category 5 to the overall risk score (`calculated_score`).
The contribution of Category 5 to the overall normalized risk score (`calculated_score_norm`).
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ebeahan @SourinPaul this was the main change I made to convey that these category scores are themselves normalized. I originally had included the phrase "normalized contribution" to be more explicit, but that seemed redundant since the contribution to a normalized score only really makes sense if they can be compared (/are normalized / exist in the same value range, etc). Let me know if you opinions/suggestions.


Risk Categories logically group risk inputs from various domain use cases. Category 5 contains inputs from Anomalies.
- name: category_5_count
Expand Down
Loading