Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Signal scores are not mapped as float. #63343

Closed
jamesspi opened this issue Apr 13, 2020 · 5 comments · Fixed by #71126
Closed

Signal scores are not mapped as float. #63343

jamesspi opened this issue Apr 13, 2020 · 5 comments · Fixed by #71126
Assignees
Labels
bug Fixes for quality problems that affect the customer experience Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:SIEM v7.9.0

Comments

@jamesspi
Copy link

Kibana version: 7.6.2

Elasticsearch version: 7.6.2

Server OS version: ESS

Describe the bug: signal.rule.risk_score is mapped as text

Steps to reproduce:

  1. Create/enable a signal, wait for it to fire.

Expected behavior: This should be mapped as float to allow users to run mathematical calculations, as well as conform to ECS.

Screenshots (if relevant):
image

@elasticmachine
Copy link
Contributor

Pinging @elastic/siem (Team:SIEM)

@jamesspi jamesspi added the bug Fixes for quality problems that affect the customer experience label Apr 13, 2020
@spong spong added the v7.9.0 label Jun 25, 2020
@spong
Copy link
Member

spong commented Jun 25, 2020

@MadameSheema this is currently being worked for 7.9 by @marshallmain

@marshallmain
Copy link
Contributor

@jamesspi I'm testing out a possible fix for this issue. Do you have an example query I can test with that would be useful to run on risk_score mapped as a float and isn't possible when it's mapped as a keyword?

@marshallmain
Copy link
Contributor

marshallmain commented Jul 7, 2020

Also, following up on the discussion in the security detections sync today to summarize and expand on the list of app features that break when changing the mapping of risk_score without reindexing:

  • The alert count histogram on the main page fails if you choose "Stack by: signal.rule.risk_score". It fails even if all alerts in the provided time range (e.g. last 24 hours) use the new mapping. It works if you delete the indices that use the old mapping.
    Stacking by risk_score was thought not to be critical in the discussion, so we may be able to remove it.

  • (new) Sorting the alert list by risk_score causes it to fail to load if the provided time range contains alerts that use both the old and new mappings. Note that unlike the "stack by" behavior above, sorting the alert list by risk_score does still work if all alerts in the time range use the same mapping.

@jamesspi
Copy link
Author

jamesspi commented Jul 8, 2020

Hey @marshallmain, mainly it would be queries like "sum of the risk score for host x over this time period", and, this will also give us the ability to run metrics based machine learning jobs - "this host typically never has a risk score of 100". Does that make sense?

@MindyRS MindyRS added the Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. label Oct 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Fixes for quality problems that affect the customer experience Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:SIEM v7.9.0
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants