Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SIEM][Detection Engine] Adds a tags service and optimizes alert_id lookups #52838

Merged
merged 4 commits into from
Dec 12, 2019

Conversation

FrankHassanabad
Copy link
Contributor

@FrankHassanabad FrankHassanabad commented Dec 12, 2019

Summary

  • Adds a tags services for use by UI's that want to get a list of all the unique tags that are on all of the rules just like an aggregation
  • Removes the horribly inefficient alert_id look up that was a full alert scan and instead uses an internal structure that it augments to the tags for fast alert_id look ups.
  • Adds unit tests for the tags and internal structure tags
  • Updates other unit tests

Usage for the UI:

GET /api/detection_engine/tags

or shell script:

./get_tags.sh

Returns:

[
  "tag_1",
  "tag_2"
]

Testing:
Ensure that the internal structure does not leak when doing any of these script/API calls

  • ./get_tags.sh
  • ./post_rule.sh ./rules/queries/query_with_tags.json
  • ./update_rule.sh ./rules/queries/query_with_tags.json
  • ./delete_rule.sh
  • ./find_rules.sh
  • ./find_rule_by_filter.sh "alert.attributes.enabled:%20true"
  • ./find_rule_by_filter.sh "alert.attributes.tags:tag_1"

Caveat:

You can do filter searches against tags that have the double underscore such as:

./find_rule_by_filter.sh "alert.attributes.tags:%20__*"

But that shouldn't be a big problem and more than likely no one will be naming something with double underscores.

Checklist

Use strikethroughs to remove checklist items you don't feel are applicable to this PR.

- [ ] This was checked for cross-browser compatibility, including a check against IE11

- [ ] Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support

- [ ] Documentation was added for features that require explanation or tutorials

- [ ] This was checked for keyboard-only and screenreader accessibility

For maintainers

- [ ] This was checked for breaking API changes and was labeled appropriately

@FrankHassanabad FrankHassanabad changed the title [SIEM][Detection Engine] Adds a tags services and optimizes alert_id lookups [SIEM][Detection Engine] Adds a tags service and optimizes alert_id lookups Dec 12, 2019
@elasticmachine
Copy link
Contributor

Pinging @elastic/siem (Team:SIEM)

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

if (ruleId == null) {
return tags;
} else {
return [...tags, `${INTERNAL_RULE_ID_KEY}:${ruleId}`];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be (slightly) more performant here to utilize concat to append the internal rule id tag.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per conversations we will keep using the ES2015+ style unless we are dealing with larger arrays.

Thanks for pointing out though that technically the concat is faster as I typically like that style more but the ES2015 is what we encourage on the team for small stuff.

Copy link
Contributor

@dhurley14 dhurley14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left an optional comment but reviewed and tested locally and everything looks awesome!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants