Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Security Solution][Detections] Update detection alert mappings to ECS v1.10.0 #101680

Merged
merged 6 commits into from
Jun 9, 2021
Merged

[Security Solution][Detections] Update detection alert mappings to ECS v1.10.0 #101680

merged 6 commits into from
Jun 9, 2021

Conversation

FrankHassanabad
Copy link
Contributor

@FrankHassanabad FrankHassanabad commented Jun 8, 2021 •
edited
Loading

Summary

  • Grabbed the ECS mappings from v1.10.0 tag
  • Updated the fields that had constant_keyword to keyword since we do many to 1 of source to signals index
  • Wrote a unit tests which tests to ensure we don't have any constant_keyword fields
  • Updated the SIGNALS_TEMPLATE_VERSION version by an increment of 10.

This should mostly fix:
#101572

Since agents add their data into _source even though they have a constant_keyword. When agents do not include the values in _source we will have to merge fields into _source before copying which are still planning on doing before release.

Checklist

@FrankHassanabad FrankHassanabad requested a review from a team as a code owner June 8, 2021 22:20
@FrankHassanabad FrankHassanabad self-assigned this Jun 8, 2021
@FrankHassanabad FrankHassanabad linked an issue Jun 8, 2021 that may be closed by this pull request
[Security Solution] For data_stream fields "Filter for value" and "Filter out value" are working opposite to their functionality. #101572
Closed
@FrankHassanabad FrankHassanabad added v8.0.0 v7.14.0 release_note:fix auto-backport Deprecated - use backport:version if exact versions are needed Team:Detections and Resp Security Detection Response Team labels Jun 8, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

rylnd
rylnd approved these changes Jun 9, 2021
View reviewed changes
Copy link
Contributor

@rylnd rylnd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thank you for removing some hardcoded versions in there, as well 👍

@FrankHassanabad FrankHassanabad enabled auto-merge (squash) June 9, 2021 21:04
@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @FrankHassanabad

@FrankHassanabad FrankHassanabad merged commit 1cb17d0 into elastic:master Jun 9, 2021
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Jun 9, 2021
@FrankHassanabad @kibanamachine
[Security Solution][Detections] Update detection alert mappings to EC…
7ce7a9e 
…S v1.10.0 (elastic#101680)

## Summary

* Grabbed the ECS mappings from [v1.10.0 tag]( https://github.com/elastic/ecs/blob/v1.10.0/generated/elasticsearch/7/template.json)
* Updated the fields that had `constant_keyword` to `keyword` since we do many to 1 of source to signals index
* Wrote a unit tests which tests to ensure we don't have any `constant_keyword` fields
* Updated the `SIGNALS_TEMPLATE_VERSION` version by an increment of 10.

This should mostly fix:
elastic#101572

Since agents add their data into `_source` even though they have a `constant_keyword`. When agents do not include the values in `_source` we will have to merge `fields` into `_source` before copying which are still planning on doing before release.

### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
@kibanamachine kibanamachine mentioned this pull request Jun 9, 2021
Merged
@kibanamachine
Copy link
Contributor

💚 Backport successful

Status Branch Result
✅ 7.x

This backport PR will be merged automatically after passing CI.

kibanamachine added a commit that referenced this pull request Jun 10, 2021
@kibanamachine @FrankHassanabad
[Security Solution][Detections] Update detection alert mappings to EC…
5f5cd72 
…S v1.10.0 (#101680) (#101847)

## Summary

* Grabbed the ECS mappings from [v1.10.0 tag]( https://github.com/elastic/ecs/blob/v1.10.0/generated/elasticsearch/7/template.json)
* Updated the fields that had `constant_keyword` to `keyword` since we do many to 1 of source to signals index
* Wrote a unit tests which tests to ensure we don't have any `constant_keyword` fields
* Updated the `SIGNALS_TEMPLATE_VERSION` version by an increment of 10.

This should mostly fix:
#101572

Since agents add their data into `_source` even though they have a `constant_keyword`. When agents do not include the values in `_source` we will have to merge `fields` into `_source` before copying which are still planning on doing before release.

### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios

Co-authored-by: Frank Hassanabad <frank.hassanabad@elastic.co>
rylnd added a commit to rylnd/kibana that referenced this pull request Jun 10, 2021
@rylnd
Merge branch 'master' into ad_hoc_enrichment
ada56e4 
* master: (173 commits)
  [kbnArchiver] convert archive names to root-relative paths (elastic#101839)
  [Reporting] Make "ScreenCapturePanel" shareable for Canvas (elastic#100623)
  [Alerting UI] Converted Rules and Connectors management pages to new layout. (elastic#101697)
  [Fleet] Support granular integrations in policy editor (elastic#101531)
  [Security Solution][Detections] Update detection alert mappings to ECS v1.10.0 (elastic#101680)
  [Fleet] Integrations UI: Adjust policies list UI (elastic#101600)
  chore(NA): moving @kbn/server-route-repository into bazel (elastic#101484)
  Support owner and description attributes inside the Manifest file, use in API docs (elastic#101786)
  [Security Solution] fix security empty overview links (elastic#101536)
  Unskips migration tests now that elastic search is fixed (elastic#101682)
  Fix endpoint -> integrations onboarding link (elastic#101804)
  [Alerting] Log warning when rules are not rescheduled due to Saved Object not found error (elastic#101591)
  Update datafeed_high_count_network_denies.json (elastic#101681)
  [Index patterns] Field editor example app (elastic#100524)
  [DOCS] Adding file upload to add data page (elastic#101674)
  [Security Solution][Endpoint] Adds Endpoint Host Isolation Status common component (elastic#101782)
  Upgrade ws v7.3.1->v7.4.2 and v6.2.1->v6.2.2 (elastic#101402)
  fixes embeddables migrate function  (elastic#101470)
  [Canvas] Update slow query in sample ecommerce workpad (elastic#100714)
  clarify which parts of TM are experimental (elastic#101757)
  ...
jloleysens added a commit to jloleysens/kibana that referenced this pull request Jun 10, 2021
@jloleysens
Merge branch 'master' of github.com:elastic/kibana into fleet/update-…
eb54fc0 
…add-agent-flyout

* 'master' of github.com:elastic/kibana: (35 commits)
  [Cases] Improve connectors mapping (elastic#101145)
  [ML] Fixes display of job group badges in recognizer wizard (elastic#101775)
  Fix es_archives path (elastic#101737)
  [kbnArchiver] convert archive names to root-relative paths (elastic#101839)
  [Reporting] Make "ScreenCapturePanel" shareable for Canvas (elastic#100623)
  [Alerting UI] Converted Rules and Connectors management pages to new layout. (elastic#101697)
  [Fleet] Support granular integrations in policy editor (elastic#101531)
  [Security Solution][Detections] Update detection alert mappings to ECS v1.10.0 (elastic#101680)
  [Fleet] Integrations UI: Adjust policies list UI (elastic#101600)
  chore(NA): moving @kbn/server-route-repository into bazel (elastic#101484)
  Support owner and description attributes inside the Manifest file, use in API docs (elastic#101786)
  [Security Solution] fix security empty overview links (elastic#101536)
  Unskips migration tests now that elastic search is fixed (elastic#101682)
  Fix endpoint -> integrations onboarding link (elastic#101804)
  [Alerting] Log warning when rules are not rescheduled due to Saved Object not found error (elastic#101591)
  Update datafeed_high_count_network_denies.json (elastic#101681)
  [Index patterns] Field editor example app (elastic#100524)
  [DOCS] Adding file upload to add data page (elastic#101674)
  [Security Solution][Endpoint] Adds Endpoint Host Isolation Status common component (elastic#101782)
  Upgrade ws v7.3.1->v7.4.2 and v6.2.1->v6.2.2 (elastic#101402)
  ...

# Conflicts:
#	x-pack/plugins/fleet/public/components/agent_enrollment_flyout/agent_policy_selection.tsx
#	x-pack/plugins/fleet/public/components/agent_enrollment_flyout/index.tsx
#	x-pack/plugins/fleet/public/components/agent_enrollment_flyout/managed_instructions.tsx
#	x-pack/plugins/fleet/public/components/agent_enrollment_flyout/standalone_instructions.tsx
jloleysens added a commit to jloleysens/kibana that referenced this pull request Jun 10, 2021
@jloleysens
Merge branch 'master' of github.com:elastic/kibana into fleet/adjust-…
96e51d9 
…add-integrations-redirect

* 'master' of github.com:elastic/kibana: (44 commits)
  Allow navigating discover flyout via arrow keys (elastic#101772)
  [Cases] Improve connectors mapping (elastic#101145)
  [ML] Fixes display of job group badges in recognizer wizard (elastic#101775)
  Fix es_archives path (elastic#101737)
  [kbnArchiver] convert archive names to root-relative paths (elastic#101839)
  [Reporting] Make "ScreenCapturePanel" shareable for Canvas (elastic#100623)
  [Alerting UI] Converted Rules and Connectors management pages to new layout. (elastic#101697)
  [Fleet] Support granular integrations in policy editor (elastic#101531)
  [Security Solution][Detections] Update detection alert mappings to ECS v1.10.0 (elastic#101680)
  [Fleet] Integrations UI: Adjust policies list UI (elastic#101600)
  chore(NA): moving @kbn/server-route-repository into bazel (elastic#101484)
  Support owner and description attributes inside the Manifest file, use in API docs (elastic#101786)
  [Security Solution] fix security empty overview links (elastic#101536)
  Unskips migration tests now that elastic search is fixed (elastic#101682)
  Fix endpoint -> integrations onboarding link (elastic#101804)
  [Alerting] Log warning when rules are not rescheduled due to Saved Object not found error (elastic#101591)
  Update datafeed_high_count_network_denies.json (elastic#101681)
  [Index patterns] Field editor example app (elastic#100524)
  [DOCS] Adding file upload to add data page (elastic#101674)
  [Security Solution][Endpoint] Adds Endpoint Host Isolation Status common component (elastic#101782)
  ...

# Conflicts:
#	x-pack/plugins/fleet/public/applications/fleet/sections/agent_policy/create_package_policy_page/index.tsx
#	x-pack/plugins/fleet/public/applications/fleet/sections/agent_policy/details_page/components/package_policies/package_policies_table.tsx
semd pushed a commit that referenced this pull request Jun 10, 2021
@FrankHassanabad @semd
[Security Solution][Detections] Update detection alert mappings to EC…
88a329e 
…S v1.10.0 (#101680)

## Summary

* Grabbed the ECS mappings from [v1.10.0 tag]( https://github.com/elastic/ecs/blob/v1.10.0/generated/elasticsearch/7/template.json)
* Updated the fields that had `constant_keyword` to `keyword` since we do many to 1 of source to signals index
* Wrote a unit tests which tests to ensure we don't have any `constant_keyword` fields
* Updated the `SIGNALS_TEMPLATE_VERSION` version by an increment of 10.

This should mostly fix:
#101572

Since agents add their data into `_source` even though they have a `constant_keyword`. When agents do not include the values in `_source` we will have to merge `fields` into `_source` before copying which are still planning on doing before release.

### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rylnd rylnd rylnd approved these changes

Assignees

@FrankHassanabad FrankHassanabad

Labels
auto-backport Deprecated - use backport:version if exact versions are needed release_note:fix Team:Detections and Resp Security Detection Response Team v7.14.0 v8.0.0
Projects
None yet
Milestone
No milestone
4 participants
@FrankHassanabad @elasticmachine @kibanamachine @rylnd