[Security Solutions] Fixes bug with the filter query compatibility for transforms #104559

FrankHassanabad · 2021-07-06T21:55:49Z

Summary

Fixes bug with the filter query compatibility to allow multiple object types and match all
Adds unit tests for the file
Fixes up the README.md a bit
Adds more unit tests to the utils folder we didn't have before
Adds more JSDocs

Checklist

Unit or functional tests were updated

…aster

elasticmachine · 2021-07-07T00:49:10Z

Pinging @elastic/security-detections-response (Team:Detections and Resp)

docs/developer/plugin-list.asciidoc

x-pack/plugins/metrics_entities/README.md

spong · 2021-07-07T20:46:35Z

x-pack/plugins/security_solution/public/transforms/utils/is_filter_query_compatible.ts

-      filterQuery === '{"bool":{"must":[],"filter":[{"match_all":{}}],"should":[],"must_not":[]}}'
-    );
+    try {
+      const filterQueryObject = JSON.parse(filterQuery);


This filterQuery will only ever be the filters from the main page-level KQL bar, correct? No need to worry about excessively large filters here?

That is correct.

spong

Checked out, verified fix locally, and reviewed code. Now seeing transforms activate with KQL filters like *:* when appropriately configured. Appreciate the fix and added tests/docs @FrankHassanabad -- thanks! LGTM! 👍

Note: Did see the following stack trace in testing, but was able to reproduce on master so doesn't look to be introduced here. Will follow-up with another ticket if not already being tracked.

Stack trace

server    log   [16:40:29.535] [info][plugins][reporting] Browser executable: /Users/garrettspong/dev/kibana-master/x-pack/plugins/reporting/chromium/headless_shell-darwin_x64/headless_shell
server    log   [16:41:00.733] [error][http] KbnServerError: index_not_found_exception: [index_not_found_exception] Reason: no such index [*elastic-cloud-logs-*]
    at getKbnServerError (/Users/garrettspong/dev/kibana-master/src/plugins/kibana_utils/server/report_server_error.ts:27:10)
    at /Users/garrettspong/dev/kibana-master/src/plugins/data/server/autocomplete/value_suggestions_route.ts:65:15
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at Router.handle (/Users/garrettspong/dev/kibana-master/src/core/server/http/router/router.ts:273:30)
    at handler (/Users/garrettspong/dev/kibana-master/src/core/server/http/router/router.ts:228:11)
    at exports.Manager.execute (/Users/garrettspong/dev/kibana-master/node_modules/@hapi/hapi/lib/toolkit.js:60:28)
    at Object.internals.handler (/Users/garrettspong/dev/kibana-master/node_modules/@hapi/hapi/lib/handler.js:46:20)
    at exports.execute (/Users/garrettspong/dev/kibana-master/node_modules/@hapi/hapi/lib/handler.js:31:20)
    at Request._lifecycle (/Users/garrettspong/dev/kibana-master/node_modules/@hapi/hapi/lib/request.js:370:32)
    at Request._execute (/Users/garrettspong/dev/kibana-master/node_modules/@hapi/hapi/lib/request.js:279:9) {
  statusCode: 500,
  errBody: undefined
}
server   error  [16:41:00.466]  Error: Internal Server Error
    at HapiResponseAdapter.toInternalError (/Users/garrettspong/dev/kibana-master/src/core/server/http/router/response_adapter.ts:53:19)
    at Router.handle (/Users/garrettspong/dev/kibana-master/src/core/server/http/router/router.ts:287:34)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at handler (/Users/garrettspong/dev/kibana-master/src/core/server/http/router/router.ts:228:11)
    at exports.Manager.execute (/Users/garrettspong/dev/kibana-master/node_modules/@hapi/hapi/lib/toolkit.js:60:28)
    at Object.internals.handler (/Users/garrettspong/dev/kibana-master/node_modules/@hapi/hapi/lib/handler.js:46:20)
    at exports.execute (/Users/garrettspong/dev/kibana-master/node_modules/@hapi/hapi/lib/handler.js:31:20)
    at Request._lifecycle (/Users/garrettspong/dev/kibana-master/node_modules/@hapi/hapi/lib/request.js:370:32)
    at Request._execute (/Users/garrettspong/dev/kibana-master/node_modules/@hapi/hapi/lib/request.js:279:9)

kibanamachine · 2021-07-08T01:22:47Z

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`securitySolution`	6.3MB	6.3MB	+443.0B

History

💚 Build #137019 succeeded 839b693
💚 Build #136760 succeeded fb255ae
💔 Build #136736 failed 2267628

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @FrankHassanabad

…r transforms (elastic#104559) ## Summary * Fixes bug with the filter query compatibility to allow multiple object types and match all * Adds unit tests for the file * Fixes up the README.md a bit * Adds more unit tests to the utils folder we didn't have before * Adds more JSDocs ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated

kibanamachine · 2021-07-08T03:07:00Z

💚 Backport successful

Status	Branch	Result
✅	7.x

This backport PR will be merged automatically after passing CI.

…-of-max-results * 'master' of github.com:elastic/kibana: (36 commits) Lower Kibana app bundle limits (elastic#104688) [Security Solutions] Fixes bug with the filter query compatibility for transforms (elastic#104559) [RAC] Add mapping update logic to RuleDataClient (elastic#102586) Fix import workpad (elastic#104722) [canvas] Fix Storybook service decorator (elastic#104750) [Detection Rules] Add 7.14 rules (elastic#104772) [Enterprise Search] Fix beta notification in sidebar (elastic#104763) Fix engine routes that are meta engine or non-meta-engine specific (elastic#104757) [Fleet] Fix policy revision number getting bumped for no reason (elastic#104696) persistable state migrations (elastic#103680) [Fleet] Fix add agent in the package policy table (elastic#104749) [DOCS] Creates separate doc for security in production (elastic#103973) [SO Migration] fix reindex race on multi-instance mode (elastic#104516) [Security Solution] Update text in Endpoint Admin pages (elastic#104649) [package testing] Decrease timeout to 2 hours (elastic#104668) Fix background styling of waterfall chart sidebar tooltip. (elastic#103997) [Fleet + Integrations UI] Integrations UI Cleanup (elastic#104641) [Fleet] Link to download page of current stack version on Agent install instructions (elastic#104494) [Workplace Search] Fix Media Type field preview is unformatted bug (elastic#104684) [ML] add marker body (elastic#104672) ... # Conflicts: # x-pack/plugins/fleet/public/search_provider.test.ts

…r transforms (#104559) (#104798) ## Summary * Fixes bug with the filter query compatibility to allow multiple object types and match all * Adds unit tests for the file * Fixes up the README.md a bit * Adds more unit tests to the utils folder we didn't have before * Adds more JSDocs ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated Co-authored-by: Frank Hassanabad <frank.hassanabad@elastic.co>

## Summary Removes the `metrics_entities` plugin and POC. As a different direction will be taken and people can look back at the git history for it as they see fit if they need to refer to it. Once it's re-added it it will be through an RFC process and re-discussed. Earlier PR's which added the POC: #96446 #104559 ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios

Fixes small bug with the filter query compatibility from changes in m…

2267628

…aster

FrankHassanabad requested a review from a team as a code owner July 6, 2021 21:55

FrankHassanabad self-assigned this Jul 6, 2021

FrankHassanabad added v8.0.0 v7.15.0 auto-backport Deprecated - use backport:version if exact versions are needed release_note:skip Skip the PR/issue when compiling release notes labels Jul 6, 2021

FrankHassanabad requested a review from spong July 6, 2021 21:57

FrankHassanabad added 3 commits July 6, 2021 17:39

Adds more unit tests

ee1284e

More unit tests

32a726c

Updated docs from CI

fb255ae

FrankHassanabad added the Team:Detections and Resp Security Detection Response Team label Jul 7, 2021

Merge branch 'master' into fixes-transform-bug-with-query-filter

839b693

spong reviewed Jul 7, 2021

View reviewed changes

docs/developer/plugin-list.asciidoc Outdated Show resolved Hide resolved

spong reviewed Jul 7, 2021

View reviewed changes

x-pack/plugins/metrics_entities/README.md Outdated Show resolved Hide resolved

spong reviewed Jul 7, 2021

View reviewed changes

spong approved these changes Jul 7, 2021

View reviewed changes

FrankHassanabad added 3 commits July 7, 2021 17:06

Added more unit tests

a92b471

Merge branch 'master' into fixes-transform-bug-with-query-filter

c68bd83

Fixed wording

b10e7dd

spong mentioned this pull request Jul 7, 2021

[Detections][Security Solution] Values are not populated on exceptions dialog #104371

Closed

FrankHassanabad merged commit 6e21285 into elastic:master Jul 8, 2021

kibanamachine mentioned this pull request Jul 8, 2021

[7.x] [Security Solutions] Fixes bug with the filter query compatibility for transforms (#104559) #104798

Merged

FrankHassanabad mentioned this pull request Apr 7, 2022

[Security Solutions] Removes POC transforms #129673

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Security Solutions] Fixes bug with the filter query compatibility for transforms #104559

[Security Solutions] Fixes bug with the filter query compatibility for transforms #104559

FrankHassanabad commented Jul 6, 2021 •

edited

Loading

elasticmachine commented Jul 7, 2021

spong Jul 7, 2021

FrankHassanabad Jul 7, 2021

spong left a comment

kibanamachine commented Jul 8, 2021

kibanamachine commented Jul 8, 2021

[Security Solutions] Fixes bug with the filter query compatibility for transforms #104559

[Security Solutions] Fixes bug with the filter query compatibility for transforms #104559

Conversation

FrankHassanabad commented Jul 6, 2021 • edited Loading

Summary

Checklist

elasticmachine commented Jul 7, 2021

spong Jul 7, 2021

Choose a reason for hiding this comment

FrankHassanabad Jul 7, 2021

Choose a reason for hiding this comment

spong left a comment

Choose a reason for hiding this comment

kibanamachine commented Jul 8, 2021

💚 Build Succeeded

Metrics [docs]

Async chunks

History

kibanamachine commented Jul 8, 2021

💚 Backport successful

FrankHassanabad commented Jul 6, 2021 •

edited

Loading