Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] [Timeline] Timeline manager tweaks #69988

Merged
merged 11 commits into from
Jul 7, 2020

Conversation

stephmilovic
Copy link
Contributor

@stephmilovic stephmilovic commented Jun 25, 2020

Summary

A couple of small refactors to manage_timeline:

  1. Passes row data to the functions that generate timelineRowActions ( for Prince @spong )
  2. Moves the filterManager up to initializeTimeline and gets rid of the separate setTimelineFilterManager step (for Prince King @XavierM )

@stephmilovic stephmilovic added Team:SIEM v8.0.0 release_note:skip Skip the PR/issue when compiling release notes v7.9.0 labels Jun 25, 2020
@stephmilovic stephmilovic self-assigned this Jun 25, 2020
@stephmilovic stephmilovic marked this pull request as ready for review June 25, 2020 19:52
@stephmilovic stephmilovic requested review from a team as code owners June 25, 2020 19:52
@elasticmachine
Copy link
Contributor

Pinging @elastic/siem (Team:SIEM)

@peluja1012
Copy link
Contributor

Hi @stephmilovic, I noticed that in this commit 63e15d3, you stopped passing in nonEcsData to the TimelineRowActions. I was wondering if you could add that back. We are working on a PR for Exceptions where we need access to the nonEcsData in the action.

@dplumlee
Copy link
Contributor

Hey, just echoing @peluja1012 's comment, was wondering if there was a reason to remove those 2 fields other than not being utilized yet? The exceptions pr relies on access to both and seems fitting to include in this pr. Thanks

@stephmilovic
Copy link
Contributor Author

stephmilovic commented Jun 29, 2020

@peluja1012 @dplumlee an unnamed engineer told me to remove it as it was not necessary, but i can certainly put it back. See Xavier's comment below

@XavierM
Copy link
Contributor

XavierM commented Jun 29, 2020

@peluja1012

Hi @stephmilovic, I noticed that in this commit 63e15d3, you stopped passing in nonEcsData to the TimelineRowActions. I was wondering if you could add that back. We are working on a PR for Exceptions where we need access to the nonEcsData in the action.

I am the unnamed engineer, why it is not useful to pass you the NonEcsData. It is because this attribute data: TimelineNonEcsData[] only brings back the data for the columns in the timeline and they will also be in the ECS attributes. Since everything in the timeline should be ECS format.

@peluja1012
Copy link
Contributor

I am the unnamed engineer, why it is not useful to pass you the NonEcsData. It is because this attribute data: TimelineNonEcsData[] only brings back the data for the columns in the timeline and they will also be in the ECS attributes. Since everything in the timeline should be ECS format.

@stephmilovic @XavierM For the “Add Exception” action, we need to fetch more fields that the ones displayed in the Timeline columns. @spong recommended that we add the fields we need here https://github.com/elastic/kibana/blob/master/x-pack/plugins/security_solution/public/alerts/components/alerts_table/default_config.tsx#L165. Those fields only seem to be present in nonEcsData. Is there another way to get these additional fields into the action?

@XavierM
Copy link
Contributor

XavierM commented Jun 30, 2020

I am the unnamed engineer, why it is not useful to pass you the NonEcsData. It is because this attribute data: TimelineNonEcsData[] only brings back the data for the columns in the timeline and they will also be in the ECS attributes. Since everything in the timeline should be ECS format.

@stephmilovic @XavierM For the “Add Exception” action, we need to fetch more fields that the ones displayed in the Timeline columns. @spong recommended that we add the fields we need here https://github.com/elastic/kibana/blob/master/x-pack/plugins/security_solution/public/alerts/components/alerts_table/default_config.tsx#L165. Those fields only seem to be present in nonEcsData. Is there another way to get these additional fields into the action?

There are already in the ECS data, because of that https://github.com/elastic/kibana/blob/master/x-pack/plugins/security_solution/public/timelines/containers/index.gql_query.ts#L198

Copy link
Contributor

@andrew-goldstein andrew-goldstein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for these tweaks @stephmilovic
Desk tested the integration with the Investigate in Resolver functionality, + some ad hoc testing, and it's (still) looking good
LGTM 🚀

@kibanamachine
Copy link
Contributor

💛 Build succeeded, but was flaky


Test Failures

Firefox UI Functional Tests.test/functional/apps/visualize/_tsvb_chart·ts.visualize app visual builder "before each" hook for "should verify topN label and count display"

Link to Jenkins

Standard Out

[00:00:00]       │
[00:12:03]         └-: visualize app
[00:12:03]           └-> "before all" hook
[00:12:03]           └-> "before all" hook
[00:12:03]             │ debg Starting visualize before method
[00:12:03]             │ info [logstash_functional] Loading "mappings.json"
[00:12:03]             │ info [logstash_functional] Loading "data.json.gz"
[00:12:03]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1594150454246726161] [logstash-2015.09.22] creating index, cause [api], templates [], shards [1]/[0]
[00:12:03]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1594150454246726161] current.health="GREEN" message="Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[logstash-2015.09.22][0]]])." previous.health="YELLOW" reason="shards started [[logstash-2015.09.22][0]]"
[00:12:03]             │ info [logstash_functional] Created index "logstash-2015.09.22"
[00:12:03]             │ debg [logstash_functional] "logstash-2015.09.22" settings {"index":{"analysis":{"analyzer":{"url":{"max_token_length":"1000","tokenizer":"uax_url_email","type":"standard"}}},"number_of_replicas":"0","number_of_shards":"1"}}
[00:12:03]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1594150454246726161] [logstash-2015.09.20] creating index, cause [api], templates [], shards [1]/[0]
[00:12:04]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1594150454246726161] current.health="GREEN" message="Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[logstash-2015.09.20][0]]])." previous.health="YELLOW" reason="shards started [[logstash-2015.09.20][0]]"
[00:12:04]             │ info [logstash_functional] Created index "logstash-2015.09.20"
[00:12:04]             │ debg [logstash_functional] "logstash-2015.09.20" settings {"index":{"analysis":{"analyzer":{"url":{"max_token_length":"1000","tokenizer":"uax_url_email","type":"standard"}}},"number_of_replicas":"0","number_of_shards":"1"}}
[00:12:04]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1594150454246726161] [logstash-2015.09.21] creating index, cause [api], templates [], shards [1]/[0]
[00:12:04]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1594150454246726161] current.health="GREEN" message="Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[logstash-2015.09.21][0]]])." previous.health="YELLOW" reason="shards started [[logstash-2015.09.21][0]]"
[00:12:04]             │ info [logstash_functional] Created index "logstash-2015.09.21"
[00:12:04]             │ debg [logstash_functional] "logstash-2015.09.21" settings {"index":{"analysis":{"analyzer":{"url":{"max_token_length":"1000","tokenizer":"uax_url_email","type":"standard"}}},"number_of_replicas":"0","number_of_shards":"1"}}
[00:12:12]             │ info [logstash_functional] Indexed 4633 docs into "logstash-2015.09.22"
[00:12:12]             │ info [logstash_functional] Indexed 4757 docs into "logstash-2015.09.20"
[00:12:12]             │ info [logstash_functional] Indexed 4614 docs into "logstash-2015.09.21"
[00:12:13]             │ info [long_window_logstash] Loading "mappings.json"
[00:12:13]             │ info [long_window_logstash] Loading "data.json.gz"
[00:12:13]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1594150454246726161] [long-window-logstash-0] creating index, cause [api], templates [], shards [1]/[0]
[00:12:13]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1594150454246726161] current.health="GREEN" message="Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[long-window-logstash-0][0]]])." previous.health="YELLOW" reason="shards started [[long-window-logstash-0][0]]"
[00:12:13]             │ info [long_window_logstash] Created index "long-window-logstash-0"
[00:12:13]             │ debg [long_window_logstash] "long-window-logstash-0" settings {"index":{"analysis":{"analyzer":{"makelogs_url":{"max_token_length":"1000","tokenizer":"uax_url_email","type":"standard"}}},"number_of_replicas":"0","number_of_shards":"1"}}
[00:12:23]             │ info progress: 13748
[00:12:23]             │ info [long_window_logstash] Indexed 14005 docs into "long-window-logstash-0"
[00:12:23]             │ info [visualize] Loading "mappings.json"
[00:12:23]             │ info [visualize] Loading "data.json"
[00:12:23]             │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-centos-tests-xxl-1594150454246726161] [.kibana_2/5dMmxbhbQ7GxBfYQCQB3Sw] deleting index
[00:12:23]             │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-centos-tests-xxl-1594150454246726161] [.kibana_1/CoBUMxDiSdKLfjtsxAuyqA] deleting index
[00:12:23]             │ info [visualize] Deleted existing index [".kibana_2",".kibana_1"]
[00:12:23]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1594150454246726161] [.kibana] creating index, cause [api], templates [], shards [1]/[1]
[00:12:23]             │ info [visualize] Created index ".kibana"
[00:12:23]             │ debg [visualize] ".kibana" settings {"index":{"number_of_replicas":"1","number_of_shards":"1"}}
[00:12:23]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-centos-tests-xxl-1594150454246726161] [.kibana/ArSnOyVfR7W6xSptGIa63g] update_mapping [_doc]
[00:12:23]             │ info [visualize] Indexed 12 docs into ".kibana"
[00:12:23]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-centos-tests-xxl-1594150454246726161] [.kibana/ArSnOyVfR7W6xSptGIa63g] update_mapping [_doc]
[00:12:23]             │ debg Migrating saved objects
[00:12:23]             │ proc [kibana]   log   [20:10:20.933] [info][savedobjects-service] Creating index .kibana_2.
[00:12:23]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1594150454246726161] [.kibana_2] creating index, cause [api], templates [], shards [1]/[1]
[00:12:23]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1594150454246726161] updating number_of_replicas to [0] for indices [.kibana_2]
[00:12:24]             │ proc [kibana]   log   [20:10:20.982] [info][savedobjects-service] Reindexing .kibana to .kibana_1
[00:12:24]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1594150454246726161] [.kibana_1] creating index, cause [api], templates [], shards [1]/[1]
[00:12:24]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1594150454246726161] updating number_of_replicas to [0] for indices [.kibana_1]
[00:12:24]             │ info [o.e.t.LoggingTaskListener] [kibana-ci-immutable-centos-tests-xxl-1594150454246726161] 9052 finished with response BulkByScrollResponse[took=24.5ms,timed_out=false,sliceId=null,updated=0,created=12,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[00:12:24]             │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-centos-tests-xxl-1594150454246726161] [.kibana/ArSnOyVfR7W6xSptGIa63g] deleting index
[00:12:24]             │ proc [kibana]   log   [20:10:21.320] [info][savedobjects-service] Migrating .kibana_1 saved objects to .kibana_2
[00:12:24]             │ proc [kibana]   log   [20:10:21.330] [error][savedobjects-service] Error: Unable to migrate the corrupt Saved Object document index-pattern:test_index*. To prevent Kibana from performing a migration on every restart, please delete or fix this document by ensuring that the namespace and type in the document's id matches the values in the namespace and type fields.
[00:12:24]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-centos-tests-xxl-1594150454246726161] [.kibana_2/GBx3xkq6RXCyu1dEopatMA] update_mapping [_doc]
[00:12:24]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-centos-tests-xxl-1594150454246726161] [.kibana_2/GBx3xkq6RXCyu1dEopatMA] update_mapping [_doc]
[00:12:24]             │ proc [kibana]   log   [20:10:21.415] [info][savedobjects-service] Pointing alias .kibana to .kibana_2.
[00:12:24]             │ proc [kibana]   log   [20:10:21.463] [info][savedobjects-service] Finished in 531ms.
[00:12:24]             │ debg applying update to kibana config: {"accessibility:disableAnimations":true,"dateFormat:tz":"UTC"}
[00:12:24]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-centos-tests-xxl-1594150454246726161] [.kibana_2/GBx3xkq6RXCyu1dEopatMA] update_mapping [_doc]
[00:12:26]             │ debg replacing kibana config doc: {"defaultIndex":"logstash-*","format:bytes:defaultPattern":"0,0.[000]b"}
[00:12:59]           └-: 
[00:12:59]             └-> "before all" hook
[00:12:59]             └-: visual builder
[00:12:59]               └-> "before all" hook

Stack Trace

Error: retry.try timeout: Error: retry.try timeout: TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="createVisualizationPromptButton"])
Wait timed out after 10012ms
    at /dev/shm/workspace/kibana/node_modules/selenium-webdriver/lib/webdriver.js:842:17
    at process._tickCallback (internal/process/next_tick.js:68:7)
    at onFailure (test/common/services/retry/retry_for_success.ts:28:9)
    at retryForSuccess (test/common/services/retry/retry_for_success.ts:68:13)
    at onFailure (test/common/services/retry/retry_for_success.ts:28:9)
    at retryForSuccess (test/common/services/retry/retry_for_success.ts:68:13)

Build metrics

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@stephmilovic stephmilovic merged commit 3f5ebdd into elastic:master Jul 7, 2020
@stephmilovic stephmilovic deleted the timeline-manager-tweaks branch July 7, 2020 21:27
stephmilovic added a commit to stephmilovic/kibana that referenced this pull request Jul 7, 2020
gmmorris added a commit to gmmorris/kibana that referenced this pull request Jul 8, 2020
* master: (36 commits)
  fixed api url in example plugin (elastic#70934)
  [data.search.aggs]: Remove remaining client dependencies (elastic#70251)
  [Security Solution][Endpoint] Fix base64 download bug and adopt new user artifact/manifest format (elastic#70998)
  [Security Solution][Exceptions] - Exception Modal Part I (elastic#70639)
  [SIEM][Detection Engine][Lists] Adds additional data types to value based lists
  [SIEM][Detection Engine][Lists] Removes feature flag for lists
  [APM] Show license callout in ML settings (elastic#70959)
  Migrate service settings test to jest (elastic#70992)
  [APM] Add cloud attributes to data telemetry (elastic#71008)
  Fix breadcrumb on panels for visibility / round corners (elastic#71010)
  Improve search typescript (elastic#69333)
  [savedObjects field count] run in baseline job (elastic#70999)
  [Security Solution] [Timeline] Timeline manager tweaks (elastic#69988)
  [Endpoint] Support redirect from Policy Details to Ingest when user initiates Edit Policy from Datasource Edit page (elastic#70874)
  [APM] Add API tests (elastic#70740)
  [Security Solution][Exceptions] - Tie server and client code together (elastic#70918)
  [Audit Logging] Add AuditTrail service (elastic#69278)
  [Usage Collection] Ensure no type duplicates (elastic#70946)
  [Security Solution] [Timeline] Bugfix for timeline row actions disappear sometimes (elastic#70958)
  [CI] Add pipeline task queue framework and merge workers into one (elastic#64011)
  ...
@MindyRS MindyRS added the Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. label Sep 23, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release_note:skip Skip the PR/issue when compiling release notes Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:SIEM v7.9.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants