Resolver in timeline #69728

oatkiller · 2020-06-23T18:10:01Z

Summary

based on #70111

This displays resolver in the security solution timeline.

Screenshots

Panning between nodes using the panel, in the timeline

Browsing resolver's via event panel

Loading resolver w/ artificially added latency (to show the loading dialog)

Artificially causing the request for entity_id to fail (the request made with _id)

Artificially causing the main request for resolver data to fail (the request made w/ entity_id)

Some data from the generator, in the timeline (not animated)

To fix before merge

Follow up work

API integration test plan

what happens if the indices aren't found
happy path

Checklist

Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
This was checked for keyboard-only and screenreader accessibility
This renders correctly on smaller devices using a responsive layout. (You can test this in your browser
This was checked for cross-browser compatibility, including a check against IE11

For maintainers

This was checked for breaking API changes and was labeled appropriately

x-pack/plugins/security_solution/public/timelines/components/timeline/body/helpers.ts

x-pack/plugins/security_solution/public/resolver/store/middleware/resolver_tree_fetcher.ts

x-pack/plugins/security_solution/public/resolver/store/data/selectors.ts

bkimmel · 2020-06-24T20:58:46Z

x-pack/plugins/security_solution/public/resolver/store/middleware/resolver_tree_fetcher.ts

+ * you may not use this file except in compliance with the Elastic License.
+ */
+
+/* eslint-disable no-duplicate-imports */


❔ what's this for

editor automatically fixing superfluous eslint rules

bkimmel · 2020-06-24T21:07:57Z

x-pack/plugins/security_solution/public/resolver/store/middleware/resolver_tree_fetcher.ts

+ const entityIDToFetch = selectors.entityIDToFetch(state);
+
+ if (selectors.shouldAbortRequest(state) && lastRequest) {
+ lastRequest.abortController.abort();


❔ Should check https://devdocs.io/dom/abortsignal/aborted before .abort?

The signal only seems to raise an abort event once regardless of how many times you call abort, so it's probably fine.

var ac = new AbortController(); var signal = ac.signal; signal.addEventListener('abort', ()=>{ console.log('aborted') }); ac.abort(); ac.abort();//only logs `abort` once

bkimmel · 2020-06-24T21:11:38Z

x-pack/plugins/security_solution/public/resolver/store/middleware/resolver_tree_fetcher.ts

+ });
+ } catch {
+ api.dispatch({
+ type: 'serverFailedToReturnResolverData',


❔ This runs when a signal aborts right? Seems like we should have a distinct catch for AbortError and an action like appReceivedAbortSignalDuringResolverDataFetch

seems like a good idea. i'm going to try that.

Something like:

} catch (error) { // https://developer.mozilla.org/en-US/docs/Web/API/DOMException#exception-AbortError if (error instanceof DOMException && error.name === 'AbortError') { api.dispatch({ type: 'appAbortedResolverDataRequest', payload: databaseDocumentIDToFetch, }); } else { api.dispatch({ type: 'serverFailedToReturnResolverData', payload: databaseDocumentIDToFetch, }); } }

?

oatkiller

Need to do some cleanup before merging

x-pack/plugins/security_solution/public/resolver/models/resolver_tree.ts

+/**
+ * Return each node of the tree.
+ * used to calculate related event stas
+ * TODO, this is used twice, by two separate things? probably wrong


oatkiller · 2020-06-26T19:58:44Z

x-pack/plugins/security_solution/public/resolver/store/data/reducer.test.ts

+/**
+ * Test the data reducer and selector.
+ */
+describe('Resolver Data Middleware', () => {


@bkimmel i moved your tests here

x-pack/plugins/security_solution/public/resolver/store/data/selectors.test.ts

x-pack/plugins/security_solution/public/resolver/view/styles.tsx

oatkiller · 2020-06-26T20:17:04Z

x-pack/plugins/security_solution/public/resolver/view/use_camera.test.tsx

@@ -27,7 +28,7 @@ describe('useCamera on an unpainted element', () => {
 let simulator: SideEffectSimulator;

 beforeEach(async () => {
- ({ store } = storeFactory());
+ store = storeFactory();


this was returning an object with the store in it.

oatkiller · 2020-06-26T20:18:43Z

x-pack/plugins/security_solution/public/timelines/components/timeline/styles.tsx

@@ -29,7 +29,7 @@ export const TimelineBody = styled.div.attrs(({ className = '' }) => ({
 overflow: auto;
 scrollbar-width: thin;
 flex: 1;
- visibility: ${({ visible }) => (visible ? 'visible' : 'hidden')};
+ display: ${({ visible }) => (visible ? 'block' : 'none')};


If we use 'visibility' here, it still occupies space. The parent is 'flex' and Resolver is a sibling of this component. We need Resolver to grow. Seems like display: none did the trick. Anyone know of any reason why this would be bad?

@XavierM seems to know many things

x-pack/plugins/security_solution/server/endpoint/routes/resolver/entity.ts

oatkiller · 2020-06-26T20:20:56Z

x-pack/plugins/security_solution/server/endpoint/routes/resolver/entity.ts

+ * what happens if there are more matches than can be returned from es?
+ * do we even need more than one match?
+ */
+export function handleEntities(


@jonathan-buttner if ya get a chance, could ya give it a quick look?

elasticmachine · 2020-06-26T21:51:31Z

Pinging @elastic/endpoint-app-team (Feature:Endpoint)

oatkiller · 2020-06-27T13:22:42Z

@elasticmachine merge upstream

…r deps check script we run.

elasticmachine · 2020-06-29T13:12:56Z

Pinging @elastic/endpoint-data-visibility-team (Team:Endpoint Data Visibility)

jonathan-buttner · 2020-06-29T13:39:46Z

.../security_solution/public/resolver/models/indexed_process_tree/isometric_taxi_layout.test.ts

+ endgame: {
+ process_name: '',
+ event_type_full: 'process_event',
+ event_subtype_full: 'creation_event',


Do we also need to mock the new style endpoint events in ecs?

I don't think it should matter here. The code operates on the ResolverEvent interface which supports either old or new events. As long as the methods works correctly on old and new style events, this code should work. We could also test those methods in isolation.

At some point, old events and new ECS events will probably stop being equivalent to each other. At that point we'd probably want a version of mockProcessEvent that returns the new style as well. At that point, if we wanted to be exhaustive, we could make a version of the mockProcessEvent that returned new ECS events and then run these tests w/ those objects as well.

Does that all add up?

Since most of these tests are for code paths that occur after we normalize unique_pid/entity_id I don't think it's super important now

Sounds good, just wanted to double check 👍

jonathan-buttner · 2020-06-29T13:48:32Z

x-pack/plugins/security_solution/public/resolver/store/data/reducer.test.ts

+ beforeEach(() => {
+ const generator = new EndpointDocGenerator('seed');
+ const tree = mockResolverTree({
+ events: generator.generateTree({ ancestors: 1, generations: 2, children: 2 }).allEvents,


@bkimmel don't know if it matters but the generated tree will have parents with up to 2 children (randomly generated). If you always want the nodes to have exactly 2 children you can pass in alwaysGenMaxChildrenPerNode: true

I think we should do that

jonathan-buttner · 2020-06-29T13:49:51Z

x-pack/plugins/security_solution/public/resolver/store/data/reducer.ts

@@ -58,3 +89,5 @@ export const dataReducer: Reducer<DataState, ResolverAction> = (state = initialS
 return state;
 }
 };
+
+// TODO, handle abort scenario


Do we need to handle this scenario in this PR?

abort is handled. TODO comment should be removed. ty

jonathan-buttner · 2020-06-29T13:54:23Z

x-pack/plugins/security_solution/public/resolver/store/middleware/resolver_tree_fetcher.ts

+ result = await context.services.http.get(`/api/endpoint/resolver/${entityIDToFetch}`, {
+ signal: lastRequestAbortController.signal,
+ query: {
+ children: 5,


Hopefully we can bump these number up after I do some performance testing.

jonathan-buttner · 2020-06-29T14:09:43Z

x-pack/plugins/security_solution/server/endpoint/routes/resolver/entity.ts

+ bool: {
+ filter: [
+ {
+ match: {


I think we can switch this to an IDs search: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-ids-query.html

@andrew-goldstein the _id that we're passing should be the unique ES document ID (i.e. _id field right) right?

I'll give this a shot

worked just fine.

jonathan-buttner · 2020-06-29T14:11:12Z

x-pack/plugins/security_solution/server/endpoint/routes/resolver/entity.ts

+ };
+ }
+
+ const queryResponse: ExpectedQueryResponse = await context.core.elasticsearch.legacy.client.callAsCurrentUser(


Should we try/catch around this? If ES fails for some reason maybe the server should log and return a 500?

Let me look into this.

As the code is written now:

Logs in Kibana:

And the response was: {"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}

This seems fine to me. The internal details are logged, while the client gets a generic 500.

👍 Sounds good, thanks for looking.

michaelolo24 · 2020-06-29T14:51:28Z

x-pack/plugins/security_solution/public/resolver/models/resolver_tree.ts

+ * ResolverTree type is returned by the server. It organizes events into a complex structure. The
+ * organization of events in the tree is done to associate metadata with the events. The client does not
+ * use this metadata. Intsead, the client flattens the tree into an array. Therefore we can safely
+ * make a malformed RelsoverTree for the purposes of the tests, so long as it is flattened in a predicatable way.


🤷 Resolver spelled wrong in the comment

Intsead and predicatable too. (i turned on spellcheck lol)

michaelolo24 · 2020-06-29T15:06:34Z

x-pack/plugins/security_solution/public/resolver/store/data/selectors.test.ts

+ * This prints out all of the properties of the data state.
+ * This way we can see the overall behavior of the selector easily.
+ */
+ const viewAsAString = (dataState: DataState) => {


kibanamachine · 2020-06-29T16:53:29Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: b2e698e

Build metrics

✅ unchanged

History

💚 Build #57124 succeeded 435f2e5
💔 Build #57109 failed 7cc4b6b
💔 Build #57076 failed d045a37
💔 Build #56936 failed b958bbb
💔 Build #56935 failed c1d8bd5

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Display Resolver in Security Solution's Timeline.

This comment has been minimized.

Sign in to view

oatkiller force-pushed the resolver-in-timeline branch from 4ab8422 to 902fa3b Compare June 24, 2020 16:44

oatkiller commented Jun 24, 2020

View reviewed changes

x-pack/plugins/security_solution/public/timelines/components/timeline/body/helpers.ts Outdated Show resolved Hide resolved

oatkiller commented Jun 24, 2020

View reviewed changes

x-pack/plugins/security_solution/public/resolver/store/middleware/resolver_tree_fetcher.ts Outdated Show resolved Hide resolved

oatkiller commented Jun 24, 2020

View reviewed changes

x-pack/plugins/security_solution/public/resolver/store/data/selectors.ts Outdated Show resolved Hide resolved

oatkiller commented Jun 24, 2020

View reviewed changes

x-pack/plugins/security_solution/public/resolver/store/data/selectors.ts Outdated Show resolved Hide resolved

oatkiller commented Jun 24, 2020

View reviewed changes

x-pack/plugins/security_solution/public/resolver/store/data/selectors.ts Outdated Show resolved Hide resolved

bkimmel reviewed Jun 24, 2020

View reviewed changes

oatkiller force-pushed the resolver-in-timeline branch 4 times, most recently from c6d911d to 08bfddb Compare June 25, 2020 13:13

oatkiller mentioned this pull request Jun 25, 2020

Rename Resolver types to include 'Resolver' #69926

Merged

7 tasks

oatkiller force-pushed the resolver-in-timeline branch 7 times, most recently from b9756e8 to 627d5f8 Compare June 26, 2020 19:40

oatkiller commented Jun 26, 2020

View reviewed changes

oatkiller force-pushed the resolver-in-timeline branch 4 times, most recently from e22f482 to 1d88a9e Compare June 26, 2020 21:23

oatkiller marked this pull request as ready for review June 26, 2020 21:44

oatkiller requested review from a team as code owners June 26, 2020 21:44

oatkiller added release_note:enhancement v7.9.0 v8.0.0 labels Jun 26, 2020

oatkiller requested a review from kqualters-elastic June 26, 2020 21:52

kqualters-elastic approved these changes Jun 26, 2020

View reviewed changes

delete tests for old page

c1d8bd5

elasticmachine and others added 3 commits June 27, 2020 08:22

Merge branch 'master' into resolver-in-timeline

b958bbb

move types around to get around another false positive in the circula…

d045a37

…r deps check script we run.

fix import errors

c3e0623

oatkiller added the Team:Endpoint Data Visibility Team managing the endpoint resolver label Jun 29, 2020

oatkiller added 2 commits June 29, 2020 09:21

more type fixes

7cc4b6b

more type fixes

435f2e5

jonathan-buttner reviewed Jun 29, 2020

View reviewed changes

michaelolo24 reviewed Jun 29, 2020

View reviewed changes

oatkiller added 3 commits June 29, 2020 11:15

cleanup comment spelling

536c4c9

remove bad comment

3ec37fd

use ids instead of match to search for _id

b2e698e

jonathan-buttner approved these changes Jun 29, 2020

View reviewed changes

michaelolo24 approved these changes Jun 29, 2020

View reviewed changes

oatkiller merged commit 8ffdd45 into elastic:master Jun 29, 2020

oatkiller deleted the resolver-in-timeline branch June 29, 2020 17:10

oatkiller mentioned this pull request Jun 29, 2020

[7.x] [Security Solution] Resolver in Timeline (#69728) #70242

Merged

oatkiller pushed a commit to oatkiller/kibana that referenced this pull request Jun 29, 2020

[Security Solution] Resolver in Timeline (elastic#69728)

bc401cd

Display Resolver in Security Solution's Timeline.

oatkiller pushed a commit that referenced this pull request Jun 29, 2020

[Security Solution] Resolver in Timeline (#69728) (#70242)

da0f779

Display Resolver in Security Solution's Timeline.

Bamieh pushed a commit to Bamieh/kibana that referenced this pull request Jul 1, 2020

[Security Solution] Resolver in Timeline (elastic#69728)

efee168

Display Resolver in Security Solution's Timeline.

Resolver in timeline #69728

Resolver in timeline #69728

Conversation

oatkiller commented Jun 23, 2020 • edited Loading

Summary

Screenshots

To fix before merge

Follow up work

API integration test plan

Checklist

For maintainers

This comment has been minimized.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oatkiller left a comment

Choose a reason for hiding this comment

This comment was marked as resolved.

This comment was marked as resolved.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elasticmachine commented Jun 26, 2020

oatkiller commented Jun 27, 2020

elasticmachine commented Jun 29, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oatkiller Jun 29, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kibanamachine commented Jun 29, 2020

💚 Build Succeeded

Build metrics

History

oatkiller commented Jun 23, 2020 •

edited

Loading

oatkiller Jun 29, 2020 •

edited

Loading