Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs]Timeline and Template UI updates #84

Merged
merged 7 commits into from
Aug 4, 2020
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file modified docs/siem/images/timeline-ui.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 4 additions & 0 deletions docs/siem/index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,8 @@ include::detections/machine-learning/ml-index.asciidoc[]

include::detections/detections-index.asciidoc[]

include::timeline/timeline-ui-overview.asciidoc[]

include::timeline/timeline-templates.asciidoc[]

include::cases/cases-index.asciidoc[]
7 changes: 0 additions & 7 deletions docs/siem/siem-ui.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -233,7 +233,6 @@ The Cases page is used to open and track security issues directly in the
image::images/cases-ui-home.png[]

[float]
[[timelines-ui]]
== Timelines

Use Timeline as your workspace for alert investigations or threat hunting.
Expand All @@ -260,7 +259,6 @@ can also link to timelines from Cases and external ticketing systems.


[discrete]
[[raw]]
==== Focus on signals or raw events

Many security events in Timeline are presented in an easy-to-follow rendered
Expand All @@ -272,7 +270,6 @@ You can click and expand events from within Timeline to see the underlying
event data, either in tabular form, as as {es} JSON.

[discrete]
[[narrow-expand]]
==== Narrow or expand your query

You can specify logical `AND` and `OR` operations with an item's placement in
Expand All @@ -281,7 +278,6 @@ sets are `OR`-ed together. As you hover the item over the drop area, you can see
whether your placement is on target to create an `AND` or `OR` filters.

[discrete]
[[pivot]]
==== Pivot on a data point

Click a filter to access additional operations such as exclude, temporarily
Expand All @@ -290,7 +286,6 @@ item so that it is excluded.

[discrete]
[[row-renderer]]
==== Get more context for each event

As you build and modify your queries, you can see the results of your
interactions in the details pane below.
Expand All @@ -301,7 +296,6 @@ event. If you see a particular item that interests you, you can drag it to the
drop area for further introspection.

[discrete]
[[import-export-timelines]]
==== Export and import timelines

You can import and export timelines, which enables importing timelines from one
Expand All @@ -320,7 +314,6 @@ then select _Export selected_.
the timeline `ndjson` file.

[discrete]
[[other]]
==== Other actions

The Timeline is flexible and highly interactive. As you would expect, the
Expand Down
Binary file added docs/siem/timeline/images/create-template-ui.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/siem/timeline/images/timeline-ui.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
107 changes: 107 additions & 0 deletions docs/siem/timeline/timeline-templates.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
[[timeline-templates-ui]]
== About Timeline templates

You can attach Timeline templates to detection rules. When attached, the rule's
alerts use the template when they are investigated in Timeline. This enables
immediately viewing the alert's most interesting fields when you start an
investigation.

Templates can include two type of queries:

* *'Regular' queries*: These are like any other {kib} HQL queries. They define
both the source event field and its value. For example:
`host.name : "win-server"`.
* *Template fields*: These only define the event field. When you investigate
an alert in Timeline, the field's value is taken from the alert.

For example, if you define the `host.name: "{host.name}"` template field, when
alerts generated by the rule are investigated in Timeline, the alert's
`host.name` value is used in the query. If the alert's `host.name` value is
`Linux_staforshire-061`, the Timeline query is:
`host.name: "Linux_staforshire-061"`.

NOTE: For information on how to add Timeline templates to rules, see
<<create-rule-ui>>.

When you load {es-sec} prebuilt rules, some prebuilt Timeline templates are
also loaded. You can attach these templates to detection rules and use them as
the basis of your own custom templates. The following templates are loaded:

* `Generic Endpoint Timeline`: Useful for investigating Elastic Endpoint
security alerts.
* `Generic Process Timeline`: Useful for investigating process-related alerts.
* `Generic Network Timeline`: Useful for investigating network-related alerts.

[discrete]
=== Create a Timeline template

. Go to *Security* -> *Timelines*.
. Click the *Templates* tab, and then *Create new timeline template*.
+
[role="screenshot"]
image::images/create-template-ui.png[]

. Give the template a title.
. Optionally, add a description and notes.
. To add template fields and queries, click *Add field*, and then select the
required option:

* _Add field_: Add a query (field and field value).
* _Add template field_: Add a field without a predetermined value.

TIP: You can also drag and send items to the template from the *Overview*,
*Hosts*, *Network*, and *Detections* pages.

*Example*

To create a template for process-related alerts on a specific host:

* Add an 'ordinary' query for the host name:
`host.name: "Linux_staforshire-061"`
* Add template field for process names: `process.name: "{process.name}"`

[role="screenshot"]
image::images/template-query-example.png[]

When alerts generated by rules associated with this template are investigated
in Timeline, the host name is `Linux_staforshire-061`, whereas the process name
value is retrieved from the alert's `process.name` field.

[discrete]
=== Manage existing Timeline templates

You can view, duplicate, delete, and create templates from existing Timelines:

. Go to *Security* -> *Timelines*.
. Click the *Templates* tab.
. Click the More actions icon in the relevant row, and then select the action:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we briefly mention about what the More actions icon looks like and its position as well, many users couldn't find for the first time


* _Create timeline from template_
* _Duplicate template_
* _Delete template_

TIP: To perform the same action on multiple templates, select templates and
then the required action from the _Bulk actions_ menu.

NOTE: You cannot delete prebuilt templates.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

users cannot favourite it, or edit it either.
I think it would be nice to let them know that if they want to update the elastic prebuilt templates, they can duplicate them first and then they becomes custom templates, and they can do all the changes.


[discrete]
=== Export and import Timeline templates

You can import and export Timeline templates, which enables importing templates
from one {kib} space or instance to another. Exported templates are saved in an
http://ndjson.org[`ndjson`] file.

. Go to *Security* -> *Timelines*.
. Click the *Templates* tab.
. To export templates, do one of the following:

* To export one template, click the More actions icon in the relevant row and
then select _Export selected_.
* To export multiple templates, select all the required templates and then click
*Bulk actions* -> _Export selected_.

. To import templates, click *Import Timeline* and then select or drap-and-drop
the template `ndjson` file.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we put a note here to let them know the content of the ndjson has to be in minimised format to import it properly.

e.g.

This is ok

{"savedObjectId":"67664480-d191-11ea-ae67-4f4be8c1847b","version":"WzU1NSwxXQ==","columns":[{"indexes":null,"name":null,"columnHeaderType":"not-filtered","id":"@timestamp","searchable":null},{"indexes":null,"name":null,"columnHeaderType":"not-filtered","id":"message","searchable":null},{"indexes":null,"name":null,"columnHeaderType":"not-filtered","id":"event.category","searchable":null},{"indexes":null,"name":null,"columnHeaderType":"not-filtered","id":"event.action","searchable":null},{"indexes":null,"name":null,"columnHeaderType":"not-filtered","id":"host.name","searchable":null},{"indexes":null,"name":null,"columnHeaderType":"not-filtered","id":"source.ip","searchable":null},{"indexes":null,"name":null,"columnHeaderType":"not-filtered","id":"destination.ip","searchable":null},{"indexes":null,"name":null,"columnHeaderType":"not-filtered","id":"user.name","searchable":null}],"dataProviders":[],"description":"","eventType":"all","filters":[],"kqlMode":"filter","timelineType":"default","kqlQuery":{"filterQuery":{"serializedQuery":"{\"bool\":{\"should\":[{\"exists\":{\"field\":\"@timestamp\"}}],\"minimum_should_match\":1}}","kuery":{"expression":"@timestamp : * ","kind":"kuery"}}},"title":"x2","sort":{"columnId":"@timestamp","sortDirection":"desc"},"created":1596036895488,"createdBy":"angela","updated":1596491470411,"updatedBy":"angelachuang","templateTimelineId":null,"templateTimelineVersion":null,"dateRange":{"start":"2020-04-10T14:10:58.373Z","end":"2020-05-30T14:16:58.373Z"},"savedQueryId":null,"eventNotes":[{"noteId":"7d875ba0-d5d3-11ea-9899-ebec3d084fe0","version":"WzU1NiwxXQ==","eventId":"8KtMKnIBOS_moQ_K9fAe","note":"hi Xavier","timelineId":"67664480-d191-11ea-ae67-4f4be8c1847b","created":1596491490806,"createdBy":"angelachuang","updated":1596491490806,"updatedBy":"angelachuang"}],"globalNotes":[],"pinnedEventIds":["K99zy3EBDTDlbwBfpf6x","GKpFKnIBOS_moQ_Ke5AO","8KtMKnIBOS_moQ_K9fAe"]}

This will fail

{
  "savedObjectId": "67664480-d191-11ea-ae67-4f4be8c1847b",
  "version": "WzU1NSwxXQ==",
  "columns": [
    {
      "indexes": null,
      "name": null,
      "columnHeaderType": "not-filtered",
      "id": "@timestamp",
      "searchable": null
    },
    {
      "indexes": null,
      "name": null,
      "columnHeaderType": "not-filtered",
      "id": "message",
      "searchable": null
    },
    {
      "indexes": null,
      "name": null,
      "columnHeaderType": "not-filtered",
      "id": "event.category",
      "searchable": null
    },
    {
      "indexes": null,
      "name": null,
      "columnHeaderType": "not-filtered",
      "id": "event.action",
      "searchable": null
    },
    {
      "indexes": null,
      "name": null,
      "columnHeaderType": "not-filtered",
      "id": "host.name",
      "searchable": null
    },
    {
      "indexes": null,
      "name": null,
      "columnHeaderType": "not-filtered",
      "id": "source.ip",
      "searchable": null
    },
    {
      "indexes": null,
      "name": null,
      "columnHeaderType": "not-filtered",
      "id": "destination.ip",
      "searchable": null
    },
    {
      "indexes": null,
      "name": null,
      "columnHeaderType": "not-filtered",
      "id": "user.name",
      "searchable": null
    }
  ],
  "dataProviders": [],
  "description": "",
  "eventType": "all",
  "filters": [],
  "kqlMode": "filter",
  "timelineType": "default",
  "kqlQuery": {
    "filterQuery": {
      "serializedQuery": "{\"bool\":{\"should\":[{\"exists\":{\"field\":\"@timestamp\"}}],\"minimum_should_match\":1}}",
      "kuery": {
        "expression": "@timestamp : * ",
        "kind": "kuery"
      }
    }
  },
  "title": "x2",
  "sort": {
    "columnId": "@timestamp",
    "sortDirection": "desc"
  },
  "created": 1596036895488,
  "createdBy": "angela",
  "updated": 1596491470411,
  "updatedBy": "angelachuang",
  "templateTimelineId": null,
  "templateTimelineVersion": null,
  "dateRange": {
    "start": "2020-04-10T14:10:58.373Z",
    "end": "2020-05-30T14:16:58.373Z"
  },
  "savedQueryId": null,
  "eventNotes": [
    {
      "noteId": "7d875ba0-d5d3-11ea-9899-ebec3d084fe0",
      "version": "WzU1NiwxXQ==",
      "eventId": "8KtMKnIBOS_moQ_K9fAe",
      "note": "hi Xavier",
      "timelineId": "67664480-d191-11ea-ae67-4f4be8c1847b",
      "created": 1596491490806,
      "createdBy": "angelachuang",
      "updated": 1596491490806,
      "updatedBy": "angelachuang"
    }
  ],
  "globalNotes": [],
  "pinnedEventIds": [
    "K99zy3EBDTDlbwBfpf6x",
    "GKpFKnIBOS_moQ_Ke5AO",
    "8KtMKnIBOS_moQ_K9fAe"
  ]
}


NOTE: You cannot export prebuilt templates.
126 changes: 126 additions & 0 deletions docs/siem/timeline/timeline-ui-overview.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
[[timelines-ui]]
[role="xpack"]
= Investigating events in Timeline

Use Timeline as your workspace for investigations and threat hunting.
Data from multiple indices can be added to a Timeline, which enables
investigating complex threats.

You can drag or send items of interest to Timeline to create the query you need
to investigate an alert or event. You can add items from tables and histograms
on the *Overview*, *Detections*, *Hosts* and *Network* pages, as well as from
within a Timeline itself. Additionally, you can add a query directly in Timeline
(click `+ Add field`).

[role="screenshot"]
image::images/timeline-ui.png[]

Timelines are responsive, and they persist as you move through the {es-sec-app}
collecting data. Auto-saving ensures that the results of your investigation are
available for review by other analysts and incident response teams. To record
and share your findings with others, attach your Timeline to a
<<cases-overview, case>>.

TIP: An untitled Timeline is saved as a draft. To attach a Timeline to a
<<cases-overview, case>>, you must give it a title.

In addition to Timelines, you can create and attach Timeline templates to
<<detection-engine-overview, detection rules>>. Timeline templates allow you to
define the source event fields used when you investigate a detections alert in
Timeline. You can select whether the fields use predefined values or values
from the alert. For more information, see <<timeline-templates-ui>>.


[discrete]
[[refine-timeline-results]]
== View and refine Timeline results

Using the drop-down options
by the KQL bar, you can select whether <<det-engine-terminology, alerts>>,
other raw events, or both are displayed in the Timeline.

[discrete]
[[conf-timeline-display]]
== Configure Timeline event context and display

Many events in Timeline are presented in easy-to-follow rendered views that
include relevant contextual information. These views are called
*Event Renderers*, and they can be configured via the Settings icon in the upper
left corner of the result's pane:

[role="screenshot"]
image::images/timeline-ui-renderer.png[]

The example above displays a flow event renderer. If you see a particular item
that interests you, you can drag it to the dropzone for further investigation.

Other display options include:

* View in full screen mode
* Add, remove, reorder, and resize columns
* Add notes to individual events
* Add investigation notes for the entire Timeline
* Pin interesting events in the Timeline

[discrete]
[[narrow-expand]]
== Narrow or expand your query

You can specify logical `AND` and `OR` operations with an item's placement in
the drop area. Multiple horizontal filters use `AND` logic, vertical filters use
`OR`.

[discrete]
[[pivot]]
== Pivot on a data point

Click a filter to access additional operations such as exclude, temporarily
disable, or delete items from the query. For example, you can change an included
item so that it is excluded.

[discrete]
[[timeline-to-cases-ui]]
== Attach Timeline to a case

To attach a Timeline to a new or existing case, click the Settings in the upper
right corner, and then select one of these:

* _Attach timeline to new case_
* _Attach timeline to existing case_

For more information about cases, see <<cases-overview, Cases>>.

[discrete]
[[manage-timelines-ui]]
== Manage existing Timelines

You can view, duplicate, delete, and create templates from existing Timelines:

. Go to *Security* -> *Timelines*.
. Click the More actions icon in the relevant row, and then select the action:

* _Create template from timeline_ (see <<timeline-templates-ui>>)
* _Duplicate timeline_
* _Delete timeline_

TIP: To perform the same action on multiple Timelines, select Timelines and
then the required action from the _Bulk actions_ menu.

[discrete]
[[import-export-timelines]]
== Export and import Timelines

You can import and export Timelines, which enables importing Timelines from one
{kib} space or instance to another. Exported Timelines are saved in an
http://ndjson.org[`ndjson`] file.

. Go to *Security* -> *Timelines*.
. To export Timelines, do one of the following:

* To export one Timeline, click the More actions icon in the relevant row and
then select _Export selected_.
* To export multiple Timelines, select all the required Timelines and then click
*Bulk actions* -> _Export selected_.

. To import Timelines, click *Import Timeline* and then select or drap-and-drop
the Timeline `ndjson` file.