[Security Solution] Custom dashboard UI prerequisite

[angorayc]

To be compatible with custom dashboards, we'd like to migrate existing charts to Lens Embeddables.
Apart from adaptable with custom dashboards, users can have aligned experience when interacting with charts in security solution and Lens, means that without additional implementation, we can access the inspect, open in dev tools ...etc actions.
Therefore we can remove many of our existing implantation and make this easier to maintain.

These are all behind feature flag chartEmbeddablesEnabled. The feature flag is expected to be lifted in 8.8

Tasks

Migration to Lens Embeddable

• Allow using custom inspector - [Security Solution] Add custom features to lens #138995 , [SecuritySolution] Replace KPIs on Host, Network, and Users with Lens Embeddables #138369 - v8.6

• UI Style adjustment for Metric - [Security Solution] Add custom features to lens #138995 , [SecuritySolution] Replace KPIs on Host, Network, and Users with Lens Embeddables #138369 - v8.6

• Add custom legend actions to Lens - v8.7 - [Security Solution][Lens] New trigger actions for chart legends and table cell actions #146779

• Bug: [Lens] Table shows extra rows when embedded on a dashboard - 8.7 - [Lens] Table shows extra rows when embedded on a dashboard #147922

Feature request on Lens' side

---

1. Donut size with the legend

• The size of the donut should be the same with or without a legend.

Example

When we display the donut Lens Embeddable without legend it looks alright.

But after enabling the legend, the donut becomes very small and it's no longer at the center of the space.

Expected

Could we have an option to overlap the legend over the donut so the size of the donut could remain the same with or without the legend. (Like we can choose to display legend inside the bar chart)

---

2. Donut title - To be followed up

• [Lens] An option to display title / subtitle for charts #149675 - To be followed up

Expected:

This is what we have in Security Solution:

If that's not an option, could we consider apply the title / subtitle like this to the donut chart?

---

3. Donut click custom actions

• [Lens][Embeddable] Provide a way to override default action on trigger #149027 - 8.8

Use case
Navigate to a new page on clicking a portion of the donut.

---

4. Query without time-range Can be achieved by adHocDataView without timestamp

• Could we support query without a time range?

Use case
SecuritySolution has some charts based on Transformed result, for example: displaying the latest value. The value has been calculated by Transform, so shouldn't worry about the performance when no timerange given.

---

5. Show legend items with zero value

• It doesn't show the legend item if its value is zero #149681 - 8.8

Example:

The current legend items in this donut should be Critical, High, Medium, and Low, but all bucket value is zero except Low, so there's only Low appears in the legend.

Expected:
The Legend show all the buckets, Critical, High, Medium, Low even the bucket value is zero, and each bucket stick to it's color and order, color and order shouldn't change because any of the buckets is zero

---

6. Color and sort filter series

Could we choose a custom color for each filter and render the legend with the same order as the filters here?

• [Lens] Keep the filters order in the partition chart legend #149034 - 8.8
• [Lens] Support categorical coloring by name #101942 - 8.8 - Follow up in Legend order does not match with query result #154533

Example

Expected

---

7. Always show values on legend items - To be followed up

• An option to always display the value on each legend item #149676 - To be followed up

Example
At the moment, no value show on legend item by default

Value of each legend item only show on hovering the chart:

Expected
Could we show the sum of the bucket by default, and only if users hovering on the chart, it display the value of the selected bucket.

---

8. Display count on treemap legend - To be followed up

• Display count on tree map legend #149678 - To be followed up

Example

Expected

Display doc count in the legend.

---

9. An option to disable legend actions by passing a prop to Lens Embeddable

• Disable legend actions by passing a prop to Lens Embeddable #149220 - 8.8

---

10. Color palette by Metric - To be followed up

• Decide Color Palette by Metric for Treemap #149679 - To be followed up

Expected

---

11. Display multiple metric on treemap - To be followed up

• Display multiple metric on tree map #149680 - To be followed up

Expected

After putting everything together, ideally it'll look like this:

12. Icon decoration is cropped - added24th Jan 2023

• Annotation Icon is cropped on reference lines #149414

Replace visualisations with Lens Embeddable

• Replace KPI with Lens Embeddable - Hosts, Network, Users - [SecuritySolution] Replace KPIs on Host, Network, and Users with Lens Embeddables #138369

• Replace histograms with Lens Embeddable, including events, alerts, and rule preview - v8.7 - [SecuritySolution] Render histograms with Lens #147261

• Replace alerts donut charts with Lens Embeddable - v8.7 - [SecuritySolution] Replace donuts with Lens #148519 (Not included in 8.8 because of the known issues 2-6 above)

• Replace risk score donut charts with Lens Embeddable - v8.7 - [SecuritySolution] Replace risk scores donut charts with Lens #148755 (Not included in 8.8 because of the known issues 2-6 above)

• Replace host / user risk score over time with Lens Embeddable - 8.7 #149015 - 8.7 - [SecuritySolution] Replace risk score over time with Lens Embeddable #149035

• [SecuritySolution] Designer's review and redesign on migration to Lens Embeddables #149123 - 8.8

• [SecuritySolution] Enable chartEmbeddablesEnabled feature flag #150531 - 8.8

• Unsupported charts - [SecuritySolution] Follow up unsupported charts #149592 - 8.9

• Remove feature flag & clean up - 8.9

Bugs:

• Bug: Page crashes after editing the filter applied from a Lens Embeddable - 8.7 - [SecuritySolution] Page crashes after editing the filter applied from a Lens Embeddable. #148710
• Bug: Page crashes once navigating out of Security Solution #149196 - 8.7
• Bug: [Security Solution] Resolve JS warnings triggered by incorrect state changed #148552 - 8.7
• Bug: [SecuritySolution] Fix data view not found when rendering with adHocDataView #149513 - 8.7
• Bug: https://github.com/elastic/security-team/issues/5817 - 8.7
• Bug: Open in timeline and copy to clip board cellActions have the wrong query in Lens Embeddable #150048 - 8.7
• Lens does not support nested fields, we have to exclude them from the dropdown #150158 - 8.7
• Reset group by field is missing when no data to display #149828 - 8.7
• [Security Solution] [Sourcerer] Sourcerer matched indices not picking up .alerts index on fresh kibana + ES #150818 - 8.8
• [SecuritySolution][Bug] KPI components do not update when time range changed #153782 - 8.8
• [SecuritySolution] Unnecessary cell action expand icon #152090
• [SecuritySolution] Fix DNS histogram query #154548 - 8.8

============================================================

Migration to generic inspector

• Add Index Pattern to generic inspector - Show index pattern in Inspector #136407 - v8.4
• Refactoring existing search strategy to useSearchStrategy hook - Migrate explore team API calls to useSearchStrategy. #129054 - v8.5
• Replace custom inspector with Lens inspector - inspector plugins #135400 - [Pending]

[elasticmachine]

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[angorayc]

The target for 8.6 is to replace the KPIs on hosts, network and users page with Lens Embeddables. This feature will be hidden behind a feature flag until we are ready.

[angorayc]

Here is an example of how it may look after migrating most of the charts to Lens: https://kibana-pr-148624.kb.us-west2.gcp.elastic-cloud.com:9243/s/data/app/security/overview

https://p.elstc.co/paste/N8VOV4gE#4yoL5fiVG+8F8BeTCV7GP77LKSBXq2MU5pOatGUbb2y