[ML] Anomaly Explorer charts

[walterra]

Tracking issue for the implementation of a custom charts for detector functions rare, over, lat_long.

Investigate

• sample data and job configurations
• stand-alone prototype
• prototypish initial anomaly explorer implementation

Implementation

Phase 1 #23423

• select correct anomalies
• filter by partition field
• sorted by rarity
• implement sampling
• optimize legends/title [ML] Improve Explorer Chart labels. #23494
• fix flex based layout 829016e
• feature flag 832e21b
• add hint about context data (explain that it's a sampled preview) 0625fcd
• fix/update tests
• benchmarking (https://gist.github.com/walterra/5e249719c15b5a83f70e0d60557a198c)
• fix error when results length is 0
• fix badge overflow breaking layout 9537d87
• show rare chart only if no over field is used in the detector for now 84d85a9
• fix blank rare charts [ML] Anomaly Explorer Rare/Population Charts #23423 (comment) — Problem identified: When the time range shown in the chart is smaller than the time range covered by a swimlane cell, a rare anomaly could actually be outside the covered time range. This is actually also a problem on master even without PR 1 so not a problem introduced by the rare chart itself. 2f75c7f
• investigate mapping anomalies to dots in population chart [ML] Anomaly Explorer Rare/Population Charts #23423 (comment) — that screenshot is from a detector using over and by, the by field can have an influencer on the outcome and a total high count could be less anomalous than a lower count one.
• Say 'rarer' instead of 'more rare'. a1acf52
• fix undefined in tooltip [ML] Anomaly Explorer Rare/Population Charts #23423 (comment) 5f5ae7a
• Show typical value in tooltips for anomalies with over but no by field or only 1 numberOfCauses. 4e2b015
• Move entity value in distribution chart tooltip just below date above other info. 001f328

Phase 2

• use arrow functions [ML] Anomaly Explorer Rare/Population Charts #23423 (comment) — c60385f
• support for more detector functions for population charts using over fields — 33cbdb3
• optimize sampling and normalize skewed results from count and sum based aggregations. — [ML] Improve sampling and normalization of population chart. #24402
• add a y-axis description to chart header for rare charts [ML] Fixes missing y-axis description for rare chart. #24823
• for rare charts with low cardinality (e.g. status codes or http methods in contrast to URLs or IPs) category on the y axis could be displayed. [ML] Anomaly Explorer Rare/Population Charts #23423 (comment) - done in [ML] Display a ordinal y axis for low cardinality rare charts. #24852
• fix an issue where screen width would expect 3 columns but only 1-2 charts are available
• fix flexbox/chartwidth overflow issue [ML] Anomaly Explorer Rare/Population Charts #23423 (comment)
• fix axis label issue [ML] Anomaly Explorer Rare/Population Charts #23423 (comment)
• make SAMPLER_TOP_TERMS_SHARD_SIZE a function argument [ML] Anomaly Explorer Rare/Population Charts #23423 (comment)
• display both entity-field-name and entity-field-value in tooltip (PR1 shows value only)

Phase 3 / future work / nice to have

• support for freq_rare https://www.elastic.co/guide/en/x-pack/current/ml-rare-functions.html#ml-freq-rare
• support for rare/freq_rare plus over fields
• consolidate explorer charts code, remove redundancies
• Use EuiDescriptionList instead of mlDescriptionList styles. [ML] Anomaly Explorer Rare/Population Charts #23423 (comment)
• Move individual client side request to its own summarized Kibana API endpoint [ML] Anomaly Explorer Rare/Population Charts #23423 (comment)
• display a distribution chart for population detectors which use a script field defined in the datafeed configuration, or when the function does not map to an ES aggregation (e.g. varp, info_content) if model plot is enabled. Currently a metric style chart is displayed in these cases.
• Migrate to TypeScript

---

---

Part of #22922.

[elasticmachine]

Pinging @elastic/ml-ui