[ML] Improve sampling and normalization of population chart.

[walterra]

Summary

This PR optimizes how contextual data is fetched for the population analysis chart. The previous structure of nested aggregations to get the data was like this:

byTime (date histogram) -> sample (sampling on each time slice) -> entities

This is now changed to the following structure:

sample (sample on whole dataset) => byTime (date histogram) -> entities

• Previously the sampling was done inside each time slice. That means the longer the time range queried, the more sampling would have been done. The updated version does the sampling in the outer most aggregation. This makes the sampling and performance more predictable.
• Additionally, a random document score is now used to get a more random sample of documents. Sampling takes the top N documents based on their score. Because the previous query was a bool query with must clauses only, all documents had the same score.
• And finally, another optimization is the normalization of values for metric functions based on count and sum. In contrast to functions like mean, the results of the former could be heavily skewed because of the sampling. The normalization adjusts the values to take into account the total amount of documents without sampling.

The screenshots above show results before (left) and after (right) the optimization.

• The currently used sampling size is now hard coded at 50000, the examples above are just to give you an impression how the sampling and normalization work before and after.
• The left versions did the sampling for each time slice so the results look better for lower sampling values, but the queries were also much heavier.
• Looking at the left versions, you can see that the results for the contextual data grow with higher sampling, that's the result of the lack of normalization.
• The right side with sampling of 200 show very limited results, because the sampling is now applied to the outer aggregation. However, you can already notice the normalization kicking in with some dots showing up higher than any in the left version.
• For comparison, the center bottom chart is done without sampling.

Checklist

Checklist not applicable (existing tests pass, no DOM changes).

---

Part of #21163.

[elasticmachine]

Pinging @elastic/ml-ui

[elasticmachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request

[peteharverson]

Looks like you now need to use a field parameter here as well as seed to get reproducible results - see https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#function-random

[walterra]

Thanks for pointing that out, I was working off some outdated docs. Push an update in 43ced43.

[alvarezmelissa87]

This LGTM overall. 👍 I'm just waiting on responses to Pete's comments since I have less context on that bit.

[alvarezmelissa87]

LGTM

[peteharverson]

LGTM

[elasticmachine]

💔 Build Failed

• continuous-integration/kibana-ci/pull-request

[walterra]

retest

[elasticmachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request