Random sampler aggregation support in Lens

[ruflin]

The random sampler aggregation can help significantly improve query performance for metric aggregations in some scenarios (see for more details: elastic/elasticsearch#91743 ). Unfortunately it is not possible to use it at the moment as there is not support for it in Lens.

As can be seen in elastic/elasticsearch#91743 there are quite a few discussions needed on how exactly the random sampler aggregation would work in Lens, what default it picks and how. Lets use this issue to do that.

It seems as part of the ML feature, random sampler is already supported (#146323, #146160) but we require it to be part of Lens so we can use it in our dashboards.

Related issues

• Constant query time for (some) metrics aggregations elasticsearch#91743
• [ML] Explain Log Rate Spikes: Random sampler support #146160
• [ML] Index data visualizer - wrong initial label on sampling percentage slider #146323

[felixbarny]

One idea discussed in the linked Elasticsearch issue is to progressively load the charts: We'd start with a very low sampling percentage, render the graph, and do another query if the error from the first query is too high. For the second query, we'll calculate the probability, given the response of the first query, so that the error is kept within a reasonable bound.

[elasticmachine]

Pinging @elastic/kibana-visualizations @elastic/kibana-visualizations-external (Team:Visualizations)

[ruflin]

For reference, turns out some work had been already done on this: #143221

[stratoula]

Hey! Yes we had started working on that but we decided to not release it because there were more details that needed to be discussed. @ninoslavmiskovic is on it, gathering all the requirements and talking with stakeholders.

[dej611]

This has been finally enabled in Lens with #151749 resolving all raised concerns from the ML team.