Store various metrics (sum,count,etc) as aggregate_metric_double

[dgieselaar]

In 7.11 Elasticsearch added an aggregate_metric_double field type:

Stores pre-aggregated numeric values for metric aggregations. An aggregate_metric_double field is an object containing one or more of the following metric sub-fields: min, max, sum, and value_count.

We could consider doing this for breakdown and span metrics as well. Storage cost might be lower but selfishly/more importantly, it means we can just use an avg aggregation on a field rather than a sum + count aggregation and some Kibana-side post-processing:

When you run certain metric aggregations on an aggregate_metric_double field, the aggregation uses the related sub-field’s values. For example, a min aggregation on an aggregate_metric_double field returns the minimum value of all min sub-fields.

This field type is also used by rollups I think.

See https://www.elastic.co/guide/en/elasticsearch/reference/master/aggregate-metric-double.html

[axw]

👍

I'd like to introduce explicit types for metrics, one of which would be "summary" ("aggregate"?) or similar, which would enable bundling these together. We probably won't be able to change existing metrics until 8.0 though.

[axw]

We should also consider setting _doc_count for these metrics. Although we can answer the question with value_count, we should see if adding _doc_count would improve performance (reduce the need for additional aggs?) or otherwise reduce tech debt in the UI code.

[axw]

I think we should defer this a bit longer (post 8.0), until Elasticsearch TSDB work has firmed up a bit more. I'd like to avoid having to migrate twice, if there are any fundamental changes to summary types.