Infer types in Prometheus remote_write API

[exekias]

Describe the enhancement:

We introduced different type behavior (use_types, rate_counters) when using Prometheus collector in 7.7. It would be nice to offer something similar for metrics coming through remote_write, for consistency and users profit.

Remote write API schema is typeless so we would need some heuristics to detect the original type where needed. For instance:

• Metrics ending up in _bucket with a le label are part of a histogram.
• Metrics matching *_total are counters. We could make this a configurable regexp list, so users can tweak it to its needs.

[elasticmachine]

Pinging @elastic/integrations-platforms (Team:Platforms)

[exekias]

would love your thoughts here @ChrsMark

[ChrsMark]

Sounds good @exekias!

Only note here that this extra "compute" step might have an impact in the performance of remote endpoint. In this we might add an option to disable this type casting if users do not actually need it.

Wondering regarding Point 2 if it would be useful in cases with a lot of timeseries? Would the users be willing to customize that much?

[exekias]

Only note here that this extra "compute" step might have an impact in the performance of remote endpoint. In this we might add an option to disable this type casting if users do not actually need it.

Right now collector metricset only has this behavior if enabled by the user (use_types and rate_counters, both disabled by default). So if we match that this will still be optional.

Wondering regarding Point 2 if it would be useful in cases with a lot of timeseries? Would the users be willing to customize that much?

tbh I'm not sure, I would expect a regexp to be fast enough