Discuss Metricbeat input-like metricsets mapping behavior

[exekias]

Several Metricbeat metricsets can be used as an input to create new lightweight modules. We should think about how do we want "input" metricsets to work when related to mapping the metrics they retrieve. I can see two important features to watch for:

Namespacing

How are metrics organised within output documents? How do I know what module/metricset some metrics belong to (ie, give me all metrics you have on MySQL)? Inputs doing namespacing put metrics under some common root key, related to the module/metricset. This has the benefit of making them easy to discover when using typeahead (for instance in metrics explorer).

When we don't do namespacing, we can still use the metrics, but may need previous filtering based on some labels (for instance filter on event.dataset).

Field names

Field names can sometimes be decided automatically. When this is the case, the input provides a dynamic mapping covering all possible cases. This has the benefit for the developer where we don't really need to write a new fields.yml for a new module. It also adapts to metric changes in the source without updates to the module.

On the down side, because we have a generic mapping, we do not set formatting settings for the different metrics, letting that as a decision to do when building visualizations.

Still, this is not always possible, or would be too cumbersome, in those cases the developer is still defining how to map source metrics to the names we will finally push to Elasticsearch. Jolokia/JMX is a good example of this.

This is the current behavior of our input type metricsets:

input	namespacing	field names
graphite	namespace setting	auto
http	namespace setting	auto
jolokia	namespace setting	dev maps them
statsd	no namespacing *	auto
aws cloudwatch	no namespacing	auto
prometheus	no namespacing *	auto

• this is normally part of the source metric names

[exekias]

Related to #13316 and #14653

[jsoriano]

I think we should continue removing namespace as we did for some modules as prometheus and munin. As mentioned, there are some ECS fields like event.dataset or service.name that can serve for similar purposes. If we implement #14742 we could still rename fields that make more sense in an specific namespace.

#14742 would also add the possibility to define additional mappings using processors to any module, without needing specific implementations like the one in jolokia.

If we find that current processors are not enough, we could consider exposing the schema package as a processor.

[exekias]

I'm worried about discover-ability of metrics for an integration if we go all in there. The way things work as of now in the Metrix Explorer, user will start typing the module name (ie, tomcat.) to see what metrics are available under there. What can we do about it if they are under jolokia.*?

On the other side, for some metrics that are common to a group of modules, like for instance, JVM internal metrics, I think it would make sense for them to share the same names. So far this is the only case where I see a fit for common names not tied to the module namespace.

[jsoriano]

Yeah, I thought that discoverability from Metrics Explorer was something that would need to be improved in any case when thinking on metrics that are common to several services (metrics in ECS, runtime metrics as the ones for Go or the JVM and so on). But this is less important if we keep namespacing.

[ruflin]

event.dataset describes which data structure is there so I think we should rely more on it. Perhaps we need a second "select" in the Metrics UI to either start typing on the fields or the event.dataset? So if a user types tomcat. he sees all the jolokia.* fields.

[exekias]

That's a good point, definitely this would be less of a problem if UIs were "dataset aware". That said, this not only applies to Metrics UI, it's also about TSVB and all other visualizations, plus machine learning, alerting etc.

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.