Ability to use kubernetes hints and custom metricsets

[jguay]

Describe the enhancement:
With the current implementation user must choose between pre-defined module setup and his custom setup defined in the raw hint.

It would be nice if the following could be implemented:

metricbeat.autodiscover:
 providers:
   - type: kubernetes
     hints.enabled: true
     hints.combination.enabled: true

"hints.combination.enabled: true" would mean if a module has multiple metricset defined, metricbeat processes metricset(s) defined in metricbeat config, then it processes metricset(s) defined in the raw hint.
Describe a specific use case for the enhancement or feature:
Users want to :

• subscribe to pre-defined collections
• and add their own custom metric

[adelbot]

Nobody is interested in this functionality that would allow to simultaneously static configs and dynamic configs?

[jsoriano]

We will have to think more about how to support this. At the moment there are several parts involved in generating configuration for beats when using hints-based autodiscover: there can be configuration in files, in autodiscover templates and in autodiscover hints. It is sometimes confusing to know what configuration is being applied when these methods are mixed in the same deployment, so we usually recommend that only one of the methods is used if possible.

The proposed solution would be really useful, but it would add another configuration method, in my opinion we should try to avoid that.

Our mid/long-term idea for hints is that when using them, everything about modules (metricsets, filesets...) can be configured via annotations (or labels in docker, or similar things in other technologies). This allows to configure observation of dynamic services without needing to touch the configuration of the beat itself, what for example in kubernetes allows to separate responsibilities between people managing the cluster and people deploying services on it.

So we'd prefer to add configuration options by the use of hints before adding another method that relies on beats configuration, some options I can think now could be:

• Increase the number of settings exposed by hints, so the raw hint is needed with less frequency
• Allow to define hints in namespaces, this would allow to define namespace-wide settings or defaults
• Allow to reference config maps from hints, to load defaults, parameters or more complex configurations
• Allow to add autodiscover templates (conditional configs) from namespace annotations or config maps

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[botelastic]

This issue doesn't have a Team:<team> label.