How to limit / filter kafka metrics exposed over Prometheus endpoint

[sgv0007]

Hi All ,

We need to filter or limit kafka metrics which we don't monitor but unable to find where to add the filter criteria . We have a third part app scraping metrics from prometheus endpoint and they want filter logic to be applied at kafka broker level as they don't want to do that after reading all the metrics .

Thanks,
Shiva

[scholzj]

it depends a bit what metrics are you talking about. You can configure the metrics exposed through the JMX Prometheus Exporter in the Config Map that configures it. For Kafka Exporter, you can configure some things such as topic or group filters in the Kafka CR.

[sgv0007]

Thanks @scholzj

If my understanding is correct then in kafka metrics config map , If I provide exact metric name instead of pattern it would emit only that particular metric right?

rules:

Special cases and very specific rules

• pattern : kafka_log_log_logstartoffset
  name: kafka_log_log_logstartoffset

[scholzj]

I'm not sure it is that simple as you are matching the JMX metrics and translating them into Prometheus metrics. But essentially yes, the rules there control what is exposed. So by replacing some of the wildcards with more specific values, you reduce the exposed metrics.

[sgv0007]

@scholzj I was able to filter the metrics for kafka and kafka connect using config map but not able to do that same for Kafka Exporter . Any other option apart from filtering topicnames and groups?

[scholzj]

I'm not aware of any other way to filter something. As far as I know you can drop some metrics on the Prometheus side as well when scraping them.