Skip to content

Latest commit

 

History

History
107 lines (90 loc) · 4.78 KB

README.md

File metadata and controls

107 lines (90 loc) · 4.78 KB
Raw

Kafka Metrics Receiver

Status
Stability beta: metrics
Distributions contrib
Issues Open issues Closed issues
Code Owners @dmitryax

Kafka metrics receiver collects kafka metrics (brokers, topics, partitions, consumer groups) from kafka server, converting into otlp.

Prerequisites

This receiver supports Kafka versions:

  • 2.X
  • 3.X

Getting Started

Required settings (no defaults):

  • protocol_version: Kafka protocol version
  • scrapers: any combination of the following scrapers can be enabled.
    • topics
    • consumers
    • brokers

Metrics collected by the associated scraper are listed here

Optional Settings (with defaults):

  • cluster_alias: Alias name of the cluster. Adds kafka.cluster.alias resource attribute.
  • brokers (default = localhost:9092): the list of brokers to read from.
  • resolve_canonical_bootstrap_servers_only (default = false): whether to resolve then reverse-lookup broker IPs during startup.
  • topic_match (default = ^[^_].*$): regex pattern of topics to filter on metrics collection. The default filter excludes internal topics (starting with _).
  • group_match (default = .*): regex pattern of consumer groups to filter on for metrics.
  • client_id (default = otel-metrics-receiver): consumer client id
  • collection_interval (default = 1m): frequency of metric collection/scraping.
  • initial_delay (default = 1s): defines how long this receiver waits before starting.
  • auth (default none)
    • plain_text
      • username: The username to use.
      • password: The password to use
    • tls
      • ca_file: path to the CA cert. For a client this verifies the server certificate. Should only be used if insecure is set to true.
      • cert_file: path to the TLS cert to use for TLS required connections. Should only be used if insecure is set to true.
      • key_file: path to the TLS key to use for TLS required connections. Should only be used if insecure is set to true.
      • insecure (default = false): Disable verifying the server's certificate chain and host name (InsecureSkipVerify in the tls config)
      • server_name_override: ServerName indicates the name of the server requested by the client in order to support virtual hosting.
    • kerberos
      • service_name: Kerberos service name
      • realm: Kerberos realm
      • use_keytab: Use of keytab instead of password, if this is true, keytab file will be used instead of password
      • username: The Kerberos username used for authenticate with KDC
      • password: The Kerberos password used for authenticate with KDC
      • config_file: Path to Kerberos configuration. i.e /etc/krb5.conf
      • keytab_file: Path to keytab file. i.e /etc/security/kafka.keytab
      • disable_fast_negotiation: Disable PA-FX-FAST negotiation (Pre-Authentication Framework - Fast). Some common Kerberos implementations do not support PA-FX-FAST negotiation. This is set to false by default.

Examples:

  1. Basic configuration with all scrapers:
receivers:
  kafkametrics:
    protocol_version: 2.0.0
    scrapers:
      - brokers
      - topics
      - consumers
  1. Configuration with more optional settings:

For this example:

  • collection interval is 5 secs.
receivers:
  kafkametrics:
    cluster_alias: kafka-prod
    brokers: 10.10.10.10:9092
    protocol_version: 2.0.0
    scrapers:
      - brokers
      - topics
      - consumers
    auth:
      tls:
        ca_file: ca.pem
        cert_file: cert.pem
        key_file: key.pem
    collection_interval: 5s