We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Describe the bug When we use Kafka target in Promtail, we can't use metrics stage due to the /metrics call failure with the following error.
14 error(s) occurred: * collected metric "my_http_request_duration" { label:<name:"service" value:"nova-api" > label:<name:"env" value:"dev" > histogram:<sample_count:3151 sample_sum:81.74437090000009 bucket:<cumulative_count:3051 upper_bound:0.1 > bucket:<cumulative_count:3130 upper_bound:0.5 > bucket:<cumulative_count:3130 upper_bound:1 > bucket:<cumulative_count:3139 upper_bound:2.5 > bucket:<cumulative_count:3151 upper_bound:5 > > } was collected before with the same name and label values
In kafka target, this ConsumeClaim function is called in every kafka partition. https://github.com/grafana/loki/blob/main/clients/pkg/promtail/targets/kafka/consumer.go#L86
ConsumeClaim
It causes the duplicate Register call as the following.
(c *consumer) func ConsumeClaim -> (t *targetSyncer) NewTarget -> stages.NewPipeline -> stages.New -> newMetrics
https://github.com/grafana/loki/blob/main/clients/pkg/logentry/stages/metrics.go#L127
To Reproduce Steps to reproduce the behavior:
scrape_configs: - job_name: dev-kafka kafka: use_incoming_timestamp: true version: 2.4.2 brokers: - xxxx group_id: promtail-dev topics: - grafana-loki-dev labels: env: dev pipeline_stages: - match: selector: '{env="dev"} |= "access_log"' stages: - json: expressions: request_time: - metrics: http_request_duration: type: Histogram prefix: my_ source: request_time description: "http request duration in API" max_idle_duration: 24h config: action: inc buckets: - 0.1 - 0.5 - 1.0 - 2.5 - 5.0
Expected behavior A clear and concise description of what you expected to happen.
The text was updated successfully, but these errors were encountered:
Let me fix this.
Sorry, something went wrong.
Successfully merging a pull request may close this issue.
Describe the bug
When we use Kafka target in Promtail, we can't use metrics stage due to the /metrics call failure with the following error.
In kafka target, this
ConsumeClaim
function is called in every kafka partition.https://github.com/grafana/loki/blob/main/clients/pkg/promtail/targets/kafka/consumer.go#L86
It causes the duplicate Register call as the following.
https://github.com/grafana/loki/blob/main/clients/pkg/logentry/stages/metrics.go#L127
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen.
The text was updated successfully, but these errors were encountered: