Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metrics of type Summary has been dropped #6988

Closed
paulojmdias opened this issue Feb 6, 2020 · 0 comments · Fixed by #6997
Closed

Metrics of type Summary has been dropped #6988

paulojmdias opened this issue Feb 6, 2020 · 0 comments · Fixed by #6997
Assignees
Labels
area/prometheus bug unexpected problem or unintended behavior
Milestone

Comments

@paulojmdias
Copy link
Contributor

paulojmdias commented Feb 6, 2020

Hi,

We are using MGOB (https://github.com/stefanprodan/mgob), and MGOB exposes metrics in Prometheus format natively. When we scrape directly from MGOB, the metrics mgob_scheduler_backup_latency (Type summary) are scrapped successfully, but when we use inputs.prometheus, these metrics are probably dropped.

We have metric_version = 2 in inputs.prometheus and outputs.prometheus. If we use metric_version = 1 in inputs.prometheus and outputs.prometheus, this issue won't occurs.

Relevant telegraf.conf:

[[inputs.prometheus]]
  urls = ["http://localhost:8090/metrics"]
  metric_version = 2
  [inputs.prometheus.tags]
      cluster_name = "mongodb-cluster"

System info:

  • Telegraf v1.13.2
  • Debian 9

Steps to reproduce:

  1. Get info from MGOB Exporter
curl -s http://localhost:8090/metrics | grep 'mgob_'

# HELP mgob_scheduler_backup_latency Backup duration in seconds.
# TYPE mgob_scheduler_backup_latency summary
mgob_scheduler_backup_latency{plan="mongodb-cluster",status="200",quantile="0.5"} NaN
mgob_scheduler_backup_latency{plan="mongodb-cluster",status="200",quantile="0.9"} NaN
mgob_scheduler_backup_latency{plan="mongodb-cluster",status="200",quantile="0.99"} NaN
mgob_scheduler_backup_latency_sum{plan="mongodb-cluster",status="200"} 4004.679645687
mgob_scheduler_backup_latency_count{plan="mongodb-cluster",status="200"} 1
# HELP mgob_scheduler_backup_size The size of backup.
# TYPE mgob_scheduler_backup_size gauge
mgob_scheduler_backup_size{plan="mongodb-cluster",status="200"} 1.9250215041e+10
# HELP mgob_scheduler_backup_total The total number of backups.
# TYPE mgob_scheduler_backup_total counter
mgob_scheduler_backup_total{plan="mongodb-cluster",status="200"} 1
  1. Get scrape from Telegraf
curl -s http://localhost:9273/metrics | grep 'mgob_'

# HELP mgob_scheduler_backup_size Telegraf collected metric
# TYPE mgob_scheduler_backup_size gauge
mgob_scheduler_backup_size{cluster_name="mongodb-cluster",plan="mongodb-cluster",status="200",url="http://localhost:8090/metrics"} 1.9250215041e+10
# HELP mgob_scheduler_backup_total Telegraf collected metric
# TYPE mgob_scheduler_backup_total counter
mgob_scheduler_backup_total{cluster_name="mongodb-cluster",plan="mongodb-cluster",status="200",url="http://localhost:8090/metrics"} 1

Expected behavior:

Have the same metrics in both requests.

Actual behavior:

Metrics mgob_scheduler_backup_latency (Type summary) has been dropped by Telegraf if we use metric_version = 2.

Additional info:

The summary vector from MGOB: https://github.com/stefanprodan/mgob/blob/master/pkg/metrics/metrics.go#L34

@danielnelson danielnelson self-assigned this Feb 7, 2020
@danielnelson danielnelson added area/prometheus bug unexpected problem or unintended behavior labels Feb 7, 2020
@danielnelson danielnelson added this to the 1.13.4 milestone Feb 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/prometheus bug unexpected problem or unintended behavior
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants