Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BGwriter stats cannot be scraped from postgres 17 cluster #36784

Closed
sprayzcs opened this issue Dec 11, 2024 · 1 comment · Fixed by #36813
Closed

BGwriter stats cannot be scraped from postgres 17 cluster #36784

sprayzcs opened this issue Dec 11, 2024 · 1 comment · Fixed by #36813
Labels
bug Something isn't working needs triage New item requiring triage receiver/postgresql

Comments

@sprayzcs
Copy link

sprayzcs commented Dec 11, 2024

Component(s)

receiver/postgresql

What happened?

Description

After updating to postgres 17, the scraper began throwing errors. This is caused by a change in some internal postgres tables. Some values of the pg_stat_bgwriter were moved to the new table pg_stat_checkpointer

Documentation from postgres 16: https://www.postgresql.org/docs/16/monitoring-stats.html#MONITORING-PG-STAT-BGWRITER-VIEW
Documentation from postgres 17: https://www.postgresql.org/docs/17/monitoring-stats.html#MONITORING-PG-STAT-BGWRITER-VIEW

Steps to Reproduce

Configure the postgresqlreceiver to scrape metrics from a postgres 17 cluster

Expected Result

Metrics are scraped

Actual Result

Metrics are not scraped, error in log:

{"level":"error","ts":"2024-12-11T12:56:18.376Z","caller":"scraperhelper/obs_metrics.go:56","msg":"Error scraping metrics","kind":"receiver","name":"postgresql","data_type":"metrics","scraper":"postgresql","error":"pq: column \"checkpoints_req\" does not exist"}

There is also an error in the log of the postgres cluster:

column "checkpoints_req" does not exist
(SQL):	SELECT
	checkpoints_req AS checkpoint_req,
	checkpoints_timed AS checkpoint_scheduled,
	checkpoint_write_time AS checkpoint_duration_write,
	checkpoint_sync_time AS checkpoint_duration_sync,
	buffers_clean AS bg_writes,
	buffers_backend AS backend_writes,
	buffers_backend_fsync AS buffers_written_fsync,
	buffers_checkpoint AS buffers_checkpoints,
	buffers_alloc AS buffers_allocated,
	maxwritten_clean AS maxwritten_count
	FROM pg_stat_bgwriter;

Collector version

0.115.1

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

receivers:
  postgresql:
    endpoint: localhost:5432
    username: ${env:POSTGRES_USER}
    password: ${env:POSTGRES_PASSWORD}
    tls:
      insecure: true

processors:
  batch: {}

exporters:
  otlp: 
    endpoint: 10.1.20.4:5317
    tls:
      insecure: true

service:
  pipelines:
    metrics:
      receivers:
        - postgresql
      processors:
        - batch
      exporters:
        - otlp

Log output

{"level":"error","ts":"2024-12-11T12:48:18.383Z","caller":"scraperhelper/obs_metrics.go:56","msg":"Error scraping metrics","kind":"receiver","name":"postgresql","data_type":"metrics","scraper":"postgresql","error":"pq: column \"checkpoints_req\" does not exist"}

Additional context

No response

@sprayzcs sprayzcs added bug Something isn't working needs triage New item requiring triage labels Dec 11, 2024
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

sbylica-splunk pushed a commit to sbylica-splunk/opentelemetry-collector-contrib that referenced this issue Dec 17, 2024
…pen-telemetry#36813)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Support collecting bgwriter metrics from postgres17. 

<!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes
open-telemetry#36784

<!--Describe what testing was performed and which tests were added.-->
#### Testing
Updated integration tests, tested locally.

<!--Please delete paragraphs that you did not use before submitting.-->
mterhar pushed a commit to mterhar/opentelemetry-collector-contrib that referenced this issue Dec 19, 2024
…pen-telemetry#36813)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Support collecting bgwriter metrics from postgres17. 

<!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes
open-telemetry#36784

<!--Describe what testing was performed and which tests were added.-->
#### Testing
Updated integration tests, tested locally.

<!--Please delete paragraphs that you did not use before submitting.-->
AkhigbeEromo pushed a commit to sematext/opentelemetry-collector-contrib that referenced this issue Jan 13, 2025
…pen-telemetry#36813)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Support collecting bgwriter metrics from postgres17. 

<!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes
open-telemetry#36784

<!--Describe what testing was performed and which tests were added.-->
#### Testing
Updated integration tests, tested locally.

<!--Please delete paragraphs that you did not use before submitting.-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs triage New item requiring triage receiver/postgresql
Projects
None yet
1 participant