Checkpoint related columns in PG 17 have been moved from pg_stat_bgwriter to pg_stat_checkpointer

[n-rodriguez]

Fix #1060

See:

• https://www.dbi-services.com/blog/postgresql-17-new-catalog-view-pg_stat_checkpointer/
• https://www.postgresql.org/docs/16/monitoring-stats.html#MONITORING-PG-STAT-BGWRITER-VIEW
• https://www.postgresql.org/docs/17/monitoring-stats.html#MONITORING-PG-STAT-BGWRITER-VIEW

Thank you!

[n-rodriguez]

Hi there! Any news?

[n-rodriguez]

Ping @sysadmind

[BupycHuk]

@SuperQ @wrouesnel @sysadmind, could you please reiew this PR, it's important for PG17 monitoring.

[michael-todorovic]

Hello,
I was interested in this PR as we start to get some PG17. However, we moved to multi-target pattern and we're encountering some issues with PG<17. Here's what I did:

I start a very simple PG16:

docker run --rm -p 5416:5432 -e POSTGRES_PASSWORD=postgres postgres:16 -d postgres

I have a config.yml so the exporter can remotely connect to PG:

auth_modules:
  monitoring:
    type: userpass
    userpass:
      username: postgres
      password: postgres

I start the exporter (with the merged PR):

./postgres_exporter --config.file=config.yml --collector.stat_checkpointer

and I call the exporter:

curl 'http://localhost:9187/probe?auth_module=monitoring&target=localhost:5416/postgres?sslmode=disable'

The exporter outputs this error:

time=2024-11-27T14:32:02.679+01:00 level=ERROR source=collector.go:201 msg="collector failed" target="localhost:5416/postgres?sslmode=disable" name=stat_checkpointer duration_seconds=0.013500983 err="pq: relation \"pg_stat_checkpointer\" does not exist"

I'll try to check how to improve the current PR so it can work in this mode 😄

[n-rodriguez]

ping @sysadmind @wrouesnel @SuperQ

[sysadmind]

I think the code is good and should be good to merge as soon as the Copyright dates are updated.

[michael-todorovic]

👋 @n-rodriguez , would you please check the changes in n-rodriguez#1?
In multi-target mode, the current MR is preventing the exporter to work so I provided some fixes for it :)

[sysadmind]

The last commit is missing the DCO

[n-rodriguez]

The last commit is missing the DCO

Done :)

[djhoese]

Is there any roadmap to a release that includes this bug fix?

[SangeethKarunaratne]

@sysadmind can we get a release for this fix? 👀

[kerams]

Any recommendations for a different Postgres monitoring project, ideally Prometheus-based? This repo is as good as dead. Having to drag one of the maintainers kicking and screaming to merge a couple of PRs every quarter just doesn't cut it for a project that aspires to be the semi-official Postgres exporter.

[pcace]

can you not just deploy quay.io/prometheuscommunity/postgres-exporter:master? its in there..

[SuperSandro2000]

No. That will definitely be forgotten and byte back in the future.

[michael-todorovic]

On our side, we have built a versionned image with patches coming from existing PRs, waiting for them to be merged. This requires low maintenance but that the current cost to have required feats for our production

[michael-todorovic]

If that can help someone, here's my Dockerfile to build a custom image with PRs patches below. I use it for multi-target setup with consul service discovery, hosted in k8s. You're free to use it the way you prefer, the image doesn't force anything 😄
Note that SOFTWARE_VERSION can be a commit sha from master as well. That's the way I used it until 0.16.0 came out with a patch I needed.

FROM golang:1.23.0 AS build

ARG UPSTREAM_REPO=github.com/prometheus-community/postgres_exporter
ARG SOFTWARE_VERSION=v0.16.0

WORKDIR ${GOPATH}/src/${UPSTREAM_REPO}

RUN apt-get update -qyy && apt install -qyy --no-install-recommends patch curl git ca-certificates
RUN git clone https://${UPSTREAM_REPO} ${GOPATH}/src/${UPSTREAM_REPO} && git checkout ${SOFTWARE_VERSION}
# index collector
RUN curl -s -f https://patch-diff.githubusercontent.com/raw/prometheus-community/postgres_exporter/pull/1071.diff | patch -p1 -i -
# pg17 fix about checkpointer
RUN curl -s -f https://patch-diff.githubusercontent.com/raw/prometheus-community/postgres_exporter/pull/1072.diff | patch -p1 -i -
# fix for #1072 to handle pg<17
RUN curl -s -f https://patch-diff.githubusercontent.com/raw/n-rodriguez/postgres_exporter/pull/1.diff | patch -p1 -i -
# fix: handle pg_replication_slots on pg<13
RUN curl -s -f https://patch-diff.githubusercontent.com/raw/prometheus-community/postgres_exporter/pull/1098.diff | patch -p1 -i -
RUN make build

FROM busybox:stable

COPY --from=build /go/src/github.com/prometheus-community/postgres_exporter/postgres_exporter /bin/postgres_exporter
COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/

ENTRYPOINT [ "/bin/postgres_exporter" ]

[CarpathianUA]

We need this to be released, for adequate Postgres 17 support!

[sysadmind]

I have a release staged. It looks like #1112 would be important to merge before the release, but needs a fix for the tests. I'm hoping to have the next release out this week and then take another pass to review open PRs here.

[sysadmind]

https://github.com/prometheus-community/postgres_exporter/releases/tag/v0.17.0

[SangeethKarunaratne]

Thank you @sysadmind !! 🎉 🚀

[n-rodriguez]

Sadly I don't appear in contributors list : https://github.com/prometheus-community/postgres_exporter/releases/tag/v0.17.0 😢 (I think because of the typo: [BIGFIX] => [BUGFIX]

[sysadmind]

I fixed the typo. I see you show up in the list.