Skip to content

Add pg_database collector #613

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Feb 15, 2022
Merged

Add pg_database collector #613

merged 4 commits into from
Feb 15, 2022

Conversation

sysadmind
Copy link
Contributor

Converts the pg_database metrics from queries.yaml to a built in collector. This is enabled by default because it is not likely to be a performance problem and likely very useful data.

Signed-off-by: Joe Adams github@joeadams.io

@sysadmind
Add pg_database collector
21a19ed 
Converts the pg_database metrics from queries.yaml to a built in collector. This is enabled by default because it is not likely to be a performance problem and likely very useful data.

Signed-off-by: Joe Adams <github@joeadams.io>
@sysadmind sysadmind requested a review from SuperQ February 10, 2022 02:30
sysadmind
sysadmind commented Feb 10, 2022
View reviewed changes
cmd/postgres_exporter/server.go Outdated
@@ -128,6 +131,17 @@ func (s *Server) Scrape(ch chan<- prometheus.Metric, disableSettingsMetrics bool
err = fmt.Errorf("queryNamespaceMappings returned %d errors", len(errMap))
}

{
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is in a block to not reset the err var that gets set above. I don't want to return early because I don't think the errors are fatal, just informational.

@sysadmind
Add missing server label
1981623 
Signed-off-by: Joe Adams <github@joeadams.io>
SuperQ
SuperQ reviewed Feb 10, 2022
View reviewed changes
@sysadmind
Add collector interface
c3b0206 
Uses node_exporter style collector registration

Signed-off-by: Joe Adams <github@joeadams.io>
@sysadmind
Comment unused var
9dad33c 
Signed-off-by: Joe Adams <github@joeadams.io>
@sysadmind
Copy link
Contributor Author

@SuperQ Setting up the node_exporter style collector registration required a lot more code changes so this PR got a lot larger. I had to copy a lot of the node exporter functions.

This will be a much better strategy moving forward however.

SuperQ
SuperQ approved these changes Feb 15, 2022
View reviewed changes
Copy link
Contributor

@SuperQ SuperQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@SuperQ SuperQ merged commit 3d7edd4 into prometheus-community:master Feb 15, 2022
@sysadmind sysadmind deleted the pg_database branch February 23, 2022 01:58
SuperQ added a commit that referenced this pull request Jul 28, 2022
@SuperQ
Release v0.11.0
da63601 
NOTE: pg_stat_bgwriter counter metrics had the `_total` suffix added #556

* [CHANGE] refactor pg_stat_bgwriter metrics into standalone collector #556
* [FEATURE] Add pg_database collector #613
* [ENHANCEMENT] Add pg_database_size_bytes metric #613
* [BUGFIX] Avoid parsing error from bogus Azure Flexible Server custom GUC #587
* [BUGFIX] Fix pg_stat_archiver error in 9.4 and earlier. #599
* [BUGFIX] Sanitize setting values because of Aurora irregularity #620

Signed-off-by: SuperQ <superq@gmail.com>
@SuperQ SuperQ mentioned this pull request Jul 28, 2022
Merged
@idoqo idoqo mentioned this pull request Sep 24, 2022
Closed
@sysadmind sysadmind mentioned this pull request Oct 4, 2022
Closed
ademidoff pushed a commit to percona/postgres_exporter that referenced this pull request Oct 12, 2022
@tshcherban @SuperQ
@prombot @godber @sysadmind @dependabot @bravosierrasierra @n-rodriguez
PMM-10820 exporter update (#76)
7199257 
* Update build

* Update to Go 1.18.
* Update minimum Go version to 1.17.
* Update Go modules for 1.17 format.
* Bump Go modules
* Enable dependabot.
* Update Prometheus common files.
* Fixup yamllint.

Signed-off-by: SuperQ <superq@gmail.com>

* Update common Prometheus files

Signed-off-by: prombot <prometheus-team@googlegroups.com>

* Update common Prometheus files (prometheus-community#650)

Signed-off-by: prombot <prometheus-team@googlegroups.com>

* Update common Prometheus files

Signed-off-by: prombot <prometheus-team@googlegroups.com>

* Update readme to include Postgres 14 support

It looks like postgres 14.1 was added to CI here:

prometheus-community@fcb2535

See also: prometheus-community#651 (comment)

Signed-off-by: Austin Godber <godber@uberhip.com>

* Bump github.com/prometheus/common from 0.34.0 to 0.35.0

Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.34.0 to 0.35.0.
- [Release notes](https://github.com/prometheus/common/releases)
- [Commits](prometheus/common@v0.34.0...v0.35.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Release v0.11.0

NOTE: pg_stat_bgwriter counter metrics had the `_total` suffix added prometheus-community#556

* [CHANGE] refactor pg_stat_bgwriter metrics into standalone collector prometheus-community#556
* [FEATURE] Add pg_database collector prometheus-community#613
* [ENHANCEMENT] Add pg_database_size_bytes metric prometheus-community#613
* [BUGFIX] Avoid parsing error from bogus Azure Flexible Server custom GUC prometheus-community#587
* [BUGFIX] Fix pg_stat_archiver error in 9.4 and earlier. prometheus-community#599
* [BUGFIX] Sanitize setting values because of Aurora irregularity prometheus-community#620

Signed-off-by: SuperQ <superq@gmail.com>

* fix for exporter issue 633

fix for exporter issue 633: prometheus-community#633

"Scan error on column index 2, name \"checkpoint_write_time\": converting driver.Value type float64 (\"6.594096e+06\") to a int: invalid syntax prometheus-community#633"

Signed-off-by: bravosierrasierra <bravosierrasierra@users.noreply.github.com>

* Fix checkpoint_sync_time value type

Error:
sql: Scan error on column index 3, name \"checkpoint_sync_time\": converting driver.Value type float64 (\"1.876469e+06\") to a int: invalid syntax

See also:
prometheus-community#633
prometheus-community#666

Signed-off-by: Nicolas Rodriguez <nico@nicoladmin.fr>

* Release 0.11.1

* [BUGFIX] Fix checkpoint_write_time value type prometheus-community#666
* [BUGFIX] Fix checkpoint_sync_time value type prometheus-community#667

Signed-off-by: SuperQ <superq@gmail.com>

* PMM-10820 missing metric restore

* PMM-10820 missing metric drop

* PMM-10820 merge fix

* PMM-10820 PR review fix

Signed-off-by: SuperQ <superq@gmail.com>
Signed-off-by: prombot <prometheus-team@googlegroups.com>
Signed-off-by: Austin Godber <godber@uberhip.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: bravosierrasierra <bravosierrasierra@users.noreply.github.com>
Signed-off-by: Nicolas Rodriguez <nico@nicoladmin.fr>
Co-authored-by: SuperQ <superq@gmail.com>
Co-authored-by: prombot <prometheus-team@googlegroups.com>
Co-authored-by: Austin Godber <godber@uberhip.com>
Co-authored-by: Joe Adams <github@joeadams.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: bravosierrasierra <bravosierrasierra@users.noreply.github.com>
Co-authored-by: Nicolas Rodriguez <nico@nicoladmin.fr>
ahmadsherif added a commit to ahmadsherif/helm-charts that referenced this pull request Nov 29, 2022
@ahmadsherif
[postgres-exporter] Remove pg_database query
863011e 
After the merge of prometheus-community/postgres_exporter#613,
`postgres-exporter` installed by recent versions of the prometheus-postgres-exporter Helm chart is throwing 500 errors like:
```
HTTP/1.1 500 Internal Server Error
Content-Type: text/plain; charset=utf-8
X-Content-Type-Options: nosniff
Date: Tue, 29 Nov 2022 11:08:41 GMT
Content-Length: 1418

An error has occurred while serving metrics:

4 error(s) occurred:
* collected metric "pg_database_size_bytes" { label:<name:"datname" value:"template0" > label:<name:"server" value:"10.188.0.2:5432" > gauge:<value:8.225283e+06 > } was collected before with the same name and label values
* collected metric "pg_database_size_bytes" { label:<name:"datname" value:"postgres" > label:<name:"server" value:"10.188.0.2:5432" > gauge:<value:8.373103e+06 > } was collected before with the same name and label values
* collected metric "pg_database_size_bytes" { label:<name:"datname" value:"template1" > label:<name:"server" value:"10.188.0.2:5432" > gauge:<value:8.373103e+06 > } was collected before with the same name and label values
* collected metric "pg_database_size_bytes" { label:<name:"datname" value:"default" > label:<name:"server" value:"10.188.0.2:5432" > gauge:<value:8.373103e+06 > } was collected before with the same name and label values
```
So we're removing the YAML query to avoid such conflict.

Signed-off-by: Ahmad Sherif <me@ahmadsherif.com>
@ahmadsherif ahmadsherif mentioned this pull request Nov 29, 2022
Merged
3 tasks
ahmadsherif added a commit to ahmadsherif/helm-charts that referenced this pull request Dec 6, 2022
@ahmadsherif
[postgres-exporter] Remove pg_database query
da5d2c4 
After the merge of prometheus-community/postgres_exporter#613,
`postgres-exporter` installed by recent versions of the prometheus-postgres-exporter Helm chart is throwing 500 errors like:
```
HTTP/1.1 500 Internal Server Error
Content-Type: text/plain; charset=utf-8
X-Content-Type-Options: nosniff
Date: Tue, 29 Nov 2022 11:08:41 GMT
Content-Length: 1418

An error has occurred while serving metrics:

4 error(s) occurred:
* collected metric "pg_database_size_bytes" { label:<name:"datname" value:"template0" > label:<name:"server" value:"10.188.0.2:5432" > gauge:<value:8.225283e+06 > } was collected before with the same name and label values
* collected metric "pg_database_size_bytes" { label:<name:"datname" value:"postgres" > label:<name:"server" value:"10.188.0.2:5432" > gauge:<value:8.373103e+06 > } was collected before with the same name and label values
* collected metric "pg_database_size_bytes" { label:<name:"datname" value:"template1" > label:<name:"server" value:"10.188.0.2:5432" > gauge:<value:8.373103e+06 > } was collected before with the same name and label values
* collected metric "pg_database_size_bytes" { label:<name:"datname" value:"default" > label:<name:"server" value:"10.188.0.2:5432" > gauge:<value:8.373103e+06 > } was collected before with the same name and label values
```
So we're removing the YAML query to avoid such conflict.

Signed-off-by: Ahmad Sherif <me@ahmadsherif.com>
monotek pushed a commit to prometheus-community/helm-charts that referenced this pull request Dec 13, 2022
@ahmadsherif @zanhsieh
[prometheus-postgres-exporter] Remove pg_database query (#2752)
8cf2418 
* [postgres-exporter] Remove pg_database query

After the merge of prometheus-community/postgres_exporter#613,
`postgres-exporter` installed by recent versions of the prometheus-postgres-exporter Helm chart is throwing 500 errors like:
```
HTTP/1.1 500 Internal Server Error
Content-Type: text/plain; charset=utf-8
X-Content-Type-Options: nosniff
Date: Tue, 29 Nov 2022 11:08:41 GMT
Content-Length: 1418

An error has occurred while serving metrics:

4 error(s) occurred:
* collected metric "pg_database_size_bytes" { label:<name:"datname" value:"template0" > label:<name:"server" value:"10.188.0.2:5432" > gauge:<value:8.225283e+06 > } was collected before with the same name and label values
* collected metric "pg_database_size_bytes" { label:<name:"datname" value:"postgres" > label:<name:"server" value:"10.188.0.2:5432" > gauge:<value:8.373103e+06 > } was collected before with the same name and label values
* collected metric "pg_database_size_bytes" { label:<name:"datname" value:"template1" > label:<name:"server" value:"10.188.0.2:5432" > gauge:<value:8.373103e+06 > } was collected before with the same name and label values
* collected metric "pg_database_size_bytes" { label:<name:"datname" value:"default" > label:<name:"server" value:"10.188.0.2:5432" > gauge:<value:8.373103e+06 > } was collected before with the same name and label values
```
So we're removing the YAML query to avoid such conflict.

Signed-off-by: Ahmad Sherif <me@ahmadsherif.com>

* [prometheus-postgres-exporter] Bump version to 3.3.1

Signed-off-by: Ahmad Sherif <me@ahmadsherif.com>

* [prometheus-postgres-exporter] Bump version to 4.0.0

Signed-off-by: Ahmad Sherif <me@ahmadsherif.com>

* Add upgrading notes to README.md

Signed-off-by: Ahmad Sherif <me@ahmadsherif.com>

Signed-off-by: Ahmad Sherif <me@ahmadsherif.com>
Co-authored-by: MH <zanhsieh@gmail.com>
cx-asafd pushed a commit to CheckmarxDev/prometheus-helm-charts that referenced this pull request Dec 29, 2022
@ahmadsherif @zanhsieh @cx-asafd
[prometheus-postgres-exporter] Remove pg_database query (prometheus-c…
ba2e9e4 
…ommunity#2752)

* [postgres-exporter] Remove pg_database query

After the merge of prometheus-community/postgres_exporter#613,
`postgres-exporter` installed by recent versions of the prometheus-postgres-exporter Helm chart is throwing 500 errors like:
```
HTTP/1.1 500 Internal Server Error
Content-Type: text/plain; charset=utf-8
X-Content-Type-Options: nosniff
Date: Tue, 29 Nov 2022 11:08:41 GMT
Content-Length: 1418

An error has occurred while serving metrics:

4 error(s) occurred:
* collected metric "pg_database_size_bytes" { label:<name:"datname" value:"template0" > label:<name:"server" value:"10.188.0.2:5432" > gauge:<value:8.225283e+06 > } was collected before with the same name and label values
* collected metric "pg_database_size_bytes" { label:<name:"datname" value:"postgres" > label:<name:"server" value:"10.188.0.2:5432" > gauge:<value:8.373103e+06 > } was collected before with the same name and label values
* collected metric "pg_database_size_bytes" { label:<name:"datname" value:"template1" > label:<name:"server" value:"10.188.0.2:5432" > gauge:<value:8.373103e+06 > } was collected before with the same name and label values
* collected metric "pg_database_size_bytes" { label:<name:"datname" value:"default" > label:<name:"server" value:"10.188.0.2:5432" > gauge:<value:8.373103e+06 > } was collected before with the same name and label values
```
So we're removing the YAML query to avoid such conflict.

Signed-off-by: Ahmad Sherif <me@ahmadsherif.com>

* [prometheus-postgres-exporter] Bump version to 3.3.1

Signed-off-by: Ahmad Sherif <me@ahmadsherif.com>

* [prometheus-postgres-exporter] Bump version to 4.0.0

Signed-off-by: Ahmad Sherif <me@ahmadsherif.com>

* Add upgrading notes to README.md

Signed-off-by: Ahmad Sherif <me@ahmadsherif.com>

Signed-off-by: Ahmad Sherif <me@ahmadsherif.com>
Co-authored-by: MH <zanhsieh@gmail.com>
ritbl pushed a commit to heniek/postgres_exporter that referenced this pull request Mar 19, 2023
@SuperQ
Merge pull request prometheus-community#613 from sysadmind/pg_database
b63425c 
Add pg_database collector
ritbl pushed a commit to heniek/postgres_exporter that referenced this pull request Mar 19, 2023
@SuperQ
Release v0.11.0
8612885 
NOTE: pg_stat_bgwriter counter metrics had the `_total` suffix added prometheus-community#556

* [CHANGE] refactor pg_stat_bgwriter metrics into standalone collector prometheus-community#556
* [FEATURE] Add pg_database collector prometheus-community#613
* [ENHANCEMENT] Add pg_database_size_bytes metric prometheus-community#613
* [BUGFIX] Avoid parsing error from bogus Azure Flexible Server custom GUC prometheus-community#587
* [BUGFIX] Fix pg_stat_archiver error in 9.4 and earlier. prometheus-community#599
* [BUGFIX] Sanitize setting values because of Aurora irregularity prometheus-community#620

Signed-off-by: SuperQ <superq@gmail.com>
ritbl pushed a commit to heniek/postgres_exporter that referenced this pull request Mar 19, 2023
@tshcherban @SuperQ
@prombot @godber @sysadmind @dependabot @bravosierrasierra @n-rodriguez
PMM-10820 exporter update (percona#76)
a852822 
* Update build

* Update to Go 1.18.
* Update minimum Go version to 1.17.
* Update Go modules for 1.17 format.
* Bump Go modules
* Enable dependabot.
* Update Prometheus common files.
* Fixup yamllint.

Signed-off-by: SuperQ <superq@gmail.com>

* Update common Prometheus files

Signed-off-by: prombot <prometheus-team@googlegroups.com>

* Update common Prometheus files (prometheus-community#650)

Signed-off-by: prombot <prometheus-team@googlegroups.com>

* Update common Prometheus files

Signed-off-by: prombot <prometheus-team@googlegroups.com>

* Update readme to include Postgres 14 support

It looks like postgres 14.1 was added to CI here:

prometheus-community@fcb2535

See also: prometheus-community#651 (comment)

Signed-off-by: Austin Godber <godber@uberhip.com>

* Bump github.com/prometheus/common from 0.34.0 to 0.35.0

Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.34.0 to 0.35.0.
- [Release notes](https://github.com/prometheus/common/releases)
- [Commits](prometheus/common@v0.34.0...v0.35.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Release v0.11.0

NOTE: pg_stat_bgwriter counter metrics had the `_total` suffix added prometheus-community#556

* [CHANGE] refactor pg_stat_bgwriter metrics into standalone collector prometheus-community#556
* [FEATURE] Add pg_database collector prometheus-community#613
* [ENHANCEMENT] Add pg_database_size_bytes metric prometheus-community#613
* [BUGFIX] Avoid parsing error from bogus Azure Flexible Server custom GUC prometheus-community#587
* [BUGFIX] Fix pg_stat_archiver error in 9.4 and earlier. prometheus-community#599
* [BUGFIX] Sanitize setting values because of Aurora irregularity prometheus-community#620

Signed-off-by: SuperQ <superq@gmail.com>

* fix for exporter issue 633

fix for exporter issue 633: prometheus-community#633

"Scan error on column index 2, name \"checkpoint_write_time\": converting driver.Value type float64 (\"6.594096e+06\") to a int: invalid syntax prometheus-community#633"

Signed-off-by: bravosierrasierra <bravosierrasierra@users.noreply.github.com>

* Fix checkpoint_sync_time value type

Error:
sql: Scan error on column index 3, name \"checkpoint_sync_time\": converting driver.Value type float64 (\"1.876469e+06\") to a int: invalid syntax

See also:
prometheus-community#633
prometheus-community#666

Signed-off-by: Nicolas Rodriguez <nico@nicoladmin.fr>

* Release 0.11.1

* [BUGFIX] Fix checkpoint_write_time value type prometheus-community#666
* [BUGFIX] Fix checkpoint_sync_time value type prometheus-community#667

Signed-off-by: SuperQ <superq@gmail.com>

* PMM-10820 missing metric restore

* PMM-10820 missing metric drop

* PMM-10820 merge fix

* PMM-10820 PR review fix

Signed-off-by: SuperQ <superq@gmail.com>
Signed-off-by: prombot <prometheus-team@googlegroups.com>
Signed-off-by: Austin Godber <godber@uberhip.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: bravosierrasierra <bravosierrasierra@users.noreply.github.com>
Signed-off-by: Nicolas Rodriguez <nico@nicoladmin.fr>
Co-authored-by: SuperQ <superq@gmail.com>
Co-authored-by: prombot <prometheus-team@googlegroups.com>
Co-authored-by: Austin Godber <godber@uberhip.com>
Co-authored-by: Joe Adams <github@joeadams.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: bravosierrasierra <bravosierrasierra@users.noreply.github.com>
Co-authored-by: Nicolas Rodriguez <nico@nicoladmin.fr>
Matiasmct pushed a commit to giffgaff/prometheus-charts-backup that referenced this pull request May 16, 2023
@ahmadsherif @zanhsieh @Matiasmct
[prometheus-postgres-exporter] Remove pg_database query (prometheus-c…
911d757 
…ommunity#2752)

* [postgres-exporter] Remove pg_database query

After the merge of prometheus-community/postgres_exporter#613,
`postgres-exporter` installed by recent versions of the prometheus-postgres-exporter Helm chart is throwing 500 errors like:
```
HTTP/1.1 500 Internal Server Error
Content-Type: text/plain; charset=utf-8
X-Content-Type-Options: nosniff
Date: Tue, 29 Nov 2022 11:08:41 GMT
Content-Length: 1418

An error has occurred while serving metrics:

4 error(s) occurred:
* collected metric "pg_database_size_bytes" { label:<name:"datname" value:"template0" > label:<name:"server" value:"10.188.0.2:5432" > gauge:<value:8.225283e+06 > } was collected before with the same name and label values
* collected metric "pg_database_size_bytes" { label:<name:"datname" value:"postgres" > label:<name:"server" value:"10.188.0.2:5432" > gauge:<value:8.373103e+06 > } was collected before with the same name and label values
* collected metric "pg_database_size_bytes" { label:<name:"datname" value:"template1" > label:<name:"server" value:"10.188.0.2:5432" > gauge:<value:8.373103e+06 > } was collected before with the same name and label values
* collected metric "pg_database_size_bytes" { label:<name:"datname" value:"default" > label:<name:"server" value:"10.188.0.2:5432" > gauge:<value:8.373103e+06 > } was collected before with the same name and label values
```
So we're removing the YAML query to avoid such conflict.

Signed-off-by: Ahmad Sherif <me@ahmadsherif.com>

* [prometheus-postgres-exporter] Bump version to 3.3.1

Signed-off-by: Ahmad Sherif <me@ahmadsherif.com>

* [prometheus-postgres-exporter] Bump version to 4.0.0

Signed-off-by: Ahmad Sherif <me@ahmadsherif.com>

* Add upgrading notes to README.md

Signed-off-by: Ahmad Sherif <me@ahmadsherif.com>

Signed-off-by: Ahmad Sherif <me@ahmadsherif.com>
Co-authored-by: MH <zanhsieh@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SuperQ SuperQ SuperQ approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@sysadmind @SuperQ