Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aurora 13.5 returns shared_buffers setting with unit causing a panic #619

Closed
angaz opened this issue Mar 3, 2022 · 4 comments · Fixed by #620
Closed

Aurora 13.5 returns shared_buffers setting with unit causing a panic #619

angaz opened this issue Mar 3, 2022 · 4 comments · Fixed by #620

Comments

@angaz
Copy link
Contributor

angaz commented Mar 3, 2022

I have upgraded my AWS Aurora server to 13.5 today and I noticed that postgres-exporter started crashing.

From the logs, I see that the error is:

panic: Error converting setting "shared_buffers" value "88413056kB" to float: strconv.ParseFloat: parsing "88413056kB": invalid syntax

I looked at the code and I found a query selecting from pg_settings, so I tried it on the server, and indeed, shared_buffers is returning a value with the unit combined.

I tested the same query on a postgres in docker on my machine and 13.5 does not return this value with the unit, so this is something related to Aurora specifically.

I can work on a pull request. But I'm not sure what would be the best method. My thought was to have a test to see if the unit is included in the setting's value and strip it if it is the case.

This is the shared_buffer line from the query:

                        name                         | setting    | coalesce |                                                           short_desc                                                           | vartype
 shared_buffers                                      | 88413056kB | 8kB      | Sets the number of shared memory buffers used by the server.                                                                   | integer
@mbanck
Copy link
Contributor

mbanck commented Mar 16, 2022

Did you report this to Amazon Aurora?

@angaz
Copy link
Contributor Author

angaz commented Mar 16, 2022

Did you report this to Amazon Aurora?

Yes, I have sent feedback. Not sure if that was the correct place to do it.

@saqibarfeen
Copy link

saqibarfeen commented Mar 21, 2022

My postgres-exporter is also crashing with errors:
panic: Error converting setting "shared_buffers" value "10512416kB" to float: strconv.ParseFloat: parsing "10512416kB": invalid syntax
We are using Amazon Aurora in AWS RDS. Not sure if there are more errors, because the exporter doesnot start.

Edit: I build this feature and published on my docker hub. It works perfectly!
https://hub.docker.com/r/saqibarfeen/postgres-exporter

sysadmind added a commit to sysadmind/postgres_exporter that referenced this issue Mar 23, 2022
Adds --strip-settings-suffix flag to make the fixes for AWS Aurora in prometheus-community#619 opt in for users. It should be safe for all users, but this code should also be unnecessary if AWS fixes their bug.

Signed-off-by: Joe Adams <github@joeadams.io>
@c16a
Copy link

c16a commented Mar 25, 2022

@sysadmind - any release tracker that we can follow? We recently upgraded Aurora and this affects us too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants