Multiple pgbouncer instances using the helm chart

[seongwon77]

Description

Currently, the helm chart deploys only one pgbouncer instance. (https://github.com/apache/airflow/blob/main/chart/templates/pgbouncer/pgbouncer-deployment.yaml#L40)

However, this could be a single point of failure, since all DB operations would be broken if that instance fails.

Can we have a replica option for pgbouncer like the other components(scheduler, worker, ...)?

Use case/motivation

No response

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[potiuk]

I think you would have to test it thoroughly and make sure it works. From this - (just googled it) - there are some constraints and limits on having multiple pgbouncers and you would also have to make sure to make it in the way that load-balancing happens transparently between the pgbouncer instances: https://www.2ndquadrant.com/en/blog/running-multiple-pgbouncer-instances-with-systemd/#:~:text=Since%20version%201.12%2C%20PgBouncer%20supports,they%20will%20share%20the%20connections.

If you want to work on I think this is a nice feature to add.

[potiuk]

Assigned you @seongwon77

[eladkal]

#27439 made it configurable