[DPE-5827] Set all nodes to synchronous replicas #672

dragomirp · 2024-11-14T11:48:05Z

Set all replicas to sync

codecov · 2024-11-14T11:49:15Z

Codecov Report

Attention: Patch coverage is 65.00000% with 7 lines in your changes missing coverage. Please review.

Project coverage is 71.96%. Comparing base (7e79b64) to head (acb98f8).
Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
src/cluster.py	55.55%	3 Missing and 1 partial ⚠️
src/charm.py	62.50%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #672      +/-   ##
==========================================
- Coverage   72.02%   71.96%   -0.07%     
==========================================
  Files          15       15              
  Lines        3482     3474       -8     
  Branches      535      532       -3     
==========================================
- Hits         2508     2500       -8     
- Misses        844      845       +1     
+ Partials      130      129       -1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

src/cluster.py

dragomirp · 2024-11-15T21:08:51Z

src/cluster.py

                r = requests.patch(
                    f"{self._patroni_url}/config",
-                    json={"synchronous_node_count": units // 2},
+                    json={"synchronous_node_count": units - 1},


REST calls should also use the new value.

lucasgameiroborges

Code LGTM, but why are we setting all nodes to be sync replicas? From the mentioned ticket it should come from a config value?

dragomirp · 2024-11-18T13:19:14Z

Code LGTM, but why are we setting all nodes to be sync replicas? From the mentioned ticket it should come from a config value?

What was discussed was to switch the default behaviour to all sync and add a config for controlling the amount of syncs. The config needs a spec. I'll comment on the ticket to clarify.

marceloneppel

Cool! Testing it, it's working fine in almost all scenarios.

The scenario where it's not working yet is the upgrade:

juju deploy postgresql --channel 14/stable -n 3

# Wait for the units to settle down, then:
curl UNIT-ZERO-IP:8008/cluster | jq # There is one leader, one sync-standby and one replica, as expected.

juju run postgresql/leader pre-upgrade-check

juju refresh --path ./*.charm postgresql

# Wait for the units to settle down, then:
curl UNIT-ZERO-IP:8008/cluster | jq # There is one leader, one sync-standby and one replica, but we should have one leader and two sync-standby only.

juju add-unit postgresql -n 1

# Wait for the units to settle down, then:
curl UNIT-ZERO-IP:8008/cluster | jq # There is one leader, and three sync-standby, as expected.

taurus-forever

LGTM, need some time to play it without rush.
Let's merge it after release the current PostgreSQL VM to stable.

dragomirp · 2024-11-20T13:09:05Z

The scenario where it's not working yet is the upgrade:

5082317 should fix the upgrade. Not adding an assertion to the integration tests because functionality will drift when the changes land on edge and stable.

marceloneppel

The scenario where it's not working yet is the upgrade:

5082317 should fix the upgrade. Not adding an assertion to the integration tests because functionality will drift when the changes land on edge and stable.

Thanks, Dragomir! LGTM!

* Use `charmcraft test` & concierge (#762) * Update charmcraft.yaml build tools (#760) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * [DPE-6020] Better promote-to-primary unit scope error handling (#759) * Bump libs * Flip default scope * Better action failure * Wrong attr * Revert scope * Bump libs * Handle async replica switchover * Unit tests * Bump cosl * Disable Nextcloud test (#767) * Update canonical/data-platform-workflows action to v30 (#770) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Use _promote_charm.yaml (#771) Use `charmcraft promote` and auto-generate release notes * [DPE-5827] Set all nodes to synchronous replicas (#672) * Set all nodes to synchronous replicas * Fix template var * Also change config patching * Update sync nodes during upgrade * Revert are_writes_increasing changes * Add back logging * Try without logs * Tactical sleep * Log removal error * Remove logs * Tweak replication test * Pass down unit * Wait for test app to idle * Add comment * Port config changes * Copy policy test * Fix import * Missed param removal * Unit test * Missing attr * Add logs * Add timeout to connection * Log conn str * Fix num of standbys * Charm fixture * Remove stepdown hook * Config description * Revert conn str * Add async scaling test * Typo * Don't remove standby and primary * Update dependency psutil to v7 (#772) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update dependency cryptography to v44.0.1 [SECURITY] (#764) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update canonical/data-platform-workflows action to v30.0.2 (#765) * Update canonical/data-platform-workflows action to v30.0.2 * Update promote.yaml --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Carl Csaposs <carl.csaposs@canonical.com> * [DPE-6323] Handle missing stanza output (#727) * Handle missing stanza output * Update libs * Unit tests * Update canonical/has-signed-canonical-cla action to v2 (#773) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * [MISC] Define charm constants (#774) * Lock file maintenance Python dependencies (#743) * Lock file maintenance Python dependencies * Backoff boto3 1.36 --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Dragomir Penev <dragomir.penev@canonical.com> * Update charmcraft.yaml build tools (#768) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update canonical/data-platform-workflows action to v30.1.3 (#776) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update dependency uv to v0.6.3 (#780) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * [MISC] Sanitize PostgreSQL extra-user-roles arg (#782) * [MISC] Fix PostgreSQL lib function signature (#786) * [MISC] Skip backup and subordinate tests without creds (#789) * Bump libs * Skip backup tests without creds * Skip subordinate tests * Update tests/integration/test_subordinates.py Co-authored-by: Carl Csaposs <carl.csaposs@canonical.com> --------- Co-authored-by: Carl Csaposs <carl.csaposs@canonical.com> * Update dependency jinja2 to v3.1.6 [SECURITY] (#788) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Disable cache * Reduce required approvals on Renovate pull requests by 1 (#787) * Sync docs from Discourse (#748) Co-authored-by: GitHub Actions <41898282+github-actions[bot]@users.noreply.github.com> * Cleanup juju 2 tests * Linting * Integration test diffs * Try with series for ubuntu pro subordinate * Filter terminated units * Bump PG version * Disable pgaudit for timescale and postgis * Linting * Remove tests * Remove param for secrets * Linting * Idle when disabling pgaudit * Actually disable audit * Disable timescale in object test * Try to disable plugins between tests * Update canonical/data-platform-workflows action to v30.2.0 (#792) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Try to disable pgaudit in general --------- Co-authored-by: Carl Csaposs <carl.csaposs@canonical.com> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Sinclert Pérez <sinclert.perez@canonical.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* WIP: enable pg_tracing * adapt render patroni function * Move pg_tracing conf to the correct location * fix lint and unit tests * use ubuntu 24.04 as base * fix issues + revert base to jammy * fix queries for plugin testing * downgrade psycopg2 version * update lock file * update psycopg2-binary too * revery psycopg2 versions and add ssl params * revert lock * use noble as base * use different workflow version * remove cache * edit plugin * fix integration tests * revert pgtracing config + add new snaps * try fixes * try new fixes * try fixing restart * try catching reload errors * adapt test_subordinates to remove ubuntu pro charm * try use newer branch for workflow * fix lock hash * use new branch for plugin too * remove old refs to pg 14 * specify cc version 3 on release workflow * remove juju 2.9 + refactor release workflow * remove libjuju constraint * fix release too * use new charm + small adjustments * fix lock file * fix build_charm issue * remove base from deploy calls * nits * Use `charmcraft test` & concierge (#762) * Update charmcraft.yaml build tools (#760) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * [DPE-6020] Better promote-to-primary unit scope error handling (#759) * Bump libs * Flip default scope * Better action failure * Wrong attr * Revert scope * Bump libs * Handle async replica switchover * Unit tests * Bump cosl * Disable Nextcloud test (#767) * Update canonical/data-platform-workflows action to v30 (#770) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Use _promote_charm.yaml (#771) Use `charmcraft promote` and auto-generate release notes * [DPE-5827] Set all nodes to synchronous replicas (#672) * Set all nodes to synchronous replicas * Fix template var * Also change config patching * Update sync nodes during upgrade * Revert are_writes_increasing changes * Add back logging * Try without logs * Tactical sleep * Log removal error * Remove logs * Tweak replication test * Pass down unit * Wait for test app to idle * Add comment * Port config changes * Copy policy test * Fix import * Missed param removal * Unit test * Missing attr * Add logs * Add timeout to connection * Log conn str * Fix num of standbys * Charm fixture * Remove stepdown hook * Config description * Revert conn str * Add async scaling test * Typo * Don't remove standby and primary * Update dependency psutil to v7 (#772) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update dependency cryptography to v44.0.1 [SECURITY] (#764) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update canonical/data-platform-workflows action to v30.0.2 (#765) * Update canonical/data-platform-workflows action to v30.0.2 * Update promote.yaml --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Carl Csaposs <carl.csaposs@canonical.com> * [DPE-6323] Handle missing stanza output (#727) * Handle missing stanza output * Update libs * Unit tests * Update canonical/has-signed-canonical-cla action to v2 (#773) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * [MISC] Define charm constants (#774) * Lock file maintenance Python dependencies (#743) * Lock file maintenance Python dependencies * Backoff boto3 1.36 --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Dragomir Penev <dragomir.penev@canonical.com> * Update charmcraft.yaml build tools (#768) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update canonical/data-platform-workflows action to v30.1.3 (#776) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update dependency uv to v0.6.3 (#780) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * [MISC] Sanitize PostgreSQL extra-user-roles arg (#782) * [MISC] Fix PostgreSQL lib function signature (#786) * [MISC] Skip backup and subordinate tests without creds (#789) * Bump libs * Skip backup tests without creds * Skip subordinate tests * Update tests/integration/test_subordinates.py Co-authored-by: Carl Csaposs <carl.csaposs@canonical.com> --------- Co-authored-by: Carl Csaposs <carl.csaposs@canonical.com> * Update dependency jinja2 to v3.1.6 [SECURITY] (#788) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Reduce required approvals on Renovate pull requests by 1 (#787) * Sync docs from Discourse (#748) Co-authored-by: GitHub Actions <41898282+github-actions[bot]@users.noreply.github.com> * Update canonical/data-platform-workflows action to v30.2.0 (#792) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update dependency uv to v0.6.5 (#785) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Pg 16 sync main (#793) * Use `charmcraft test` & concierge (#762) * Update charmcraft.yaml build tools (#760) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * [DPE-6020] Better promote-to-primary unit scope error handling (#759) * Bump libs * Flip default scope * Better action failure * Wrong attr * Revert scope * Bump libs * Handle async replica switchover * Unit tests * Bump cosl * Disable Nextcloud test (#767) * Update canonical/data-platform-workflows action to v30 (#770) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Use _promote_charm.yaml (#771) Use `charmcraft promote` and auto-generate release notes * [DPE-5827] Set all nodes to synchronous replicas (#672) * Set all nodes to synchronous replicas * Fix template var * Also change config patching * Update sync nodes during upgrade * Revert are_writes_increasing changes * Add back logging * Try without logs * Tactical sleep * Log removal error * Remove logs * Tweak replication test * Pass down unit * Wait for test app to idle * Add comment * Port config changes * Copy policy test * Fix import * Missed param removal * Unit test * Missing attr * Add logs * Add timeout to connection * Log conn str * Fix num of standbys * Charm fixture * Remove stepdown hook * Config description * Revert conn str * Add async scaling test * Typo * Don't remove standby and primary * Update dependency psutil to v7 (#772) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update dependency cryptography to v44.0.1 [SECURITY] (#764) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update canonical/data-platform-workflows action to v30.0.2 (#765) * Update canonical/data-platform-workflows action to v30.0.2 * Update promote.yaml --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Carl Csaposs <carl.csaposs@canonical.com> * [DPE-6323] Handle missing stanza output (#727) * Handle missing stanza output * Update libs * Unit tests * Update canonical/has-signed-canonical-cla action to v2 (#773) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * [MISC] Define charm constants (#774) * Lock file maintenance Python dependencies (#743) * Lock file maintenance Python dependencies * Backoff boto3 1.36 --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Dragomir Penev <dragomir.penev@canonical.com> * Update charmcraft.yaml build tools (#768) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update canonical/data-platform-workflows action to v30.1.3 (#776) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update dependency uv to v0.6.3 (#780) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * [MISC] Sanitize PostgreSQL extra-user-roles arg (#782) * [MISC] Fix PostgreSQL lib function signature (#786) * [MISC] Skip backup and subordinate tests without creds (#789) * Bump libs * Skip backup tests without creds * Skip subordinate tests * Update tests/integration/test_subordinates.py Co-authored-by: Carl Csaposs <carl.csaposs@canonical.com> --------- Co-authored-by: Carl Csaposs <carl.csaposs@canonical.com> * Update dependency jinja2 to v3.1.6 [SECURITY] (#788) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Disable cache * Reduce required approvals on Renovate pull requests by 1 (#787) * Sync docs from Discourse (#748) Co-authored-by: GitHub Actions <41898282+github-actions[bot]@users.noreply.github.com> * Cleanup juju 2 tests * Linting * Integration test diffs * Try with series for ubuntu pro subordinate * Filter terminated units * Bump PG version * Disable pgaudit for timescale and postgis * Linting * Remove tests * Remove param for secrets * Linting * Idle when disabling pgaudit * Actually disable audit * Disable timescale in object test * Try to disable plugins between tests * Update canonical/data-platform-workflows action to v30.2.0 (#792) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Try to disable pgaudit in general --------- Co-authored-by: Carl Csaposs <carl.csaposs@canonical.com> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Sinclert Pérez <sinclert.perez@canonical.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Dual branch configs * Revert lib changes * Remove legacy rels * Restore pydantic rule * Remove legacy rels metadata * Remove manual dispatch * Create schema to test admin user privileges * Cleanup markers * Workaround for cluster restore test * Promote permadiff * Apply suggestions from code review Co-authored-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com> * Bump snaps --------- Co-authored-by: Shayan Patel <shayan.patel@canonical.com> Co-authored-by: Lucas Gameiro Borges <lucas.borges@canonical.com> Co-authored-by: Carl Csaposs <carl.csaposs@canonical.com> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Dragomir Penev <6687393+dragomirp@users.noreply.github.com> Co-authored-by: Sinclert Pérez <sinclert.perez@canonical.com> Co-authored-by: Dragomir Penev <dragomir.penev@canonical.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>

Set all nodes to synchronous replicas

bcafc87

github-actions bot added the Libraries: Out of sync label Nov 14, 2024

Fix template var

bd81f55

github-actions bot added Libraries: OK and removed Libraries: Out of sync labels Nov 14, 2024

Also change config patching

026d946

dragomirp marked this pull request as ready for review November 15, 2024 20:40

dragomirp commented Nov 15, 2024

View reviewed changes

src/cluster.py Outdated Show resolved Hide resolved

dragomirp commented Nov 15, 2024

View reviewed changes

dragomirp requested review from a team, taurus-forever, marceloneppel and lucasgameiroborges and removed request for a team November 15, 2024 21:09

lucasgameiroborges approved these changes Nov 18, 2024

View reviewed changes

marceloneppel requested changes Nov 18, 2024

View reviewed changes

taurus-forever approved these changes Nov 19, 2024

View reviewed changes

dragomirp added 2 commits November 19, 2024 16:38

Merge branch 'main' into dpe-5827-all-sync

21e0b13

Update sync nodes during upgrade

5082317

dragomirp requested a review from marceloneppel November 20, 2024 13:09

marceloneppel approved these changes Nov 21, 2024

View reviewed changes

dragomirp mentioned this pull request Nov 21, 2024

[DPE-5827] Set all nodes to synchronous replicas canonical/postgresql-k8s-operator#784

Merged

dragomirp added 5 commits November 22, 2024 17:00

Merge branch 'main' into dpe-5827-all-sync

e2480e2

Merge branch 'main' into dpe-5827-all-sync

12c3e88

Merge branch 'main' into dpe-5827-all-sync

6687255

Revert are_writes_increasing changes

df8a191

Add back logging

ad8e216

dragomirp added 18 commits January 23, 2025 21:15

Merge branch 'main' into dpe-5827-all-sync-config

f188310

Missing attr

549192b

Merge branch 'main' into dpe-5827-all-sync

f99a0f3

Merge branch 'dpe-5827-all-sync' into dpe-5827-all-sync-config

bbb5c40

Merge branch 'main' into dpe-5827-all-sync-config

afadf3c

Merge branch 'main' into dpe-5827-all-sync

995627e

Merge branch 'dpe-5827-all-sync' into dpe-5827-all-sync-config

dc79593

Add logs

46b0746

Merge branch 'main' into dpe-5827-all-sync-config

356faad

Add timeout to connection

69b352b

Merge branch 'main' into dpe-5827-all-sync

5a85899

Merge branch 'dpe-5827-all-sync' into dpe-5827-all-sync-config

d07fa5d

Log conn str

642c3a0

Fix num of standbys

a570abb

Merge branch 'main' into dpe-5827-all-sync

004ba0e

Merge branch 'main' into dpe-5827-all-sync

a5593b1

Merge branch 'dpe-5827-all-sync' into dpe-5827-all-sync-config

df5ab55

Charm fixture

4433909

marceloneppel approved these changes Feb 11, 2025

View reviewed changes

dragomirp added 7 commits February 11, 2025 17:53

Remove stepdown hook

500b4ab

Config description

878182e

Revert conn str

753853a

Add async scaling test

d10f98e

Typo

7abbc83

Don't remove standby and primary

3efdb67

Merge branch 'main' into dpe-5827-all-sync

acb98f8

dragomirp merged commit 87f0755 into main Feb 13, 2025
93 checks passed

dragomirp deleted the dpe-5827-all-sync branch February 13, 2025 14:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[DPE-5827] Set all nodes to synchronous replicas #672

[DPE-5827] Set all nodes to synchronous replicas #672

Uh oh!

dragomirp commented Nov 14, 2024 •

edited

Loading

Uh oh!

codecov bot commented Nov 14, 2024 •

edited

Loading

Uh oh!

Uh oh!

dragomirp Nov 15, 2024

Uh oh!

lucasgameiroborges left a comment

Uh oh!

dragomirp commented Nov 18, 2024

Uh oh!

marceloneppel left a comment

Uh oh!

taurus-forever left a comment

Uh oh!

dragomirp commented Nov 20, 2024

Uh oh!

marceloneppel left a comment

Uh oh!

Uh oh!

Uh oh!

[DPE-5827] Set all nodes to synchronous replicas #672

[DPE-5827] Set all nodes to synchronous replicas #672

Uh oh!

Conversation

dragomirp commented Nov 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Nov 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

dragomirp Nov 15, 2024

Choose a reason for hiding this comment

Uh oh!

lucasgameiroborges left a comment

Choose a reason for hiding this comment

Uh oh!

dragomirp commented Nov 18, 2024

Uh oh!

marceloneppel left a comment

Choose a reason for hiding this comment

Uh oh!

taurus-forever left a comment

Choose a reason for hiding this comment

Uh oh!

dragomirp commented Nov 20, 2024

Uh oh!

marceloneppel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

dragomirp commented Nov 14, 2024 •

edited

Loading

codecov bot commented Nov 14, 2024 •

edited

Loading