[DPE-4533] Pause Patroni in the TLS test #534

dragomirp · 2024-07-18T09:44:14Z

Try to reduce the TLS test flakyness.

codecov · 2024-07-18T09:48:33Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.87%. Comparing base (2a944d0) to head (34a9606).
Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #534   +/-   ##
=======================================
  Coverage   70.87%   70.87%           
=======================================
  Files          11       11           
  Lines        3021     3021           
  Branches      535      535           
=======================================
  Hits         2141     2141           
  Misses        764      764           
  Partials      116      116

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

dragomirp · 2024-07-25T13:15:42Z

tests/integration/ha_tests/helpers.py



-async def get_patroni_setting(ops_test: OpsTest, setting: str) -> Optional[int]:
+async def get_patroni_setting(ops_test: OpsTest, setting: str, tls: bool = False) -> Optional[int]:


We don't really need this, but kept for compatibility.

dragomirp · 2024-07-25T13:17:58Z

tests/integration/test_tls.py

+        # Pause Patroni so it doesn't wipe the custom changes
+        await change_patroni_setting(ops_test, "pause", True, use_random_unit=True, tls=True)


Pause and upause Patroni, instead of trying to tweak the loop_wait

Great solution!

dragomirp · 2024-07-25T13:18:54Z

tests/integration/test_tls.py

+        # Pause Patroni so it doesn't wipe the custom changes
+        await change_patroni_setting(ops_test, "pause", True, use_random_unit=True, tls=True)
+
+    async with ops_test.fast_forward("24h"):


Disable update_status so it doesn't run while we're tweaking the cluster.

dragomirp · 2024-07-25T13:21:24Z

tests/integration/test_tls.py

+        for attempt in Retrying(stop=stop_after_attempt(10), wait=wait_fixed(5), reraise=True):
+            with attempt:
+                logger.info("Trying to grep for rewind logs.")
+                await run_command_on_unit(
+                    ops_test,
+                    primary,
+                    "grep 'connection authorized: user=rewind database=postgres SSL enabled' /var/snap/charmed-postgresql/common/var/log/postgresql/postgresql-*.log",
+                )


Retry for about a minute for the logs to flush.

dragomirp · 2024-07-25T13:22:11Z

tests/integration/test_tls.py

+        await change_patroni_setting(ops_test, "pause", False, use_random_unit=True, tls=True)

+    async with ops_test.fast_forward():


Reenable Patroni and ffwd for the removal test.

dragomirp · 2024-07-25T13:23:24Z

TLS test for amd64 on Juju 3.4.4 succeeded 5 times in a row here

dragomirp · 2024-07-25T15:26:06Z

Unrelated to the PR, but do we need the test_restart_machine test? It seems to be constantly skipped.

marceloneppel

LGTM!

marceloneppel · 2024-07-25T18:47:22Z

tests/integration/test_tls.py

+        # Pause Patroni so it doesn't wipe the custom changes
+        await change_patroni_setting(ops_test, "pause", True, use_random_unit=True, tls=True)


Great solution!

marceloneppel · 2024-07-25T18:48:48Z

Unrelated to the PR, but do we need the test_restart_machine test? It seems to be constantly skipped.

I think it can be removed nowadays because the issue related to it was solved a long time ago.

taurus-forever

Interesting approach. 🤞

Increase Patroni's loop_wait

e11460c

github-actions bot added the Libraries: OK label Jul 18, 2024

dragomirp changed the title ~~[DPE-4533] Increase Patroni's loop_wait~~ [DPE-4533] Increase Patroni's loop_wait in TLS test Jul 18, 2024

dragomirp added 2 commits July 18, 2024 15:33

Allow setting and getting configs using tls

9c1d30b

Wait for loop to clear

e2b9025

dragomirp force-pushed the dpe-4533-flaky-test branch from 255defd to e2b9025 Compare July 18, 2024 14:44

dragomirp added 4 commits July 19, 2024 16:06

Mover loop_wait restore

20ec17e

Merge branch 'main' into dpe-4533-flaky-test

2e99c83

Merge branch 'main' into dpe-4533-flaky-test

4ff977d

Pause patroni

a75bfa2

dragomirp changed the title ~~[DPE-4533] Increase Patroni's loop_wait in TLS test~~ [DPE-4533] Pause Patroni in the TLS test Jul 24, 2024

dragomirp force-pushed the dpe-4533-flaky-test branch from 2899902 to a75bfa2 Compare July 24, 2024 12:34

dragomirp added 2 commits July 24, 2024 15:38

Skip on fail

a7a7551

Increase update status interval

a576119

dragomirp force-pushed the dpe-4533-flaky-test branch from 3a8f445 to a576119 Compare July 24, 2024 14:10

dragomirp added 2 commits July 24, 2024 20:10

Further increase the update hook interval

b3a94e8

Different ffwd context

2cec900

dragomirp force-pushed the dpe-4533-flaky-test branch from edccdc1 to 03863f4 Compare July 24, 2024 22:03

Switch back ffwding

d5bfb7d

dragomirp force-pushed the dpe-4533-flaky-test branch from 03863f4 to d5bfb7d Compare July 25, 2024 08:33

Retry grep

476c8a9

dragomirp force-pushed the dpe-4533-flaky-test branch from f903463 to 476c8a9 Compare July 25, 2024 09:27

Reraise and log retries

e695b21

dragomirp commented Jul 25, 2024

View reviewed changes

dragomirp marked this pull request as ready for review July 25, 2024 13:23

dragomirp requested review from marceloneppel, taurus-forever and lucasgameiroborges July 25, 2024 13:23

Union signature

34a9606

marceloneppel approved these changes Jul 25, 2024

View reviewed changes

taurus-forever approved these changes Jul 25, 2024

View reviewed changes

dragomirp merged commit 4816eb5 into main Jul 26, 2024

dragomirp deleted the dpe-4533-flaky-test branch July 26, 2024 08:32

taurus-forever mentioned this pull request Jul 29, 2024

[DPE-4533] Pause Patroni in the TLS test canonical/postgresql-k8s-operator#588

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[DPE-4533] Pause Patroni in the TLS test #534

[DPE-4533] Pause Patroni in the TLS test #534

Uh oh!

dragomirp commented Jul 18, 2024 •

edited

Loading

Uh oh!

codecov bot commented Jul 18, 2024 •

edited

Loading

Uh oh!

dragomirp Jul 25, 2024

Uh oh!

dragomirp Jul 25, 2024

Uh oh!

marceloneppel Jul 25, 2024

Uh oh!

dragomirp Jul 25, 2024

Uh oh!

dragomirp Jul 25, 2024

Uh oh!

dragomirp Jul 25, 2024

Uh oh!

dragomirp commented Jul 25, 2024

Uh oh!

dragomirp commented Jul 25, 2024

Uh oh!

marceloneppel left a comment

Uh oh!

marceloneppel Jul 25, 2024

Uh oh!

marceloneppel commented Jul 25, 2024

Uh oh!

taurus-forever left a comment

Uh oh!

Uh oh!



		async def get_patroni_setting(ops_test: OpsTest, setting: str) -> Optional[int]:
		async def get_patroni_setting(ops_test: OpsTest, setting: str, tls: bool = False) -> Optional[int]:

		# Pause Patroni so it doesn't wipe the custom changes
		await change_patroni_setting(ops_test, "pause", True, use_random_unit=True, tls=True)

		await change_patroni_setting(ops_test, "pause", False, use_random_unit=True, tls=True)

		async with ops_test.fast_forward():

[DPE-4533] Pause Patroni in the TLS test #534

[DPE-4533] Pause Patroni in the TLS test #534

Uh oh!

Conversation

dragomirp commented Jul 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Jul 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

dragomirp Jul 25, 2024

Choose a reason for hiding this comment

Uh oh!

dragomirp Jul 25, 2024

Choose a reason for hiding this comment

Uh oh!

marceloneppel Jul 25, 2024

Choose a reason for hiding this comment

Uh oh!

dragomirp Jul 25, 2024

Choose a reason for hiding this comment

Uh oh!

dragomirp Jul 25, 2024

Choose a reason for hiding this comment

Uh oh!

dragomirp Jul 25, 2024

Choose a reason for hiding this comment

Uh oh!

dragomirp commented Jul 25, 2024

Uh oh!

dragomirp commented Jul 25, 2024

Uh oh!

marceloneppel left a comment

Choose a reason for hiding this comment

Uh oh!

marceloneppel Jul 25, 2024

Choose a reason for hiding this comment

Uh oh!

marceloneppel commented Jul 25, 2024

Uh oh!

taurus-forever left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dragomirp commented Jul 18, 2024 •

edited

Loading

codecov bot commented Jul 18, 2024 •

edited

Loading