Skip to content

Conversation

taurus-forever
Copy link
Contributor

@taurus-forever taurus-forever commented Jun 6, 2025

Issue

Reported as https://warthogs.atlassian.net/browse/DPE-6662

The pgBackRest activity (backup creation, stanza initialization, etc) could take minutes,
as a result every minute logs rotation could move the current log A.log to A.log_$date.log and
will be moved the next minute further as logrorate rule operates all *.log files in the folder:

/var/snap/charmed-postgresql/common/var/log/pgbackrest/*.log { ...

It results in:

-rw------- 1 postgres postgres     0 Feb 18 09:25 all-server.log
-rw------- 1 postgres postgres     0 Feb 18 09:19 all-server.log-20250218_09:18.log
-rw------- 1 postgres postgres     0 Feb 18 09:20 all-server.log-20250218_09:18.log-20250218_09:19.log
-rw------- 1 postgres postgres     0 Feb 18 09:21 all-server.log-20250218_09:18.log-20250218_09:19.log-20250218_09:20.log
-rw------- 1 postgres postgres     0 Feb 18 09:22 all-server.log-20250218_09:18.log-20250218_09:19.log-20250218_09:20.log-20250218_09:21.log
-rw------- 1 postgres postgres     0 Feb 18 09:23 all-server.log-20250218_09:18.log-20250218_09:19.log-20250218_09:20.log-20250218_09:21.log-20250218_09:22.log
-rw------- 1 postgres postgres  1793 Feb 18 09:24 all-server.log-20250218_09:18.log-20250218_09:19.log-20250218_09:20.log-20250218_09:21.log-20250218_09:22.log-20250218_09:23.log

The same time pgBackRest files names are wide and complex and depends on Juju application name (and more),
therefor there are no K.I.S.S way to reconfigure logrotare rules avoiding *.log.

Solution

  • move the log file to the new name without .log suffix
  • use datetime format matching MySQL charms

Example:

-rw------- 1 postgres postgres    12 Feb 18 09:28 all-server.log
-rw------- 1 postgres postgres   322 Feb 18 09:29 all-server.log-20250218_0918
-rw------- 1 postgres postgres  7344 Feb 18 09:30 all-server.log-20250218_0918

Checklist

  • I have added or updated any relevant documentation.
  • I have cleaned any remaining cloud resources from my accounts.

Copy link

codecov bot commented Jun 6, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.22%. Comparing base (2bb989c) to head (235ab76).
Report is 6 commits behind head on 16/edge.

Additional details and impacted files
@@           Coverage Diff            @@
##           16/edge     #939   +/-   ##
========================================
  Coverage    70.22%   70.22%           
========================================
  Files           16       16           
  Lines         3725     3725           
  Branches       542      542           
========================================
  Hits          2616     2616           
  Misses         974      974           
  Partials       135      135           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

The pgBackRest activity (backup creation, stanza initialisation, etc) could take minutes,
as a result every minute logs rotation could move the current log A.log to A.log_$date.log
and will be moved further as logrorate rule operates all *.log files in the folder:

> /var/snap/charmed-postgresql/common/var/log/pgbackrest/*.log { ...

It results in:

-rw------- 1 postgres postgres     0 Feb 18 09:25 all-server.log
-rw------- 1 postgres postgres     0 Feb 18 09:19 all-server.log-20250218_09:18.log
-rw------- 1 postgres postgres     0 Feb 18 09:20 all-server.log-20250218_09:18.log-20250218_09:19.log
-rw------- 1 postgres postgres     0 Feb 18 09:21 all-server.log-20250218_09:18.log-20250218_09:19.log-20250218_09:20.log
-rw------- 1 postgres postgres     0 Feb 18 09:22 all-server.log-20250218_09:18.log-20250218_09:19.log-20250218_09:20.log-20250218_09:21.log
-rw------- 1 postgres postgres     0 Feb 18 09:23 all-server.log-20250218_09:18.log-20250218_09:19.log-20250218_09:20.log-20250218_09:21.log-20250218_09:22.log
-rw------- 1 postgres postgres  1793 Feb 18 09:24 all-server.log-20250218_09:18.log-20250218_09:19.log-20250218_09:20.log-20250218_09:21.log-20250218_09:22.log-20250218_09:23.log

The poposed fix:
 * move the log file to the new name without .log suffix.
 * use datetime format matching MySQL charms

Example:

-rw------- 1 postgres postgres    12 Feb 18 09:28 all-server.log
-rw------- 1 postgres postgres   322 Feb 18 09:29 all-server.log-20250218_0918
-rw------- 1 postgres postgres  7344 Feb 18 09:30 all-server.log-20250218_0918
@taurus-forever taurus-forever force-pushed the alutay/fix_pgbackrest_logrotate branch from d685717 to 235ab76 Compare June 6, 2025 10:42
@taurus-forever taurus-forever marked this pull request as ready for review June 6, 2025 10:46
@taurus-forever taurus-forever changed the title DPE-6662 Fix pgbackrest logs names on logs rotation [DPE-6662] Fix pgbackrest logs names on logs rotation Jun 6, 2025
@taurus-forever taurus-forever added the bug Something isn't working as expected label Jun 6, 2025
@taurus-forever taurus-forever merged commit ffa77b2 into 16/edge Jun 9, 2025
400 of 417 checks passed
@taurus-forever taurus-forever deleted the alutay/fix_pgbackrest_logrotate branch June 9, 2025 20:49
taurus-forever added a commit that referenced this pull request Jun 9, 2025
The pgBackRest activity (backup creation, stanza initialisation, etc) could take minutes,
as a result every minute logs rotation could move the current log A.log to A.log_$date.log
and will be moved further as logrorate rule operates all *.log files in the folder:

> /var/snap/charmed-postgresql/common/var/log/pgbackrest/*.log { ...

It results in:

-rw------- 1 postgres postgres     0 Feb 18 09:25 all-server.log
-rw------- 1 postgres postgres     0 Feb 18 09:19 all-server.log-20250218_09:18.log
-rw------- 1 postgres postgres     0 Feb 18 09:20 all-server.log-20250218_09:18.log-20250218_09:19.log
-rw------- 1 postgres postgres     0 Feb 18 09:21 all-server.log-20250218_09:18.log-20250218_09:19.log-20250218_09:20.log
-rw------- 1 postgres postgres     0 Feb 18 09:22 all-server.log-20250218_09:18.log-20250218_09:19.log-20250218_09:20.log-20250218_09:21.log
-rw------- 1 postgres postgres     0 Feb 18 09:23 all-server.log-20250218_09:18.log-20250218_09:19.log-20250218_09:20.log-20250218_09:21.log-20250218_09:22.log
-rw------- 1 postgres postgres  1793 Feb 18 09:24 all-server.log-20250218_09:18.log-20250218_09:19.log-20250218_09:20.log-20250218_09:21.log-20250218_09:22.log-20250218_09:23.log

The poposed fix:
 * move the log file to the new name without .log suffix.
 * use datetime format matching MySQL charms

Example:

-rw------- 1 postgres postgres    12 Feb 18 09:28 all-server.log
-rw------- 1 postgres postgres   322 Feb 18 09:29 all-server.log-20250218_0918
-rw------- 1 postgres postgres  7344 Feb 18 09:30 all-server.log-20250218_0918

(cherry picked from commit ffa77b2)
taurus-forever added a commit that referenced this pull request Jun 11, 2025
The pgBackRest activity (backup creation, stanza initialisation, etc) could take minutes,
as a result every minute logs rotation could move the current log A.log to A.log_$date.log
and will be moved further as logrorate rule operates all *.log files in the folder:

> /var/snap/charmed-postgresql/common/var/log/pgbackrest/*.log { ...

It results in:

-rw------- 1 postgres postgres     0 Feb 18 09:25 all-server.log
-rw------- 1 postgres postgres     0 Feb 18 09:19 all-server.log-20250218_09:18.log
-rw------- 1 postgres postgres     0 Feb 18 09:20 all-server.log-20250218_09:18.log-20250218_09:19.log
-rw------- 1 postgres postgres     0 Feb 18 09:21 all-server.log-20250218_09:18.log-20250218_09:19.log-20250218_09:20.log
-rw------- 1 postgres postgres     0 Feb 18 09:22 all-server.log-20250218_09:18.log-20250218_09:19.log-20250218_09:20.log-20250218_09:21.log
-rw------- 1 postgres postgres     0 Feb 18 09:23 all-server.log-20250218_09:18.log-20250218_09:19.log-20250218_09:20.log-20250218_09:21.log-20250218_09:22.log
-rw------- 1 postgres postgres  1793 Feb 18 09:24 all-server.log-20250218_09:18.log-20250218_09:19.log-20250218_09:20.log-20250218_09:21.log-20250218_09:22.log-20250218_09:23.log

The poposed fix:
 * move the log file to the new name without .log suffix.
 * use datetime format matching MySQL charms

Example:

-rw------- 1 postgres postgres    12 Feb 18 09:28 all-server.log
-rw------- 1 postgres postgres   322 Feb 18 09:29 all-server.log-20250218_0918
-rw------- 1 postgres postgres  7344 Feb 18 09:30 all-server.log-20250218_0918

(cherry picked from commit ffa77b2)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working as expected
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants