Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Barman - check server returns wrong results for Replication Slot #1024

Closed
thoro opened this issue Oct 14, 2024 · 11 comments
Closed

Barman - check server returns wrong results for Replication Slot #1024

thoro opened this issue Oct 14, 2024 · 11 comments
Labels
bug triage

Comments

@thoro
Copy link
Contributor

thoro commented Oct 14, 2024 •
edited
Loading

Hi,

I have a configuration that sets the following variables:

conninfo = host=backup-db port=5432 user=postgres
primary_conninfo= host=live-db port=5432 user=postgres
streaming_conninfo = host=live-db port=5432 user=postgres
wal_streaming_conninfo = host=live-db port=5432 user=postgres
wal_conninfo = host=live-db port=5432 user=postgres
ssh_command = ssh -q -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no postgres@backup-db

The issue is that the check server always return that the replication slot is not created, even though receive-wal (started through barman cron) is active:

	replication slot: FAILED (replication slot 'barman_streaming_replication' doesn't exist. Please execute 'barman receive-wal --create-slot database')

I also have verified the replication slot on the main db, and it's active and working with exactly the same name.

I tracked it down I think to the change of primary_conninfo (I upgraded from 2.11). I changed a bit in postgres.py:~1709

self.primary = PostgreSQLConnection(self.primary_conninfo, slot_name=slot_name)

and also added the function:

 def fetch_remote_status(self):
        return self.primary.fetch_remote_status();

This allows the check to finish successfully. Can someone take a more detailed look?
@martinmarques
Copy link
Contributor

Can't really say why you are getting that error based on the little information here. It's not clear what backup method you are using, although it looks like it's rsync based on the ssh_command, and we are not sure the role of each of the two nodes live-db and backup-db.

I suggest looking at the output from barman diagnose and the barman and postgres logs to find the culprit. There is some misconfiguration, but I can't see it. Maybe if you provided the output from barman show-server I could have a look at that and see if it's reproducible.

@thoro
Copy link
Contributor Author

thoro commented Oct 15, 2024

I will prepare that, once our backup is done and I can revert my local changes.

Yes it's using rsync, from standby db.

live-db is the primary, which is active, and is really remote (ping 20-30ms)
backup-db is local to the barman server and a replication db.

I dug a bit into the outputs generated from fetch_remote_status and it is never connecting to the primary_conninfo and only uses the conninfo to extract the barman check results, there it can't find the replication_slot, since it doesn't exist.

@martinmarques
Copy link
Contributor

Which version did you upgrade to?

@thoro
Copy link
Contributor Author

thoro commented Oct 15, 2024 

 barman -v
3.11.1 Barman by EnterpriseDB (www.enterprisedb.com)

@thoro
Copy link
Contributor Author

thoro commented Oct 15, 2024 

barman check platon.creamfinance.com

Server database:
	PostgreSQL: OK
	superuser or standard user with backup privileges: OK
	PostgreSQL streaming: OK
	wal_level: OK
	PostgreSQL server is standby: OK
	Primary server is not a standby: OK
	Primary and standby have same system ID: OK
	replication slot: FAILED (replication slot 'barman_streaming_replication' doesn't exist. Please execute 'barman receive-wal --create-slot database')
	directories: OK
	retention policy settings: OK
	backup maximum age: OK (no last_backup_maximum_age provided)
	backup minimum size: OK (7.5 TiB)
	wal maximum age: OK (no last_wal_maximum_age provided)
	wal size: OK (0 B)
	compression settings: OK
	failed backups: FAILED (there are 6 failed backups)
	minimum redundancy requirements: OK (have 15 backups, expected at least 4)
	ssh: OK (PostgreSQL server)
	systemid coherence: OK
	pg_receivexlog: OK
	pg_receivexlog compatible: OK
	receive-wal running: OK
	archive_mode: OK
	archive_command: OK
	continuous archiving: OK
	archiver errors: OK

barman diagnose

{
    "global": {
        "config": {
            "barman_home": "/var/lib/barman",
            "barman_user": "barman",
            "configuration_files_directory": "/etc/barman.d",
            "errors_list": [],
            "log_file": "/var/lib/barman/barman.log",
            "log_level": "INFO"
        },
        "system_info": {
            "barman_ver": "3.11.1",
            "kernel_ver": "Linux barman-0 5.10.37 #1 SMP Mon May 17 13:15:38 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux",
            "python_executable": "/usr/bin/python3",
            "python_ver": "Python 3.9.18",
            "release": "RedHat Linux Rocky Linux release 9.4 (Blue Onyx)",
            "rsync_ver": "rsync  version 3.2.3  protocol version 31",
            "ssh_ver": "",
            "timestamp": "2024-10-15T14:16:26.025061+00:00"
        }
    },
    "models": {},
    "servers": {
        "database": {
            "active_model": null,
            "backups": {
                "20240609T183005": {
                    "backup_id": "20240609T183005",
                    "backup_label": "'START WAL LOCATION: 1A5C4/D4142F68 (file 000000090001A5C4000000D4)\\nCHECKPOINT LOCATION: 1A5C4/D8EBB408\\nBACKUP METHOD: streamed\\nBACKUP FROM: standby\\nSTART TIME: 2024-06-09 18:31:52 UTC\\nLABEL: Barman backup database 20240609T183005\\n'",
                    "begin_offset": 1322856,
                    "begin_time": "2024-06-09T18:30:05.448667+00:00",
                    "begin_wal": "000000090001A5C4000000D4",
                    "begin_xlog": "1A5C4/D4142F68",
                    "children_backup_ids": null,
                    "cluster_size": null,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": {
                        "analysis_time": 16.356708,
                        "analysis_time_per_item": {
                            "pgdata": 16.356708
                        },
                        "copy_time": 71376.250977,
                        "copy_time_per_item": {
                            "pg_control": 0.565026,
                            "pgdata": 71375.681805
                        },
                        "number_of_workers": 4,
                        "serialized_copy_time": 285353.398247,
                        "serialized_copy_time_per_item": {
                            "pg_control": 0.565026,
                            "pgdata": 285352.833221
                        },
                        "total_time": 71392.761057
                    },
                    "data_checksums": null,
                    "deduplicated_size": 3086560838659,
                    "end_offset": 16777064,
                    "end_time": "2024-06-10T14:21:44.969937+00:00",
                    "end_wal": "000000090001A5F900000084",
                    "end_xlog": "1A5F9/84FFFF68",
                    "error": null,
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": 7479831140182,
                    "status": "DONE",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20240616T183005": {
                    "backup_id": "20240616T183005",
                    "backup_label": null,
                    "begin_offset": 2385976,
                    "begin_time": "2024-06-16T18:30:06.356728+00:00",
                    "begin_wal": "000000090001A78C000000C7",
                    "begin_xlog": "1A78C/C7246838",
                    "children_backup_ids": null,
                    "cluster_size": null,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": null,
                    "data_checksums": null,
                    "deduplicated_size": null,
                    "end_offset": null,
                    "end_time": null,
                    "end_wal": null,
                    "end_xlog": null,
                    "error": null,
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": null,
                    "status": "STARTED",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20240623T183005": {
                    "backup_id": "20240623T183005",
                    "backup_label": "'START WAL LOCATION: 1A8EA/A0001140 (file 000000090001A8EA000000A0)\\nCHECKPOINT LOCATION: 1A8EA/ACEB39D8\\nBACKUP METHOD: streamed\\nBACKUP FROM: standby\\nSTART TIME: 2024-06-23 18:33:54 UTC\\nLABEL: Barman backup database 20240623T183005\\n'",
                    "begin_offset": 4416,
                    "begin_time": "2024-06-23T18:30:05.792788+00:00",
                    "begin_wal": "000000090001A8EA000000A0",
                    "begin_xlog": "1A8EA/A0001140",
                    "children_backup_ids": null,
                    "cluster_size": null,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": {
                        "analysis_time": 16.65817,
                        "analysis_time_per_item": {
                            "pgdata": 16.65817
                        },
                        "copy_time": 112167.268942,
                        "copy_time_per_item": {
                            "pg_control": 0.547048,
                            "pgdata": 112166.718455
                        },
                        "number_of_workers": 4,
                        "serialized_copy_time": 448425.34586,
                        "serialized_copy_time_per_item": {
                            "pg_control": 0.547048,
                            "pgdata": 448424.798812
                        },
                        "total_time": 112184.154476
                    },
                    "data_checksums": null,
                    "deduplicated_size": 4476153234432,
                    "end_offset": 16777128,
                    "end_time": "2024-06-25T01:43:39.181695+00:00",
                    "end_wal": "000000090001A9350000002A",
                    "end_xlog": "1A935/2AFFFFA8",
                    "error": null,
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": 7561733216083,
                    "status": "DONE",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20240630T183006": {
                    "backup_id": "20240630T183006",
                    "backup_label": "'START WAL LOCATION: 1AA6A/BA2B0F90 (file 000000090001AA6A000000BA)\\nCHECKPOINT LOCATION: 1AA6A/C0A89410\\nBACKUP METHOD: streamed\\nBACKUP FROM: standby\\nSTART TIME: 2024-06-30 18:30:21 UTC\\nLABEL: Barman backup database 20240630T183006\\n'",
                    "begin_offset": 2822032,
                    "begin_time": "2024-06-30T18:30:06.604137+00:00",
                    "begin_wal": "000000090001AA6A000000BA",
                    "begin_xlog": "1AA6A/BA2B0F90",
                    "children_backup_ids": null,
                    "cluster_size": null,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": null,
                    "data_checksums": null,
                    "deduplicated_size": null,
                    "end_offset": 11287816,
                    "end_time": "2024-07-01T18:13:16.983316+00:00",
                    "end_wal": "000000090001AA9C000000FB",
                    "end_xlog": "1AA9C/FBAC3D08",
                    "error": "failure copying files (data transfer failure)",
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": null,
                    "status": "FAILED",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20240707T183007": {
                    "backup_id": "20240707T183007",
                    "backup_label": "'START WAL LOCATION: 1ABEB/376BEB0 (file 000000090001ABEB00000003)\\nCHECKPOINT LOCATION: 1ABEB/AF17F48\\nBACKUP METHOD: streamed\\nBACKUP FROM: standby\\nSTART TIME: 2024-07-07 18:38:04 UTC\\nLABEL: Barman backup database 20240707T183007\\n'",
                    "begin_offset": 7782064,
                    "begin_time": "2024-07-07T18:30:07.530714+00:00",
                    "begin_wal": "000000090001ABEB00000003",
                    "begin_xlog": "1ABEB/376BEB0",
                    "children_backup_ids": null,
                    "cluster_size": null,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": {
                        "analysis_time": 15.376072,
                        "analysis_time_per_item": {
                            "pgdata": 15.376072
                        },
                        "copy_time": 94403.100679,
                        "copy_time_per_item": {
                            "pg_control": 0.633467,
                            "pgdata": 94402.463781
                        },
                        "number_of_workers": 4,
                        "serialized_copy_time": 377436.871355,
                        "serialized_copy_time_per_item": {
                            "pg_control": 0.633467,
                            "pgdata": 377436.237888
                        },
                        "total_time": 94418.62596
                    },
                    "data_checksums": null,
                    "deduplicated_size": 3898857055232,
                    "end_offset": 16769792,
                    "end_time": "2024-07-08T20:51:42.919105+00:00",
                    "end_wal": "000000090001AC1E000000B2",
                    "end_xlog": "1AC1E/B2FFE300",
                    "error": null,
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": 7630786713427,
                    "status": "DONE",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20240714T183005": {
                    "backup_id": "20240714T183005",
                    "backup_label": "'START WAL LOCATION: 1AD65/14008098 (file 000000090001AD6500000014)\\nCHECKPOINT LOCATION: 1AD65/20865140\\nBACKUP METHOD: streamed\\nBACKUP FROM: standby\\nSTART TIME: 2024-07-14 18:34:35 UTC\\nLABEL: Barman backup database 20240714T183005\\n'",
                    "begin_offset": 32920,
                    "begin_time": "2024-07-14T18:30:05.916499+00:00",
                    "begin_wal": "000000090001AD6500000014",
                    "begin_xlog": "1AD65/14008098",
                    "children_backup_ids": null,
                    "cluster_size": null,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": {
                        "analysis_time": 15.260492,
                        "analysis_time_per_item": {
                            "pgdata": 15.260492
                        },
                        "copy_time": 83220.006164,
                        "copy_time_per_item": {
                            "pg_control": 0.651049,
                            "pgdata": 83219.351738
                        },
                        "number_of_workers": 4,
                        "serialized_copy_time": 332549.868139,
                        "serialized_copy_time_per_item": {
                            "pg_control": 0.651049,
                            "pgdata": 332549.21709
                        },
                        "total_time": 83235.432954
                    },
                    "data_checksums": null,
                    "deduplicated_size": 3089200366593,
                    "end_offset": 16777160,
                    "end_time": "2024-07-15T17:41:51.009926+00:00",
                    "end_wal": "000000090001ADA3000000F5",
                    "end_xlog": "1ADA3/F5FFFFC8",
                    "error": null,
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": 7672907880276,
                    "status": "DONE",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20240721T183004": {
                    "backup_id": "20240721T183004",
                    "backup_label": "'START WAL LOCATION: 1AEFF/2B4E77E0 (file 000000090001AEFF0000002B)\\nCHECKPOINT LOCATION: 1AEFF/37CEA190\\nBACKUP METHOD: streamed\\nBACKUP FROM: standby\\nSTART TIME: 2024-07-21 18:33:23 UTC\\nLABEL: Barman backup database 20240721T183004\\n'",
                    "begin_offset": 5142496,
                    "begin_time": "2024-07-21T18:30:05.000125+00:00",
                    "begin_wal": "000000090001AEFF0000002B",
                    "begin_xlog": "1AEFF/2B4E77E0",
                    "children_backup_ids": null,
                    "cluster_size": null,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": {
                        "analysis_time": 19.363272,
                        "analysis_time_per_item": {
                            "pgdata": 19.363272
                        },
                        "copy_time": 109472.738174,
                        "copy_time_per_item": {
                            "pg_control": 0.600745,
                            "pgdata": 109472.133855
                        },
                        "number_of_workers": 4,
                        "serialized_copy_time": 437720.616288,
                        "serialized_copy_time_per_item": {
                            "pg_control": 0.600745,
                            "pgdata": 437720.015543
                        },
                        "total_time": 109492.207849
                    },
                    "data_checksums": null,
                    "deduplicated_size": 3960074732547,
                    "end_offset": 16772408,
                    "end_time": "2024-07-23T00:58:15.500531+00:00",
                    "end_wal": "000000090001AF400000003B",
                    "end_xlog": "1AF40/3BFFED38",
                    "error": null,
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": 7707152422742,
                    "status": "DONE",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20240728T183005": {
                    "backup_id": "20240728T183005",
                    "backup_label": "'START WAL LOCATION: 1B08A/1B97A288 (file 000000090001B08A0000001B)\\nCHECKPOINT LOCATION: 1B08A/2167A080\\nBACKUP METHOD: streamed\\nBACKUP FROM: standby\\nSTART TIME: 2024-07-28 18:30:10 UTC\\nLABEL: Barman backup database 20240728T183005\\n'",
                    "begin_offset": 9937544,
                    "begin_time": "2024-07-28T18:30:05.561205+00:00",
                    "begin_wal": "000000090001B08A0000001B",
                    "begin_xlog": "1B08A/1B97A288",
                    "children_backup_ids": null,
                    "cluster_size": null,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": {
                        "analysis_time": 19.559101,
                        "analysis_time_per_item": {
                            "pgdata": 19.559101
                        },
                        "copy_time": 71565.32162,
                        "copy_time_per_item": {
                            "pg_control": 0.74575,
                            "pgdata": 71564.57305
                        },
                        "number_of_workers": 4,
                        "serialized_copy_time": 285946.809941,
                        "serialized_copy_time_per_item": {
                            "pg_control": 0.74575,
                            "pgdata": 285946.064191
                        },
                        "total_time": 71584.996769
                    },
                    "data_checksums": null,
                    "deduplicated_size": 2975044666371,
                    "end_offset": 16771800,
                    "end_time": "2024-07-29T14:23:15.137761+00:00",
                    "end_wal": "000000090001B0B700000014",
                    "end_xlog": "1B0B7/14FFEAD8",
                    "error": null,
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": 7745650972502,
                    "status": "DONE",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20240804T183006": {
                    "backup_id": "20240804T183006",
                    "backup_label": "'START WAL LOCATION: 1B222/6009F10 (file 000000090001B22200000006)\\nCHECKPOINT LOCATION: 1B222/138BAED8\\nBACKUP METHOD: streamed\\nBACKUP FROM: standby\\nSTART TIME: 2024-08-04 18:30:06 UTC\\nLABEL: Barman backup database 20240804T183006\\n'",
                    "begin_offset": 40720,
                    "begin_time": "2024-08-04T18:30:06.322842+00:00",
                    "begin_wal": "000000090001B22200000006",
                    "begin_xlog": "1B222/6009F10",
                    "children_backup_ids": null,
                    "cluster_size": null,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": {
                        "analysis_time": 17.672512,
                        "analysis_time_per_item": {
                            "pgdata": 17.672512
                        },
                        "copy_time": 69349.169088,
                        "copy_time_per_item": {
                            "pg_control": 0.58291,
                            "pgdata": 69348.581819
                        },
                        "number_of_workers": 4,
                        "serialized_copy_time": 277071.470014,
                        "serialized_copy_time_per_item": {
                            "pg_control": 0.58291,
                            "pgdata": 277070.887104
                        },
                        "total_time": 69367.031965
                    },
                    "data_checksums": null,
                    "deduplicated_size": 2993009435647,
                    "end_offset": 16776672,
                    "end_time": "2024-08-05T13:46:13.518348+00:00",
                    "end_wal": "000000090001B25400000008",
                    "end_xlog": "1B254/8FFFDE0",
                    "error": null,
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": 7785518233426,
                    "status": "DONE",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20240811T183005": {
                    "backup_id": "20240811T183005",
                    "backup_label": "'START WAL LOCATION: 1B3AE/7005E620 (file 000000090001B3AE00000070)\\nCHECKPOINT LOCATION: 1B3AE/7C92A398\\nBACKUP METHOD: streamed\\nBACKUP FROM: standby\\nSTART TIME: 2024-08-11 18:34:33 UTC\\nLABEL: Barman backup database 20240811T183005\\n'",
                    "begin_offset": 386592,
                    "begin_time": "2024-08-11T18:30:05.552970+00:00",
                    "begin_wal": "000000090001B3AE00000070",
                    "begin_xlog": "1B3AE/7005E620",
                    "children_backup_ids": null,
                    "cluster_size": null,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": {
                        "analysis_time": 17.89297,
                        "analysis_time_per_item": {
                            "pgdata": 17.89297
                        },
                        "copy_time": 81543.999661,
                        "copy_time_per_item": {
                            "pg_control": 0.604855,
                            "pgdata": 81543.390486
                        },
                        "number_of_workers": 4,
                        "serialized_copy_time": 326081.865883,
                        "serialized_copy_time_per_item": {
                            "pg_control": 0.604855,
                            "pgdata": 326081.261028
                        },
                        "total_time": 81562.027804
                    },
                    "data_checksums": null,
                    "deduplicated_size": 3404260926466,
                    "end_offset": 16770744,
                    "end_time": "2024-08-12T17:13:55.783064+00:00",
                    "end_wal": "000000090001B3F600000067",
                    "end_xlog": "1B3F6/67FFE6B8",
                    "error": null,
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": 7834560509781,
                    "status": "DONE",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20240818T183005": {
                    "backup_id": "20240818T183005",
                    "backup_label": "'START WAL LOCATION: 1B54C/1F4D0560 (file 000000090001B54C0000001F)\\nCHECKPOINT LOCATION: 1B54C/26AE6A48\\nBACKUP METHOD: streamed\\nBACKUP FROM: standby\\nSTART TIME: 2024-08-18 18:34:35 UTC\\nLABEL: Barman backup database 20240818T183005\\n'",
                    "begin_offset": 5047648,
                    "begin_time": "2024-08-18T18:30:05.729827+00:00",
                    "begin_wal": "000000090001B54C0000001F",
                    "begin_xlog": "1B54C/1F4D0560",
                    "children_backup_ids": null,
                    "cluster_size": null,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": {
                        "analysis_time": 13.750049,
                        "analysis_time_per_item": {
                            "pgdata": 13.750049
                        },
                        "copy_time": 68761.368279,
                        "copy_time_per_item": {
                            "pg_control": 0.634499,
                            "pgdata": 68760.73097
                        },
                        "number_of_workers": 4,
                        "serialized_copy_time": 274900.695622,
                        "serialized_copy_time_per_item": {
                            "pg_control": 0.634499,
                            "pgdata": 274900.061123
                        },
                        "total_time": 68775.239351
                    },
                    "data_checksums": null,
                    "deduplicated_size": 2931264742403,
                    "end_offset": 16776592,
                    "end_time": "2024-08-19T13:40:50.434669+00:00",
                    "end_wal": "000000090001B577000000BF",
                    "end_xlog": "1B577/BFFFFD90",
                    "error": null,
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": 7881464231766,
                    "status": "DONE",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20240825T183005": {
                    "backup_id": "20240825T183005",
                    "backup_label": null,
                    "begin_offset": 11563136,
                    "begin_time": "2024-08-25T18:30:06.110635+00:00",
                    "begin_wal": "000000090001B6FD0000000F",
                    "begin_xlog": "1B6FD/FB07080",
                    "children_backup_ids": null,
                    "cluster_size": null,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": {
                        "analysis_time": 15.530258,
                        "analysis_time_per_item": {
                            "pgdata": 15.530258
                        },
                        "copy_time": 68511.849061,
                        "copy_time_per_item": {
                            "pg_control": 0.558343,
                            "pgdata": 68511.286608
                        },
                        "number_of_workers": 4,
                        "serialized_copy_time": 273992.966628,
                        "serialized_copy_time_per_item": {
                            "pg_control": 0.558343,
                            "pgdata": 273992.408285
                        },
                        "total_time": 68527.520708
                    },
                    "data_checksums": null,
                    "deduplicated_size": null,
                    "end_offset": null,
                    "end_time": null,
                    "end_wal": null,
                    "end_xlog": null,
                    "error": "failure issuing start backup command (Connection lost, reconnection not allowed)",
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": null,
                    "status": "FAILED",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20240901T183006": {
                    "backup_id": "20240901T183006",
                    "backup_label": "'START WAL LOCATION: 1B895/B79CB518 (file 000000090001B895000000B7)\\nCHECKPOINT LOCATION: 1B895/C0AB8CF0\\nBACKUP METHOD: streamed\\nBACKUP FROM: standby\\nSTART TIME: 2024-09-01 18:31:50 UTC\\nLABEL: Barman backup database 20240901T183006\\n'",
                    "begin_offset": 10269976,
                    "begin_time": "2024-09-01T18:30:06.594955+00:00",
                    "begin_wal": "000000090001B895000000B7",
                    "begin_xlog": "1B895/B79CB518",
                    "children_backup_ids": null,
                    "cluster_size": null,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": {
                        "analysis_time": 19.06149,
                        "analysis_time_per_item": {
                            "pgdata": 19.06149
                        },
                        "copy_time": 77345.294907,
                        "copy_time_per_item": {
                            "pg_control": 0.569102,
                            "pgdata": 77344.72159
                        },
                        "number_of_workers": 4,
                        "serialized_copy_time": 309304.196981,
                        "serialized_copy_time_per_item": {
                            "pg_control": 0.569102,
                            "pgdata": 309303.627879
                        },
                        "total_time": 77364.523472
                    },
                    "data_checksums": null,
                    "deduplicated_size": 3123054019588,
                    "end_offset": 16771608,
                    "end_time": "2024-09-02T16:01:14.742563+00:00",
                    "end_wal": "000000090001B8CF000000C9",
                    "end_xlog": "1B8CF/C9FFEA18",
                    "error": null,
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": 7963900213079,
                    "status": "DONE",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20240908T183004": {
                    "backup_id": "20240908T183004",
                    "backup_label": "'START WAL LOCATION: 1BA31/A5332938 (file 000000090001BA31000000A5)\\nCHECKPOINT LOCATION: 1BA31/B4246DF8\\nBACKUP METHOD: streamed\\nBACKUP FROM: standby\\nSTART TIME: 2024-09-08 18:32:40 UTC\\nLABEL: Barman backup database 20240908T183004\\n'",
                    "begin_offset": 3352888,
                    "begin_time": "2024-09-08T18:30:04.515009+00:00",
                    "begin_wal": "000000090001BA31000000A5",
                    "begin_xlog": "1BA31/A5332938",
                    "children_backup_ids": null,
                    "cluster_size": null,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": {
                        "analysis_time": 19.40603,
                        "analysis_time_per_item": {
                            "pgdata": 19.40603
                        },
                        "copy_time": 65250.211487,
                        "copy_time_per_item": {
                            "pg_control": 1.67603,
                            "pgdata": 65248.531593
                        },
                        "number_of_workers": 4,
                        "serialized_copy_time": 260943.226061,
                        "serialized_copy_time_per_item": {
                            "pg_control": 1.67603,
                            "pgdata": 260941.550031
                        },
                        "total_time": 65269.754553
                    },
                    "data_checksums": null,
                    "deduplicated_size": 3033526658051,
                    "end_offset": 16777136,
                    "end_time": "2024-09-09T12:40:30.407566+00:00",
                    "end_wal": "000000090001BA7200000034",
                    "end_xlog": "1BA72/34FFFFB0",
                    "error": null,
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": 8005960628054,
                    "status": "DONE",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20240915T183004": {
                    "backup_id": "20240915T183004",
                    "backup_label": null,
                    "begin_offset": 1123472,
                    "begin_time": "2024-09-15T18:30:04.915059+00:00",
                    "begin_wal": "000000090001BBE500000072",
                    "begin_xlog": "1BBE5/72112490",
                    "children_backup_ids": null,
                    "cluster_size": null,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": null,
                    "data_checksums": null,
                    "deduplicated_size": null,
                    "end_offset": null,
                    "end_time": null,
                    "end_wal": null,
                    "end_xlog": null,
                    "error": null,
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": null,
                    "status": "STARTED",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20240916T161627": {
                    "backup_id": "20240916T161627",
                    "backup_label": "'START WAL LOCATION: 1BC1F/46C301F8 (file 000000090001BC1F00000046)\\nCHECKPOINT LOCATION: 1BC1F/53453978\\nBACKUP METHOD: streamed\\nBACKUP FROM: standby\\nSTART TIME: 2024-09-16 16:18:48 UTC\\nLABEL: Barman backup database 20240916T161627\\n'",
                    "begin_offset": 12780024,
                    "begin_time": "2024-09-16T16:16:27.917435+00:00",
                    "begin_wal": "000000090001BC1F00000046",
                    "begin_xlog": "1BC1F/46C301F8",
                    "children_backup_ids": null,
                    "cluster_size": null,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": {
                        "analysis_time": 12.844616,
                        "analysis_time_per_item": {
                            "pgdata": 12.844616
                        },
                        "copy_time": 95127.352301,
                        "copy_time_per_item": {
                            "pg_control": 0.505006,
                            "pgdata": 95126.842432
                        },
                        "number_of_workers": 4,
                        "serialized_copy_time": 380055.396802,
                        "serialized_copy_time_per_item": {
                            "pg_control": 0.505006,
                            "pgdata": 380054.891796
                        },
                        "total_time": 95140.384747
                    },
                    "data_checksums": null,
                    "deduplicated_size": 3007332436689,
                    "end_offset": 16776224,
                    "end_time": "2024-09-17T18:44:29.022219+00:00",
                    "end_wal": "000000090001BC720000006D",
                    "end_xlog": "1BC72/6DFFFC20",
                    "error": null,
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": 8052249294679,
                    "status": "DONE",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20240922T183008": {
                    "backup_id": "20240922T183008",
                    "backup_label": "'START WAL LOCATION: 1BDA4/8AA99BD0 (file 000000090001BDA40000008A)\\nCHECKPOINT LOCATION: 1BDA4/97D82D70\\nBACKUP METHOD: streamed\\nBACKUP FROM: standby\\nSTART TIME: 2024-09-22 18:37:26 UTC\\nLABEL: Barman backup database 20240922T183008\\n'",
                    "begin_offset": 11115472,
                    "begin_time": "2024-09-22T18:30:09.107525+00:00",
                    "begin_wal": "000000090001BDA40000008A",
                    "begin_xlog": "1BDA4/8AA99BD0",
                    "children_backup_ids": null,
                    "cluster_size": null,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": {
                        "analysis_time": 15.257874,
                        "analysis_time_per_item": {
                            "pgdata": 15.257874
                        },
                        "copy_time": 117734.689962,
                        "copy_time_per_item": {
                            "pg_control": 0.610337,
                            "pgdata": 117734.075218
                        },
                        "number_of_workers": 4,
                        "serialized_copy_time": 470646.533831,
                        "serialized_copy_time_per_item": {
                            "pg_control": 0.610337,
                            "pgdata": 470645.923494
                        },
                        "total_time": 117750.059014
                    },
                    "data_checksums": null,
                    "deduplicated_size": 3033211495428,
                    "end_offset": 16772992,
                    "end_time": "2024-09-24T03:19:56.323013+00:00",
                    "end_wal": "000000090001BE0E000000BE",
                    "end_xlog": "1BE0E/BEFFEF80",
                    "error": null,
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": 8090872477527,
                    "status": "DONE",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20240929T183004": {
                    "backup_id": "20240929T183004",
                    "backup_label": "'START WAL LOCATION: 1BF7B/9A6017E8 (file 000000090001BF7B0000009A)\\nCHECKPOINT LOCATION: 1BF7B/A11CC110\\nBACKUP METHOD: streamed\\nBACKUP FROM: standby\\nSTART TIME: 2024-09-29 18:38:59 UTC\\nLABEL: Barman backup database 20240929T183004\\n'",
                    "begin_offset": 6297576,
                    "begin_time": "2024-09-29T18:30:04.918590+00:00",
                    "begin_wal": "000000090001BF7B0000009A",
                    "begin_xlog": "1BF7B/9A6017E8",
                    "children_backup_ids": null,
                    "cluster_size": null,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": {
                        "analysis_time": 18.555107,
                        "analysis_time_per_item": {
                            "pgdata": 18.555107
                        },
                        "copy_time": 68114.706303,
                        "copy_time_per_item": {
                            "pg_control": 1.741874,
                            "pgdata": 68112.960147
                        },
                        "number_of_workers": 4,
                        "serialized_copy_time": 272352.342498,
                        "serialized_copy_time_per_item": {
                            "pg_control": 1.741874,
                            "pgdata": 272350.600624
                        },
                        "total_time": 68133.407356
                    },
                    "data_checksums": null,
                    "deduplicated_size": 3103067728592,
                    "end_offset": 16775184,
                    "end_time": "2024-09-30T13:34:33.777638+00:00",
                    "end_wal": "000000090001BFB60000008B",
                    "end_xlog": "1BFB6/8BFFF810",
                    "error": null,
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": 8138505934678,
                    "status": "DONE",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20241006T183005": {
                    "backup_id": "20241006T183005",
                    "backup_label": "'START WAL LOCATION: 1C150/2B0094B8 (file 000000090001C1500000002B)\\nCHECKPOINT LOCATION: 1C150/3783C420\\nBACKUP METHOD: streamed\\nBACKUP FROM: standby\\nSTART TIME: 2024-10-06 18:36:33 UTC\\nLABEL: Barman backup database 20241006T183005\\n'",
                    "begin_offset": 38072,
                    "begin_time": "2024-10-06T18:30:06.416744+00:00",
                    "begin_wal": "000000090001C1500000002B",
                    "begin_xlog": "1C150/2B0094B8",
                    "children_backup_ids": null,
                    "cluster_size": null,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": {
                        "analysis_time": 16.866176,
                        "analysis_time_per_item": {
                            "pgdata": 16.866176
                        },
                        "copy_time": 70660.499387,
                        "copy_time_per_item": {
                            "pg_control": 0.554184,
                            "pgdata": 70659.94156
                        },
                        "number_of_workers": 4,
                        "serialized_copy_time": 282539.874658,
                        "serialized_copy_time_per_item": {
                            "pg_control": 0.554184,
                            "pgdata": 282539.320474
                        },
                        "total_time": 70677.475947
                    },
                    "data_checksums": null,
                    "deduplicated_size": 3145598583809,
                    "end_offset": 16777184,
                    "end_time": "2024-10-07T14:14:31.820810+00:00",
                    "end_wal": "000000090001C19A00000005",
                    "end_xlog": "1C19A/5FFFFE0",
                    "error": null,
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": 8188866516820,
                    "status": "DONE",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20241014T174246": {
                    "backup_id": "20241014T174246",
                    "backup_label": "'START WAL LOCATION: 1C37C/2A040CF8 (file 000000090001C37C0000002A)\\nCHECKPOINT LOCATION: 1C37C/36853C50\\nBACKUP METHOD: streamed\\nBACKUP FROM: standby\\nSTART TIME: 2024-10-14 17:42:47 UTC\\nLABEL: Barman backup database 20241014T174246\\n'",
                    "begin_offset": 265464,
                    "begin_time": "2024-10-14T17:42:47.097071+00:00",
                    "begin_wal": "000000090001C37C0000002A",
                    "begin_xlog": "1C37C/2A040CF8",
                    "children_backup_ids": null,
                    "cluster_size": 8241226958914,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": null,
                    "data_checksums": "off",
                    "deduplicated_size": null,
                    "end_offset": 16771760,
                    "end_time": "2024-10-14T17:43:03.578024+00:00",
                    "end_wal": "000000090001C37C00000037",
                    "end_xlog": "1C37C/37FFEAB0",
                    "error": "failure copying files (data transfer failure)",
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": null,
                    "status": "FAILED",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20241014T181109": {
                    "backup_id": "20241014T181109",
                    "backup_label": "'START WAL LOCATION: 1C37E/CD01B658 (file 000000090001C37E000000CD)\\nCHECKPOINT LOCATION: 1C37E/D9855218\\nBACKUP METHOD: streamed\\nBACKUP FROM: standby\\nSTART TIME: 2024-10-14 18:11:20 UTC\\nLABEL: Barman backup database 20241014T181109\\n'",
                    "begin_offset": 112216,
                    "begin_time": "2024-10-14T18:11:09.573962+00:00",
                    "begin_wal": "000000090001C37E000000CD",
                    "begin_xlog": "1C37E/CD01B658",
                    "children_backup_ids": null,
                    "cluster_size": 8241567205442,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": null,
                    "data_checksums": "off",
                    "deduplicated_size": null,
                    "end_offset": 1780800,
                    "end_time": "2024-10-14T18:14:06.510859+00:00",
                    "end_wal": "000000090001C37F00000064",
                    "end_xlog": "1C37F/641B2C40",
                    "error": "failure copying files (data transfer failure)",
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": null,
                    "status": "FAILED",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20241014T190621": {
                    "backup_id": "20241014T190621",
                    "backup_label": "'START WAL LOCATION: 1C385/71034828 (file 000000090001C38500000071)\\nCHECKPOINT LOCATION: 1C385/7D858E00\\nBACKUP METHOD: streamed\\nBACKUP FROM: standby\\nSTART TIME: 2024-10-14 19:07:58 UTC\\nLABEL: Barman backup database 20241014T190621\\n'",
                    "begin_offset": 215080,
                    "begin_time": "2024-10-14T19:06:21.397363+00:00",
                    "begin_wal": "000000090001C38500000071",
                    "begin_xlog": "1C385/71034828",
                    "children_backup_ids": null,
                    "cluster_size": 8242289223747,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": null,
                    "data_checksums": "off",
                    "deduplicated_size": null,
                    "end_offset": 16776424,
                    "end_time": "2024-10-14T19:08:01.100868+00:00",
                    "end_wal": "000000090001C385000000C6",
                    "end_xlog": "1C385/C6FFFCE8",
                    "error": "failure copying files (data transfer failure)",
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": null,
                    "status": "FAILED",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20241014T190933": {
                    "backup_id": "20241014T190933",
                    "backup_label": null,
                    "begin_offset": 74624,
                    "begin_time": "2024-10-14T19:09:33.172875+00:00",
                    "begin_wal": "000000090001C38600000007",
                    "begin_xlog": "1C386/7012380",
                    "children_backup_ids": null,
                    "cluster_size": 8242366654531,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": null,
                    "data_checksums": "off",
                    "deduplicated_size": null,
                    "end_offset": null,
                    "end_time": null,
                    "end_wal": null,
                    "end_xlog": null,
                    "error": "failure copying files (Connection to the Postgres server was lost during the backup.)",
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": null,
                    "status": "FAILED",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                },
                "20241014T193546": {
                    "backup_id": "20241014T193546",
                    "backup_label": "'START WAL LOCATION: 1C38A/8062BB0 (file 000000090001C38A00000008)\\nCHECKPOINT LOCATION: 1C38A/149D1868\\nBACKUP METHOD: streamed\\nBACKUP FROM: standby\\nSTART TIME: 2024-10-14 19:36:52 UTC\\nLABEL: Barman backup database 20241014T193546\\n'",
                    "begin_offset": 404400,
                    "begin_time": "2024-10-14T19:35:46.938191+00:00",
                    "begin_wal": "000000090001C38A00000008",
                    "begin_xlog": "1C38A/8062BB0",
                    "children_backup_ids": null,
                    "cluster_size": 8242790975555,
                    "compression": null,
                    "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                    "copy_stats": {
                        "analysis_time": 10.068867,
                        "analysis_time_per_item": {
                            "pgdata": 10.068867
                        },
                        "copy_time": 63874.041668,
                        "copy_time_per_item": {
                            "pg_control": 0.366971,
                            "pgdata": 63873.670578
                        },
                        "number_of_workers": 16,
                        "serialized_copy_time": 1020791.758681,
                        "serialized_copy_time_per_item": {
                            "pg_control": 0.366971,
                            "pgdata": 1020791.39171
                        },
                        "total_time": 63884.219244
                    },
                    "data_checksums": "off",
                    "deduplicated_size": 3000278125390,
                    "end_offset": 9877592,
                    "end_time": "2024-10-15T13:21:36.991940+00:00",
                    "end_wal": "000000090001C3BC00000081",
                    "end_xlog": "1C3BC/8196B858",
                    "error": null,
                    "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                    "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                    "included_files": [
                        "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                    ],
                    "mode": "rsync-concurrent",
                    "parent_backup_id": null,
                    "pgdata": "/home/postgres/pgdata/pgroot/data",
                    "server_name": "database",
                    "size": 8245077970900,
                    "status": "WAITING_FOR_WALS",
                    "summarize_wal": null,
                    "systemid": "6487084749388934874",
                    "tablespaces": null,
                    "timeline": 9,
                    "version": 100023,
                    "xlog_segment_size": 16777216
                }
            },
            "config": {
                "active": true,
                "archiver": true,
                "archiver_batch_size": 0,
                "autogenerate_manifest": false,
                "aws_await_snapshots_timeout": 3600,
                "aws_profile": null,
                "aws_region": null,
                "azure_credential": null,
                "azure_resource_group": null,
                "azure_subscription_id": null,
                "backup_compression": null,
                "backup_compression_format": null,
                "backup_compression_level": null,
                "backup_compression_location": null,
                "backup_compression_workers": null,
                "backup_directory": "/data/backups/database/backups",
                "backup_method": "rsync",
                "backup_options": "concurrent_backup",
                "bandwidth_limit": null,
                "barman_home": "/var/lib/barman",
                "barman_lock_directory": "/var/lib/barman",
                "basebackup_retry_sleep": 30,
                "basebackup_retry_times": 0,
                "basebackups_directory": "/data/backups/database/backups/base",
                "check_timeout": 30,
                "cluster": "database",
                "compression": "gzip",
                "config_changes_queue": "/var/lib/barman/cfg_changes.queue",
                "conninfo": "host=backup-db port=5432 user=postgres password=*REDACTED*",
                "create_slot": "manual",
                "custom_compression_filter": null,
                "custom_compression_magic": null,
                "custom_decompression_filter": null,
                "description": "Platon Live System",
                "disabled": false,
                "errors_directory": "/data/backups/database/wal_errors",
                "forward_config_path": false,
                "gcp_project": null,
                "gcp_zone": null,
                "immediate_checkpoint": false,
                "incoming_wals_directory": "/data/backups/database/incoming",
                "keepalive_interval": 60,
                "last_backup_maximum_age": null,
                "last_backup_minimum_size": null,
                "last_wal_maximum_age": null,
                "local_staging_path": null,
                "lock_directory_cleanup": true,
                "max_incoming_wals_queue": null,
                "minimum_redundancy": 4,
                "msg_list": [],
                "name": "database",
                "network_compression": false,
                "parallel_jobs": 16,
                "parallel_jobs_start_batch_period": 1,
                "parallel_jobs_start_batch_size": 10,
                "path_prefix": "/usr/pgsql-10/bin",
                "post_archive_retry_script": "/data/backups/database/post_archive_wal.sh",
                "post_archive_script": null,
                "post_backup_retry_script": null,
                "post_backup_script": null,
                "post_delete_retry_script": null,
                "post_delete_script": null,
                "post_recovery_retry_script": null,
                "post_recovery_script": null,
                "post_wal_delete_retry_script": null,
                "post_wal_delete_script": null,
                "pre_archive_retry_script": null,
                "pre_archive_script": null,
                "pre_backup_retry_script": null,
                "pre_backup_script": null,
                "pre_delete_retry_script": null,
                "pre_delete_script": null,
                "pre_recovery_retry_script": null,
                "pre_recovery_script": null,
                "pre_wal_delete_retry_script": null,
                "pre_wal_delete_script": null,
                "primary_checkpoint_timeout": 0,
                "primary_conninfo": "host=live-db port=5432 user=postgres",
                "primary_ssh_command": null,
                "recovery_options": "",
                "recovery_staging_path": null,
                "retention_policy": "window 240 m",
                "retention_policy_mode": "auto",
                "reuse_backup": "link",
                "slot_name": "barman_streaming_replication",
                "snapshot_disks": null,
                "snapshot_gcp_project": null,
                "snapshot_instance": null,
                "snapshot_provider": null,
                "snapshot_zone": null,
                "ssh_command": "ssh -q -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no postgres@backup-db",
                "streaming_archiver": true,
                "streaming_archiver_batch_size": 0,
                "streaming_archiver_name": "barman_receive_wal",
                "streaming_backup_name": "barman_streaming_backup",
                "streaming_conninfo": "host=live-db port=5432 user=postgres",
                "streaming_wals_directory": "/data/backups/database/streaming_wals",
                "tablespace_bandwidth_limit": null,
                "wal_conninfo": "host=live-db port=5432 user=postgres",
                "wal_retention_policy": "simple-wal 240 m",
                "wal_streaming_conninfo": "host=live-db port=5432 user=postgres",
                "wals_directory": "/data/backups/database/wals"
            },
            "status": {
                "archive_command": "/bin/true",
                "archive_mode": "on",
                "archive_timeout": 1800,
                "archived_count": 0,
                "checkpoint_timeout": 300,
                "config_file": "/home/postgres/pgdata/pgroot/data/postgresql.conf",
                "connection_error": null,
                "current_archived_wals_per_second": 0.0,
                "current_lsn": "1C3C3/D3FFFCE0",
                "current_size": 8250016516171.0,
                "current_xlog": null,
                "data_checksums": "off",
                "data_directory": "/home/postgres/pgdata/pgroot/data",
                "failed_count": 0,
                "has_backup_privileges": true,
                "has_monitoring_privileges": true,
                "hba_file": "/home/postgres/pgdata/pgroot/data/pg_hba.conf",
                "hot_standby": "on",
                "ident_file": "/home/postgres/pgdata/pgroot/data/pg_ident.conf",
                "included_files": [
                    "/home/postgres/pgdata/pgroot/data/postgresql.base.conf"
                ],
                "is_archiving": true,
                "is_in_recovery": true,
                "is_superuser": true,
                "last_archived_time": null,
                "last_archived_wal": null,
                "last_failed_time": null,
                "last_failed_wal": null,
                "max_replication_slots": "10",
                "max_wal_senders": "10",
                "pg_receivexlog_compatible": true,
                "pg_receivexlog_installed": true,
                "pg_receivexlog_path": "/usr/pgsql-10/bin/pg_receivewal",
                "pg_receivexlog_supports_slots": true,
                "pg_receivexlog_synchronous": false,
                "pg_receivexlog_version": "12.20",
                "postgres_systemid": "6487084749388934874",
                "replication_slot": null,
                "replication_slot_support": true,
                "server_txt_version": "10.23",
                "stats_reset": "2024-10-14T19:35:04.052133+00:00",
                "streaming": true,
                "streaming_supported": true,
                "streaming_systemid": "6487084749388934874",
                "synchronous_standby_names": [
                    ""
                ],
                "timeline": 9,
                "version_supported": true,
                "wal_compression": "on",
                "wal_keep_segments": "64",
                "wal_level": "logical",
                "xlog_segment_size": 16777216,
                "xlogpos": "1C3C3/D4B8FBD0"
            },
            "system_info": {
                "kernel_ver": "Linux backup-db 5.10.37 #1 SMP Mon May 17 13:15:38 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux",
                "python_executable": "/usr/bin/python3",
                "python_ver": "Python 3.9.18",
                "release": "RedHat Linux Rocky Linux release 9.4 (Blue Onyx)",
                "rsync_ver": "rsync  version 3.2.3  protocol version 31",
                "ssh_ver": ""
            },
            "wals": {
                "last_archived_wal_per_timeline": {
                    "00000001": {
                        "compression": "gzip",
                        "name": "000000010000229F000000D2",
                        "size": 2138145,
                        "time": 1503869405.0
                    },
                    "00000004": {
                        "compression": null,
                        "name": "00000004000031E2000000DC",
                        "size": 16777216,
                        "time": 1524124127.2954156
                    },
                    "00000008": {
                        "compression": "gzip",
                        "name": "000000080001658E000000A1",
                        "size": 4951535,
                        "time": 1699535232.302205
                    },
                    "00000009": {
                        "compression": "gzip",
                        "name": "000000090001C37C0000006A",
                        "size": 5165581,
                        "time": 1728896523.9662364
                    }
                }
            }
        }
    }
}

@thoro
Copy link
Contributor Author

thoro commented Oct 15, 2024

As you see in the barman diagnose the replication_slot is reported as none.

but if I do:

[root@barman-0 /]# psql -h live-db -U postgres -c "SELECT * FROM pg_replication_slots";
          slot_name           |  plugin  | slot_type | datoid |   database   | temporary | active | active_pid | xmin | catalog_x
min |  restart_lsn   | confirmed_flush_lsn
------------------------------+----------+-----------+--------+--------------+-----------+--------+------------+------+----------
----+----------------+---------------------
 barman_streaming_replication |          | physical  |        |              | f         | t      |    1454100 |      |
    | 1C3BB/4000000  |
** REDACTED**
(4 rows)

@martinmarques
Copy link
Contributor

martinmarques commented Oct 15, 2024 •
edited
Loading

I was able to reproduce it locally. We'll do some investigation and come back

@thoro
Copy link
Contributor Author

thoro commented Oct 24, 2024 •
edited
Loading

@martinmarques Have you had any progress with that? Do you know another workaround than editing the postgres.py file?

I might not have been clear on the impact, the bug breaks the backup functionality and no backups can be started!

@martinmarques martinmarques mentioned this issue Nov 6, 2024
Closed
@martinmarques
Copy link
Contributor

@martinmarques Have you had any progress with that? Do you know another workaround than editing the postgres.py file?

I might not have been clear on the impact, the bug breaks the backup functionality and no backups can be started!

This will be fixed in the next release

andremagui added a commit that referenced this issue Nov 21, 2024
@andremagui
Add release note for BAR-409
710554d 
This commit adds a release note for the ticket BAR-409 which
fixes a bug reported in the GH issue
[#1024](#1024).

References: BAR-409

Signed-off-by: Andre <andre.marchesini@enterprisedb.com>
@martinmarques
Copy link
Contributor

The fix for this issue was released yesterday

@thoro
Copy link
Contributor Author

thoro commented Nov 22, 2024

Thanks, will try!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug triage
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@thoro @martinmarques