[DPE-6964] Storage pools #941
Conversation
Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## 16/edge #941 +/- ##
===========================================
+ Coverage 73.74% 73.79% +0.04%
===========================================
Files 12 12
Lines 3554 3560 +6
Branches 509 511 +2
===========================================
+ Hits 2621 2627 +6
Misses 738 738
Partials 195 195 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
…ools Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
| if temp_location is not None: | ||
| cursor.execute("SELECT TRUE FROM pg_tablespace WHERE spcname='temp';") | ||
| if cursor.fetchone() is None: | ||
| cursor.execute(f"CREATE TABLESPACE temp LOCATION '{temp_location}';") | ||
| cursor.execute("GRANT CREATE ON TABLESPACE temp TO public;") |
There was a problem hiding this comment.
Configure PostgreSQL to use the temp storage for temporary tablespaces.
| container.exec([ | ||
| "chown", | ||
| f"{WORKLOAD_OS_USER}:{WORKLOAD_OS_GROUP}", | ||
| "/var/lib/postgresql/archive", | ||
| ]).wait() | ||
| container.exec([ | ||
| "chown", | ||
| f"{WORKLOAD_OS_USER}:{WORKLOAD_OS_GROUP}", | ||
| self._storage_path, | ||
| ]).wait() | ||
| container.exec([ | ||
| "chown", | ||
| f"{WORKLOAD_OS_USER}:{WORKLOAD_OS_GROUP}", | ||
| "/var/lib/postgresql/logs", | ||
| ]).wait() | ||
| container.exec([ | ||
| "chown", | ||
| f"{WORKLOAD_OS_USER}:{WORKLOAD_OS_GROUP}", | ||
| "/var/lib/postgresql/temp", | ||
| ]).wait() |
There was a problem hiding this comment.
Set ownership of storage volumes folders because on clouds like AWS and GCP, the ownership might be different.
There was a problem hiding this comment.
Oh testing this on Clouds will surpise us... diffrerent storages on different AZ/NAS/Performance.... we will have fun :-)
Can we add AWS test here? can be separate backlog task.
| - auth-local: trust | ||
| - encoding: UTF8 | ||
| - data-checksums | ||
| - waldir: /var/lib/postgresql/logs |
There was a problem hiding this comment.
Mapping the WAL directory to the logs storage in the primary.
| basebackup: | ||
| - waldir: /var/lib/postgresql/logs |
There was a problem hiding this comment.
Mapping the WAL directory to the logs storage in the replicas.
| ssl_cert_file: {{ storage_path }}/cert.pem | ||
| ssl_key_file: {{ storage_path }}/key.pem | ||
| {%- endif %} | ||
| temp_tablespaces: temp |
There was a problem hiding this comment.
Configure PostgreSQL to use the temp storage for temporary tablespaces.
| logger.info("Checking charm storages") | ||
| expected_storages = ["archive", "data", "logs", "temp"] | ||
| storages = await ops_test.model.list_storage() | ||
| assert len(storages) == 4, f"Expected 4 storages, got: {len(storages)}" | ||
| for index, storage in enumerate(storages): | ||
| assert ( | ||
| storage["attachments"]["unit-postgresql-k8s-0"].__dict__["storage_tag"] | ||
| == f"storage-{expected_storages[index]}-{index}" | ||
| ), f"Storage {expected_storages[index]} not found" |
There was a problem hiding this comment.
Testing the presence of multiple storages.
| # TODO: rollback to ops_test.model.deploy syntax when we release to edge. | ||
| await ops_test.juju( |
There was a problem hiding this comment.
This will be rolled back as soon as a revision is published to the 16/edge channel.
| base=CHARM_BASE, | ||
| ), | ||
| # TODO: rollback to ops_test.model.deploy syntax when we release to stable. | ||
| await ops_test.juju( |
There was a problem hiding this comment.
This will be rolled back as soon as a revision is published to the 16/beta channel.
Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
| container.exec([ | ||
| "chown", | ||
| f"{WORKLOAD_OS_USER}:{WORKLOAD_OS_GROUP}", | ||
| "/var/lib/postgresql/archive", | ||
| ]).wait() | ||
| container.exec([ | ||
| "chown", | ||
| f"{WORKLOAD_OS_USER}:{WORKLOAD_OS_GROUP}", | ||
| self._storage_path, | ||
| ]).wait() | ||
| container.exec([ | ||
| "chown", | ||
| f"{WORKLOAD_OS_USER}:{WORKLOAD_OS_GROUP}", | ||
| "/var/lib/postgresql/logs", | ||
| ]).wait() | ||
| container.exec([ | ||
| "chown", | ||
| f"{WORKLOAD_OS_USER}:{WORKLOAD_OS_GROUP}", | ||
| "/var/lib/postgresql/temp", | ||
| ]).wait() |
There was a problem hiding this comment.
Oh testing this on Clouds will surpise us... diffrerent storages on different AZ/NAS/Performance.... we will have fun :-)
Can we add AWS test here? can be separate backlog task.
Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
…ools Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
3 participants
Issue
The new PG 16 charm needs 4 different storage areas to separate the different kinds of data produced by the database system. More details are explained at DPE-2603.
Solution
Modify the charm to have 4 different storages instead of 1 and adapt the charm code to configure PostgreSQL to use 3 of them (data, logs and temp) for the needed purposes and keep the last one (archive) for local backups.
Notes:
Checklist