Skip to content

Commit

Permalink
Initial docs conversion.
Browse files Browse the repository at this point in the history
closes pulp#1212
  • Loading branch information
ipanova committed Feb 17, 2024
1 parent 73f1a1e commit 0908923
Show file tree
Hide file tree
Showing 50 changed files with 3,270 additions and 0 deletions.
38 changes: 38 additions & 0 deletions staging_docs/admin/guides/backup_and_restore/00-overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Overview of Backup/Restore Operations


In addition to provisioning Pulp components, the Operator can also be used to backup and restore them.

!!! note
Before starting a backup, make sure that the namespace has enough storage quota available.

## Backup
The backup procedure creates a *manager* `Pod` which will be used to execute all the backup tasks:

* run a `pg_dump` (database dump) on Pulp's database
* do a copy of the Pulp CR instance defined in `deployment_name`
* do a copy of the `Secrets`
* do a copy of `/var/lib/pulp` directory
* delete the *manager* `Pod` to not consume resources

These data will be stored in a new PVC defined in PulpBackup CR (`backup_pvc` or `backup_storage_class`).


!!! note
The current version of the Operator does **not** execute backups of **external** PostgreSQL instances yet.


!!! notes
Considering that files stored in Object Storage (like `AWS S3` and/or `Azure Blob`) are not kept in `/var/lib/pulp` directory, they will **not** be copied. If you still need to do a backup of the artifacts stored in Object Storage, please, contact your cloud provider to check the procedure to do so.


## Restore
The restore procedure also creates a *manager* `Pod` to execute all the tasks:

* restore the `Secrets`
* restore Pulp CR instance
* restore Pulp database
* restore `/var/lib/pulp` directory
* delete the *manager* `Pod` to not consume resources

All data restored comes from the PVC defined in PulpRestore CR (`backup_pvc`).
93 changes: 93 additions & 0 deletions staging_docs/admin/guides/backup_and_restore/01-config_and_run.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# Configure and Run Backup/Restore


All the configurations needed to run the backup or restore procedures are made through `PulpBackup` or `PulpRestore` CRs.

To get the list of fields available in each CR, check [`PulpBackupSpec`](/pulp_operator/backup/#pulpbackupspec) and [`PulpRestoreSpec`](/pulp_operator/restore/#pulprestorespec).

## Backup

To configure the backup controller, create a manifest file with the definition of [PulpBackup CR](/pulp_operator/backup/#pulpbackupspec).
For example:
```
---
apiVersion: repo-manager.pulpproject.org/v1beta2
kind: PulpBackup
metadata:
name: pulpbackup-sample
spec:
deployment_name: pulp
deployment_type: pulp
backup_storage_class: standard
admin_password_secret: example-pulp-admin-password
postgres_configuration_secret: pulp-postgres-configuration
```

In the above sample we defined:

* the name of `Pulp` instance (`deployment_name`), which can be gathered with:
```
$ kubectl get pulp
NAME AGE
pulp 3m30s
```

* the type of deployment (`deployment_type`), in this case `pulp` (but could also be `galaxy` depending on the installation).
* the name of `StorageClass` used to provision the `PVC` to store the backup data (`backup_storage_class`).
* the name of the `Secret` with Pulp admin password (`admin_password_secret`), which can be get by:
```
$ kubectl get pulp pulp -ojsonpath='{.spec.admin_password_secret}{"\n"}'
pulp-admin-password
```

* the name of the `Secret` with PostgreSQL credentials and connection information (`postgres_configuration_secret`).

After finishing to configure the file, apply the configuration and the Operator will start the backup:
```
kubectl apply -f <backup_cr_file>.yaml
```


## Restore


To configure the restore controller, create a manifest file with the definition of [PulpRestore CR](/pulp_operator/restore/#pulprestorespec).
For example:
```
---
apiVersion: repo-manager.pulpproject.org/v1beta2
kind: PulpRestore
metadata:
name: pulprestore-sample
spec:
backup_name: pulpbackup-sample
deployment_name: pulp
```

In the above sample we defined:

* the name of `PulpBackup` instance (`backup_name`)
* the name of `Pulp` instance (`deployment_name`). This should be the same defined in the `PulpBackup` CR.

After finishing to configure the file, apply the configuration and the Operator will start the restore:
```
kubectl apply -f <restore_cr_file>.yaml
```

By default, the restore procedure will reprovision the environment with a single replica of each component. This is to make it easier to review the restore status and the environment health.
It is also possible to restore with the same number of replicas running when the backup was made. To do so, just set the `keep_replicas` field to true, for example:
```
---
apiVersion: repo-manager.pulpproject.org/v1beta2
kind: PulpRestore
metadata:
name: pulprestore-sample
spec:
backup_name: pulpbackup-sample
deployment_name: pulp
keep_replicas: true
```


After finishing to restore the environment, the operator will create a `ConfigMap` called *`restore-lock`*. It is used to prevent a new controller reconciliation loop to run and override any data changed/created with the "old" data from backup.
To allow the restore controller to run again, delete the *restore-lock* `ConfigMap`.
102 changes: 102 additions & 0 deletions staging_docs/admin/guides/backup_and_restore/02-cronjob.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
# Schedule Backups

The current version of Pulp Operator does not provide a way to automate the periodic execution of the backups. The following steps can be used as **an example** of how to create a [k8s Cronjob](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/) to schedule the backup execution.

* create a configmap with the `PulpBackup CR` definition (check the [backup section](/pulp_operator/backup_and_restore/config_running/#backup) for more information on `PulpBackup CR` fields configuration):
```yaml
$ kubectl apply -f- <<EOF
apiVersion: v1
data:
pulp_backup.yaml: |
apiVersion: repo-manager.pulpproject.org/v1beta2
kind: PulpBackup
metadata:
name: pulpbackup
spec:
deployment_name: pulp
backup_storage_class: standard
kind: ConfigMap
metadata:
name: pulpbackup-cr
EOF
```

* for this example, we will create a new `ServiceAccount` that will be used by `Cronjob` pods. It is not a required step, you can skip it if your environment already has a `ServiceAccount` with the permissions to modify `PulpBackup` resources:
```yaml
$ kubectl apply -f-<<EOF
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: pulpbackup
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: pulpbackup
rules:
- apiGroups: ["repo-manager.pulpproject.org"]
resources: ["pulpbackups"]
verbs: ["get", "watch", "list","create","patch","update","delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: pulpbackup
subjects:
- kind: ServiceAccount
name: pulpbackup
roleRef:
kind: Role
name: pulpbackup
apiGroup: rbac.authorization.k8s.io
EOF
```

* create the `k8s Cronjob` to run the backups:
```yaml
$ kubectl apply -f-<<EOF
apiVersion: batch/v1
kind: CronJob
metadata:
name: pulpbackup
spec:
schedule: "00 2 * * *"
successfulJobsHistoryLimit: 1
failedJobsHistoryLimit: 1
jobTemplate:
spec:
template:
spec:
containers:
- name: bkp
image: bitnami/kubectl:latest
imagePullPolicy: IfNotPresent
volumeMounts:
- name: pulpbackup-cr
mountPath: /tmp/pulp_backup.yaml
subPath: pulp_backup.yaml
command:
- /bin/sh
- -c
args:
- "kubectl apply -f /tmp/pulp_backup.yaml && kubectl wait --for condition=BackupComplete --timeout=600s -f /tmp/pulp_backup.yaml ; kubectl delete -f /tmp/pulp_backup.yaml"
restartPolicy: Never
serviceAccountName: pulpbackup
volumes:
- name: pulpbackup-cr
configMap:
name: pulpbackup-cr
EOF
```

In this example, the job:

* will be triggered every day at *2:00 AM* (`schedule: 00 2 * * *`)
* will keep `1` successful and/or `1` failed job (`successfulJobsHistoryLimit: 1, failedJobsHistoryLimit: 1`)
* will be considered failed if the backup does not finish in `10minutes` (`--timeout=600s`)
* will run with the previously created *pulpbackup* `ServiceAccount` (`serviceAccountName: pulpbackup`)


!!! note
Don't forget to rotate the backup files from time to time to avoid filling up the storage.
52 changes: 52 additions & 0 deletions staging_docs/admin/guides/configurations/cache.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Configure Pulp Cache

Pulp can cache the metadata instead of doing "slower" requests into the database (PostgreSQL).
To do so, it uses [Redis as the cache backend](https://docs.pulpproject.org/pulpcore/configuration/settings.html#redis-settings) technology.

Pulp operator provides a single node Redis server for Pulp to use, but it is also possible to configure the operator to use an external Redis installation.

## Configure Pulp operator to deploy a Redis instance

[Pulp CR page](https://docs.pulpproject.org/pulp_operator/pulp/#cache) has all the parameters that can be set to inform Pulp operator how it should deploy the Redis container.

If no `cache` parameter is defined, Pulp operator will deploy Redis with the following configuration:

* a `Deployment` will be provisioned to handle Redis pod
* a single Redis replica will be available (it is **not** possible to form a cluster with this container)
* it will deploy a `docker.io/library/redis:latest` image

A `Service` will be created with the Redis pod as endpoint.

Here is an example of how to configure Pulp operator to deploy the Redis cache:
```
...
spec:
cache:
enabled: true
...
```

## Configure Pulp operator to use an external Redis installation

It is also possible to configure Pulp operator to point to a running Redis cluster.
To do so, create a new `Secret` with the parameters to connect to the running Redis cluster:
```
$ kubectl -npulp create secret generic external-redis \
--from-literal=REDIS_HOST=my-redis-host.example.com \
--from-literal=REDIS_PORT=6379 \
--from-literal=REDIS_PASSWORD="" \
--from-literal=REDIS_DB=""
```

Make sure to define all the keys (`REDIS_HOST`, `REDIS_PORT`, `REDIS_PASSWORD`, `REDIS_DB`) even if Redis cluster has
no authentication, like in the above example.

Now, configure Pulp operator CR to use the Secret:
```
...
spec:
cache:
enabled: true
external_cache_secret: external-redis
...
```
24 changes: 24 additions & 0 deletions staging_docs/admin/guides/configurations/content_checksums.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Configure Pulp Allowed Content Checksums

During repositories synchronization, Pulp checks the downloaded files against a list
of checksums algorithms. Together with a valid (and trusted) release file signature this will guarantee the integrity of the synchronized repository.

The list of checksums algorithms is defined using the `ALLOWED_CONTENT_CHECKSUMS` setting.
For more information on how `Pulp` uses the checksums check: [https://docs.pulpproject.org/pulp_deb/workflows/checksums.html](https://docs.pulpproject.org/pulp_deb/workflows/checksums.html)


To set the `ALLOWED_CONTENT_CHECKSUMS` in Pulp Operator, update Pulp CR with:
```yaml
spec:
allowed_content_checksums:
- sha256
```
!!! note
`sha256` is a mandatory checksum.
The possible checksums are: `md5`, `sha1`,`sha256`,`sha512`.

After modifying the `allowed_content_checksums` field in Pulp CR, the operator will create a kubernetes job to run the `pulpcore-manager handle-artifact-checksums` command to ensure database consistency.

!!! note
Missing checksums will need to be recalculated for all your artifacts which can take some time.
12 changes: 12 additions & 0 deletions staging_docs/admin/guides/configurations/customCA.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Certificate injection in Pulp containers

In OpenShift environments, it is possible to [mount additional trust bundles](https://docs.openshift.com/container-platform/4.10/networking/configuring-a-custom-pki.html#certificate-injection-using-operators_configuring-a-custom-pki) into Pulp containers.

Pulp operator handles part of the process.

When `trusted_ca: true` Pulp operator will automatically create and mount a `ConfigMap` with the custom CA into Pulp pods, but before doing so users need to first follow the steps from [Enabling the cluster-wide proxy](https://docs.openshift.com/container-platform/4.10/networking/configuring-a-custom-pki.html#nw-proxy-configure-object_configuring-a-custom-pki) to "register" the custom CA certificate into the cluster.


!!! info

It is recommended to execute the previous steps in a maintenance window because, since this is cluster-wide modification, the cluster can get unavailable if executed wrong (some cluster operators pods will be restarted).
Loading

0 comments on commit 0908923

Please sign in to comment.