Skip to content

Commit

Permalink
F OpenNebula/one#6029: Prune, Incremental restore and retention
Browse files Browse the repository at this point in the history
rsmontero committed Feb 15, 2023

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
1 parent c7b21fa commit 3ba6041
Showing 2 changed files with 9 additions and 35 deletions.
6 changes: 4 additions & 2 deletions source/management_and_operations/backups/operations.rst
Original file line number Diff line number Diff line change
@@ -12,7 +12,7 @@ Backup Types
OpenNebula supports two backup types:

- **Full**, each backup contains a full copy of the VM disks. Libvirt version >= 5.5 is required.
- **Incremental**, each backup contains only the changes since the last backup. Libvirt version >= 7.7 is required.
- **Incremental**, each backup contains only the changes since the last backup. Incremental backups track changes by creating checkpoints (disk block dirty-bitmaps) using QEMU/Libvirt. Libvirt version >= 7.7 is required.

The Backup Process
--------------------------------------------------------------------------------
@@ -113,7 +113,7 @@ Reference: Backup Configuration Attributes
+---------------------------+--------------------------------------------------------------------------------------------------------------+
| ``FS_FREEZE`` | Operation to freeze guest FS: ``NONE`` do nothing, ``AGENT`` use guest agent, ``SUSPEND`` suspend the domain |
+---------------------------+--------------------------------------------------------------------------------------------------------------+
| ``KEEP_LAST`` | Only keep the last N backups for the VM |
| ``KEEP_LAST`` | Only keep the last N backups (full backups or increments) for the VM |
+---------------------------+--------------------------------------------------------------------------------------------------------------+
| ``MODE`` | Backup type ``FULL`` or ``INCREMENT`` |
+---------------------------+--------------------------------------------------------------------------------------------------------------+
@@ -259,6 +259,8 @@ When you restore the backup you may choose to:

- Not keep the NIC addressing (i.e. IPs, or MAC)
- Not keep any NIC definition
- In the case of incremental backups you can choose which increment to restore (or last by default)
- Finally, you can pick a base name for the VM Templates and disk Images that will be created

After you restore the VM, we recommend to review the restored template to fine-tune any additional parameter. The following example shows the recovering procedure:

38 changes: 5 additions & 33 deletions source/management_and_operations/backups/restic.rst
Original file line number Diff line number Diff line change
@@ -8,11 +8,11 @@ Backup Datastore: Restic (EE)

If you are using the enterprise edition (EE) of OpenNebula, the right version of restic has been already downloaded and installed in your system as a dependency. In this guide we will use the following terminology (introduce by restic):

- *Repository*: This is the storage volume where the disk images backups will be stored. Restic creates an specific interval structure to store the backups efficiently. The restic driver access to the repository through the sftp. protocol.
- *Repository*: This is the storage volume where the disk images backups will be stored. Restic creates an specific interval structure to store the backups efficiently. The restic driver access to the repository through the sftp. protocol. OpenNebula will create a separate restic repository for each VM or backup job.

- *Snapshot*: It represents a backup and it is referenced by an unique hash (e.g. ``eda52f34``). Each snapshot stores a VM backup and includes all of its disks and the metadata description of the VM at the time you make the backup.

- *Backup Server*: A host that will store the VM backups and the restic repository.
- *Backup Server*: A host that will store the VM backups and the restic repositories.

Step 0. [Backup Server] Setup the backup server
================================================================================
@@ -46,7 +46,6 @@ The following example showcases this setup using a dedicated 1.5T volume for bac
$ ls -ld /var/lib/one/datastores/
drwxrwxr-x 2 oneadmin oneadmin 4096 Sep 3 12:04 /var/lib/one/datastores/


Step 1. [Front-end] Create a Restic Datastore
================================================================================

@@ -71,34 +70,10 @@ Now that we have the backup server prepared, let's create an OpenNebula backup d
$ onedatastore create ds_restic.txt
ID: 100

Write down the datastore ID (100) in our case, we'll need it in the next step. You can also create the DS through Sunstone like any other datastore:
You can also create the DS through Sunstone like any other datastore:

|restic_create|

Step 2. [Front-end] Setup a Restic repository
================================================================================

Now it is time to bootstrap the restic repo. For convenience we'll set a couple of environment variables, **be sure to use the IP of the server and the datastore ID of the backup datastore**. In our example we will use 192.168.1.8 and 100, respectively:

.. prompt:: bash $ auto

$ export RESTIC_REPOSITORY="sftp:oneadmin@192.168.1.8:/var/lib/one/datastores/100"
$ export RESTIC_PASSWORD="opennebula"
$ alias restic="/var/lib/one/remotes/datastore/restic/restic"
$ restic init
created restic repository d5b1499cbb at sftp:oneadmin@192.168.1.8:/var/lib/one/datastores/100

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.

In the **backup server**, you should be able to look at the structure that restic has created for the repo:

.. prompt:: bash $ auto

$ ls /var/lib/one/datastores/100
config data index keys locks snapshots

After some time, the datastore should be monitored:

.. prompt:: bash $ auto
@@ -118,15 +93,12 @@ Repository Maintenance and Troubleshooting
Repository Pruning
--------------------------------------------------------------------------------

Data not referenced by any snapshot needs to be deleted by running the ``prune`` command in the repository. This operation depending on the data to process can be very a demanding and it even locks the repository.

OpenNebula will not automatically trigger this operation, you **need to plan** your pruning so it does not interfere with any scheduled backup.
Data not referenced by any snapshot needs to be deleted by running the ``prune`` command in the repository. This operation is executed by OpenNebula whenever an image backup is deleted, either because an explicit removal or to conform the retention policy set.

Repository is locked
--------------------------------------------------------------------------------

During the operation of the VM backups it may happen that the repository is left in a locked state. You should see an error similar to:

During the operation of the VM backups you could rarely find that the repository is left in a locked state. You should see an error similar to:

.. prompt:: bash $ auto

0 comments on commit 3ba6041

Please sign in to comment.