Synchronize backups with remote servers (e.g. BorgBase)

[gbertolottiKS]

This guide aims to describe step by step what you need to do to synchronize backups created with Borg from Nextcloud AIO with an external server, so that you always have an up-to-date remote copy.

It describes in particular the flow, from start to finish, to use the BorgBase service, an open-source solution for backing up repositories based on Borg or Restic, but the same procedure can be adapted to any remote backup service: you simply need to generate an SSH key pair and have installed RClone on the Nextcloud server.

Let's get started!

BorgBase account creation

Register at https://www.borgbase.com and select a plan from those offered. However, all of them allow you to exceed the GB allowance by paying an additional fee.

SSH key pair creation

In order to perform any operation on BorgBase remotely, you need to create an SSH key pair (which can also be used for SFTP operations).

Follow this useful guide: https://docs.borgbase.com/faq/ssh/

⚠⚠⚠
Remember that a key pair must be generated for each server you want to connect, and for the user who will connect to the remote repository. As for Nextcloud backups, since they are created with root as owner, you must create a key pair for the root user
⚠⚠⚠

Creating repos on BorgBase

1. create a new repository by going to Repositories > New Repo
2. choose a name for the repo, the region, and the type (i.e. Borg)
3. under the Access tab, choose the root SSH key and set it in the Full Access field (required for SFTP)
4. under the Advanced tab, enable the SFTP flag

Manual import/export test

Import

Follow the official guide: https://docs.borgbase.com/setup/import/#import-existing-repository.

Two clarifications:

• remember to perform the operations by connecting from the terminal on the source machine as root user (in the case of Nextcloud AIO this is necessary because the backups are created by the root user)
• the command to be used must have the following format:

sftp <user-name>@<user-name>.repo.borgbase.com:repo

where <user-name> is the name of the user associated with the repo

Export

Follow the official guide: https://docs.borgbase.com/setup/import/#export-existing-repository.

RClone installation and configuration

The following guide refers to Ubuntu servers; for other OS follow the official guide: https://rclone.org/install/

1. install with the command: sudo apt install rclone.
2. check the success of the operation: rclone --version
3. start the configuration by running the command: rclone config.

Leave the default for all options, apart from the following info:

• name: set the name you want of this configuration, e.g. "borgbase"
• type = sftp
• host: host you connect to via SSH/SFTP
• user: user you connect with via SSH/SFTP
• key_file = ~/.ssh/id_ed25519 (unless another path was specified when creating the keys)

Example:

Test sync upload to BorgBase

Follow the command specified in the official guide: https://docs.borgbase.com/setup/import/#using-rclone .

In particular, the command must have the following format:

rclone sync -v <local-path-borg-backup> <config-rclone-name>:repo

where <local-path-borg-backup> is the path where the folder of the Borg backup made by Nextcloud is located; the folder is configurable from the Nextcloud AIO interface. <config-rclone-name> is the name assigned when configuring rclone

Example:

rclone sync -v /mnt/backup/borg/ borgbase:repo

Scheduled job creation

The following steps have been adapted from the following documentation: https://github.com/nextcloud/all-in-one#sync-the-backup-regularly-to-another-drive

Create scripts

• create a shell file called e.g. backup-script.sh under e.g. the /root folder
• write in it (using e.g. the text editor Nano) the following script template

#!/bin/bash

# Change all the variables below to suit your needs:
SOURCE_DIRECTORY="/mnt/backup/borg"
RCLONE_CONFIG="borgbase"

TARGET_DIRECTORY="repo" # this is the BorgBase directory name: modify it only if you want to specify a different one (discouraged)

########################################
# DO NOT modify any of the following! #
########################################

if [ "$EUID" -ne 0 ]; then 
    echo "run as root"
    output 1
fi

if ! [ -d "$SOURCE_DIRECTORY" ]; then
    echo "The root directory does not exist."
    exit 1
fi

if [ -z "$(ls -A "$SOURCE_DIRECTORY/")" ]; then
    echo "The source directory is empty, which is not allowed."
    exit 1
fi

if [ -f "$SOURCE_DIRECTORY/lock.roster" ]; then
    echo "Unable to execute the script as the backup archive is currently modified. Please try again later."
    exit 1
fi

if [ -f "$SOURCE_DIRECTORY/aio-lockfile" ]; then
    echo "Cannot continue because aio-lockfile already exists."
    exit 1
fi

touch "$SOURCE_DIRECTORY/aio-lockfile"


if ! rclone sync --exclude aio-lockfile "$SOURCE_DIRECTORY/" "$RCLONE_CONFIG:$TARGET_DIRECTORY"; then
    if docker ps --format "{{.Names}}" | grep "^nextcloud-aio-nextcloud$"; then
        docker exec -en nextcloud-aio-nextcloud bash /notify.sh "Rclone failed." "Failed to synchronise the backup repository with the target directory."
    else
        echo "Failed to synchronise the backup repository with the target directory."
    fi
    rm "$SOURCE_DIRECTORY/aio-lockfile"
    exit 1
fi

rm "$SOURCE_DIRECTORY/aio-lockfile"

if docker ps --format "{{.Names}}" | grep "^nextcloud-aio-nextcloud$"; then
    docker exec -en nextcloud-aio-nextcloud bash /notify.sh "Rclone backup successful!" "Synchronised the backup repository successfully."
else
    echo "Synchronised the backup repository successfully."
fi

where SOURCE_DIRECTORY is the path where Nextcloud backups are saved (this is configurable from the AIO interface), and RCLONE_CONFIG is the name of the previously defined RClone configuration

• Give the correct permissions to the file with the commands:

sudo chown root:root /root/backup-script.sh

sudo chmod 700 /root/backup-script.sh

Set crontab

• Open the cronjob: sudo crontab -u root -e
• set the desired schedule; for example, if you want to run it every day at 11pm, run:

0 23 * * * /root/backup-script.sh

That's all!

[szaimen]

Wow @gbertolottiKS , thanks a lot, this is great! :)

I will add it to the readme with #2249

[LeLuc4]

Thanks for the instructions.
Looks like your shebang was corrupted by github. Should be #!/bin/bash.

[gbertolottiKS]

Uuuh, yes, thanks for noticing. I'll edit it now

[gbertolottiKS]

We have created a new guide on how to use Borg and Borgmatic to make remote backups, in order to avoid making backups useless in case of a malware attack: #4391

[SamuelBorn]

Should I switch to the new way or is it fine if I run this old setup for a while?

[gbertolottiKS]

It's up to you: it's a step-up in terms of security, because it's way more robust in case your backups are ciphered/corrupted.
The new solutions allow you to not overwrite your previous versions every run, but to store them in an archive in append-only mode.

Bye