Snapshot send encrypted to a backup destination for the first time fails

[hansoostendorp]

Current behaviour:
Run ZnapZend on an encrypted source dataset, using the sendRaw feature to keep the data encrypted, on the destination dataset for the first time.
Result when encrypted destination dataset doesn't exist yet: send error, a non encrypted empty destination dataset is created.
Result when empty encrypted destination dataset exist: send error, destination dataset remains empty.

Expected behaviour:
The snapshot that is created by ZnapZend on the encrypted source dataset, is send to the destination dataset without errors and is still encrypted.

Work around:
After manually sending the first snapshot encrypted to the destination dataset, following snapshots created by ZnapZend are send encrypted successfully.

OS: Ubuntu 20.04 LTS
ZFS: 0.8.3
ZnapZend: 0.20.0

How te reproduce:

Create encrypted datapool/test source dataset
# sudo zfs create -o encryption=on -o keyformat=passphrase datapool/test

Create a backup plan for datapool/test to backuppool/test
# sudo znapzendzetup create SRC '1h=>1min' datapool/test DST:a '1h=>1min' backuppool/test

Run ZnapZend, using the sendRaw feature to keep the backup encrypted, and let ZnapZend automatically create the backup destination
# sudo znapzend --debug --runonce=datapool/test --features=sendRaw --autoCreation

It will automatically create the destination dataset using this zfs command
# zfs create -p backuppool/test

It will then run this send command resulting in an error

# zfs send -w datapool/test@2020-07-22-121805|zfs recv -F backuppool/test
cannot receive new filesystem stream: zfs receive -F cannot be used to destroy an encrypted filesystem or overwrite an unencrypted one with an encrypted one

The destination dataset that is created by ZnapZend is not encrypted and empty. When I run ZnapZend without autoCreation, it complains the destination dataset doesn't exist. When I create an encrypted destination dataset by hand, and run ZnapZend without the autoCreation option, the same error occurs and the destination dataset is still empty.

I think this is related to this ZendZnap issue: #302
Which is related to this ZFS issue: zfs receive -F cannot be used to destroy an encrypted filesystem #6793

What I take from these posts, is that the -F flag is only required if you have existing data to overwrite. When running the backup for the first time, there is no existing data, so the -F flag should be omitted.

To make the first ZnapZend run work, I manually create the first snapshot on the source dataset and manually run the send command to the destination dataset without -F:

# sudo zfs snapshot datapool/test@manualbackup
# sudo zfs send -w datapool/test@manualbackup |sudo zfs recv backuppool/test

This will create the destination dataset automatically that contains the first snapshot. Note that it is not mounted (it has no keys). The destination dataset is encrypted and can be mounted using the same keys as the source dataset.

After the first run I can use ZnapZend using the sendRaw feature without any errors and new snapshots will be added and are encrypted
# sudo znapzend --debug --runonce=datapool/test --features=sendRaw

Is it possible to make ZnapZend aware when it is run for the first time on the destination dataset and omit the -F flag so it can create the first snapshot sync without errors?

Perhaps it's enough to detect this when the destination dataset doesn't exist yet. When it does exist, ZFS can throw an error when it's another filesystem. This should be enough information for the user.

[oetiker]

so that would indicate that the create command would have to be a bit more clever :) care to provide a PR ?

[hansoostendorp]

Okay, I will give it a go ;-)