mox backup creates invisible files.

[Mihara]

So I have a mox home directory and want to create a backup to pack it up and send it to a backup server. Naively, I run ./mox backup <location> as root, because all the other maintenance operations run fine as root...

...and end up with a directory that ostensibly exists (because when I run mox backup again, it says it does) but is not listed anywhere nor otherwise accessible.

How do I even delete it now?...

[mjl-]

The "mox backup" command asks the running mox server process to make the backup (over a ctl unix domain socket). So the destination path is interpreted by the running mox process. There's a good chance the destination path is part of some private mount namespace.

I ran mox backup /tmp/moxbackup just now on a mox started from systemd, and am seeing similar symptoms. The mox systemd service has its own /tmp. The files backed up to /tmp/moxbackup were accessible on the "host" /tmp, in /tmp/systemd-private-[id]-mox.service-[id]/tmp/moxbackup.

Are you also not seeing the file when backing up to a path like data/backup?
There may also be a way to explicitly specify binds/mounts in the systemd service file.

[RobSlgm]

BTW: The /tmp directory is made private with the PrivateTmp=yes setting in the default mox.service file.

I tried to add to the mox.service another directory in the ReadWritePaths=.. ... /var/opt/somebackup. This works but produces for my taste too many warnings.

At the end of the day the practical solution is to create the backup in the $MOXROOTDIR/data/tmp/backup

A helper script looks like

#!/bin/bash
MOXDIR=>>add your base directory for mox<<
MOXBACKUPDIR=${MOXDIR}/data/tmp/backup
rm -rf ${MOXBACKUPDIR}
./mox backup ${MOXBACKUPDIR}/data
./mox verifydata ${MOXBACKUPDIR}/data
cp -rp ${MOXDIR}/config/ ${MOXBACKUPDIR}/config

and then do the real backup, i.e. restic.

[mjl-]

@RobSlgm

This works but produces for my taste too many warnings.

Are you getting many lines of output? Or are you running this under crontab and getting a message for each backup?

[RobSlgm]

Did a similar setup just now (added the path /var/backup-mox, which is mox user writeable, to the service definition) and did a backup ./mox backup /var/backup-mox/xx as user mox, which gives this output:

warning: creating hardlink to message failed, will be doing regular file copies and not warn again: link data/accounts/freddy/msg/a/1 /var/backup-mox/xx/accounts/freddy/msg/a/1: invalid cross-device link; srcpath=data/accounts/freddy/msg/a/1; dstpath=/var/backup-mox/xx/accounts/freddy/msg/a/1

this time just this one line (it's a change between 0.0.10 vs 0.0.11, I believe).

Suggestion exit code
The issue is that I want to check in the script if the backup was successful. The exit code is 0, but in my knowledge not really documented. The documentation states A clean successful backup does not print any output by default which would suggest that the backup above failed.

Suggestion
If you run the backup again, a lot of errors appear. The backup requires always an empty and new directory. This is kind of surprising, Therefore in the script above the rm -rf. Could be added to the documentation.
Using restic with the option --ignore-inode still allows for a kind of incremental backup.

[mjl-]

thanks for testing & feedback. i tried clarifying the help output for the backup command.

[Mihara]

The above script solves my practical problem (that is, getting backups to happen) -- thanks! For the moment, that's sufficient for me, though I suspect either the documentation should clarify what's happening (systemd shenanigans) and that not any random location for the backup directory will work. Or alternatively, every random location should work, and the current behavior is a bug.

In fact, maybe don't ask for a location at all? Since the message files in the backup are actually hardlinks, setting up a designated "snapshot" location somewhere in the home directory and creating a tar-able copy of the mail storage there should suit most uses.

Also, one question remains:

I have, apparently, created an invisible directory somewhere in /var while experimenting. How do I clean it up? Does it still exist after a reboot? It's hard to verify without creating it again, and the systemd documentation is fairly obtuse.

[mjl-]

I have, apparently, created an invisible directory somewhere in /var while experimenting. How do I clean it up? Does it still exist after a reboot? It's hard to verify without creating it again, and the systemd documentation is fairly obtuse.

I found mine in /tmp. /var/tmp could also be an option. It was in a directory of the form systemd-private--mox.service-. If you do a "find" in /tmp and /var/tmp for the name of the backup destination directory, it should show up.

Using a fixed backup location would indeed prevent some of these problems. But custom locations should probably still be possible. Especially if the disk mox has its data on is getting full, it would be useful to backup to another partition.

I'm also pondering if it makes sense to make mox initiate periodic backups, and calling a script to copy it off-site. I currently have a cronjob, but I'm not a fan. If builtin, we can deliver errors to the postmaster mailbox, and add prometheus metrics about success/failure.

[Mihara]

I found mine in /tmp. /var/tmp could also be an option. It was in a directory of the form systemd-private--mox.service-. If you do a "find" in /tmp and /var/tmp for the name of the backup destination directory, it should show up.

Only, I used /var/spool at some point, so it's not in either of those two...

But custom locations should probably still be possible.

There needs to be some legit way to break out of the sandbox then.

I'm also pondering if it makes sense to make mox initiate periodic backups, and calling a script to copy it off-site.

Mox runs as mox, which is not a user permitted to send anything off-site, typically, so you have to somehow invoke something as another user anyway...