zxfer doesn't transfer old snapshots #29
Comments
|
It depends on your before/after state. In ZFS it is not possible to 'backfill' snapshots. You cannot transfer a snapshot that is older than the current snapshots on the destination. So on a second run, it can only work forward from where it was last time, it cannot go backwards. When the do the very first replication, zxfer transfers the oldest snapshot, and then works forward from there. You might need to check the 'grandfather' setting, since you do have some quite old snapshots. So it seems it replicates all of the snapshots of the parent dataset, but then only some for the children? I have not seen this behaviour in my production use of zxfer. This might be a bug, but more detail will need to be figured out to identify where the bug may be. |
|
Maybe the first run of zxfer died before it could transfer all the older snapshots, and somehow this prevents subsequent runs from also transferring those snapshots? I'll try starting from scratch and see if it happens again. |
|
I have a guess as to what might be happening. I have a cronjob that creates a "frequent" snapshot every 15 minutes, and then deletes all but the last 6 such snapshots. If this cronjob runs while zxfer is running, zxfer will fail to find the snapshot that gets deleted, and will crash after syncing all snapshots up to 90 minutes ago. When I run it the next time, it will only look at the root filesystem to determine which snapshots need to be synced. Since all snapshots up to 90 minutes ago are already present for the root fs, it assumes without checking that they are also present for all the child filesystems, which is not true because the previous run crashed before transferring the child filesystems. So, that's my guess as to what's happening. Assuming I'm right, there's a couple of things that could potentially mitigate this issue:
|
|
Another possibility: transfer snapshots of child filesystems first. |
|
For what it's worth, I think you could probably reproduce this bug by first running a zxfer in non-recursive mode and then running again in recursive mode. |
|
I am seeing a strange behaviour that could be coming from the same problem as in this post: Replicating an entire dataset + children (Backup0001/ABCH029/Global) to a remote machine (Pool) with : Most filesystems get replicated with all their snapshots. Source snapshots for filesystem Backup0001/Global Snapshots at the destination after running the zxfer command several time (no error): |
In the past, I've seen issues like this when a snapshot with the same name is being created on the destination. Is zfs-auto-snapshot running on the Pool/Global machine too? Try:
The GUIDs should be the same, if there are any that are different, then they were likely created on the destination, and have munged things up on you. You should not create snapsohts on the destination of the zfs replication. |
I've been testing out zxfer as a way to backup the complete state of all filesystems, including all their snapshots, to an external hard drive. However, zxfer doesn't seem to be transferring old snapshots. It seems to have transferred all snapshots on the filesystem that I specified, but on the child filesystems I think it's only transferring snapshots that were made after the first time I ran zxfer. I am running zxfer as:
As it's running, it only mentions snapshots that were created/destroyed since the previous run, and doesn't mention the earlier snapshots at all. To count the snapshots on each filesystem, I use:
Counting the snapshots on the source (rpool/fs) and destination (backuppool/pools/fs):
As you can see, all snapshots for rpool/fs are backed up, but only about 15 snapshots from its child filesystems are backed up, despite the fact that they all have mant more. For example, I'll list only the snapshots for rpool/fs/home and its destination backuppool/pools/fs/home using the following function:
The destination copy only has filesystems created at 2:17 AM on 2016-07-16 or later, which I believe is all the snapshots made after the first time I ran the zxfer command mentioned above. Am I missing something about how to get zxfer to transfer old snapshots, or is this a bug?
The text was updated successfully, but these errors were encountered: