What happened to the backup "series" feature?

[sophie-h]

There was an extended discussion on a feature that would allow the use of something more unambiguous than prefixes to identify a series of backups on which commands like prune could be applied. It was dubbed "series" at the end of #948. It would seem much more reliable to me to have this feature for handling different series' of backups in apps like Pika and Vorta.

Somehow the issue got closed. Is the feature still on the table for 2.0?

[ThomasWaldmann]

Could be that borg2 will need something like that, but there is a lot of other stuff to do first.

[ThomasWaldmann]

#948 was closed because it was solved (see issue title).

The backup series discussion there is interesting, but something different.

local files cache vs. "previous archive"

While working on the big repository change in #8332 I noticed that we maybe could replace the files cache by reading the "previous" archive from the same series (filenames, size, time/mtime is already archived and we could add archiving the inode number so that we have that also).

When doing a backup borg2 currently does the following with the cache:

• does an exclusive lock on the cache
• reads the files cache from disk, IF it is accessed
• builds the chunks cache in memory by querying all existing chunk ids from the repo
• (does the backup, updates the files cache in memory)
• persists the files cache to disk
• releases the cache lock

That exclusive cache lock is a bit annoying (but currently needed due to that long read-modify-write operation) as the repository lock is now a shared lock for most operations.

When using the "previous archive", we wouldn't need the lock and there would not be a separate "updating the files cache" operation, because creating the new archive would be the equivalent to that.

names, uniqueness and hashes

Another slight annoyance is that borg has to check for archive name collisions and that just got a bit more expensive after I moved the archives list from a data structure within the manifest to separate archives/* store objects in #8332.

So I thought about just using the hash (object ID) as the archive identifier and either not having a name or having it more like a comment in the metadata. That would remove the need for the uniqueness check, but change the UX / cli interface quite a bit.

[sophie-h]

So I thought about just using the hash (object ID) as the archive identifier and either not having a name or having it more like a comment in the metadata.

Yes, that's what I imagine for borg2 combined with archives being part of a "series."

[ThomasWaldmann]

More precisely:

• having the hash as the unique archive id
• having the name in the metadata as an archive series identifier
• we already have the timestamp in the metadata anyway

[ThomasWaldmann]

See #7930.

• need to be a bit careful, as we still need to work with old borg 1.x repos until everyone has borg-transferred their stuff
• guess borg transfer will need some way to create a "series" from a match on the archive name. alternatively, first do the transfer with names as they are and then have a rename command that does the trick.