Indexing solution and single archive repo questions

[fullofcaffeine]

Is this a BUG / ISSUE report or a QUESTION?

QUESTION

---

I've been using borg-backup for a while and I just love it. In my quest for the "perfect" backup setup, borg has been the key piece.

I've managed to backup most of my files, including old DVDs to the cloud using borg. So far, I have a couple of borg repos, one for my main wokstation home, one for my projects, one for system (timeshift snapshots) and another big repo for "static" archives.

By static, I mean things that don't change often or perhaps will never change, i.e old DVDs, photos, movies, etc. For those, I'm experimenting a single borg repo called "static" with one archive per source, where source might be different directories from different computers (or from old CDs/DVDs) that might (or will) cease to exist after it's been archived.

1. One of the challenges now though is to find a specific file from some metadata index that could tell me exactly in what borg repo / borg archive it is without the need for me to actually access and borg list the actual repo. Think something like recoll but adapted to work with borg indexes. I could keep a manual index for now, but maybe there's something like this out there already? Does anyone know? Any insights appreciated :)

2. Regarding this big "static" archive repo, Is it okay to use a single borg repo with sources from different directories / boxes with unrelated files? (Effectively a many-in-one setup)? Any pitfalls I should be aware of? Maybe there's a better way of achieving it?

PS: Do you have a web forum or is it okay to post questions like this here?

Thanks in advance!

[ThomasWaldmann]

Partly duplicate of #4092.

You can use a single repo to store your stuff, but be aware that you can not run 2 backups to same repo at same time. If it is really a lot, multiple repos are likely a better way. If you have stuff separated, you can run backups in parallel, run checks on each separately, have less of an issue in case something gets corrupted.

It's ok to post questions here, just maybe try to first find if the question was already answered in the past.
There is also a mailing list for those who prefer that.

[fullofcaffeine]

Hi Thomas,

If it is really a lot, multiple archives are likely a better way. If you have stuff separated, you can run backups in parallel, run checks on each separately, have less of an issue in case something gets corrupted.

You mean multiple repos right?

I think you have a point, multiple repos per topic might be better, except if I expect a lot of duplicate files from different sources, them it might make sense to have them in a single repo.

Regarding indexing, thanks for the link. borg list works well, although in a perfect world something like what perkeep indexing layer does would be awesome, perhaps an indexing (separate) tool with a nice interface could be developed someday (as a separate project). One can dream :) For now, keeping a cache of the output of borg-list per repo::archive does the job.

Thanks!

[ThomasWaldmann]

yup, fixed my post, repos of course. and i've tried whoosh, see the linked issue.

[ThomasWaldmann]

guess we can close this, we still have #4092.