borg create without syncing cache

[enkore]

code exists, needs testing, integration, but not a priority now

[enkore]

master...enkore:f/avoid-cache-sync

n.b. this bricks the cache due to the manifest id string in the config written by AdHocChunksCache (or so); that's one of those things meant by "testing, integration". Also the political and protocol issue of csize. 1.1b5 at earliest.

[ThomasWaldmann]

@enkore that adhoc chunks cache is a cool idea, I like it. \o/

I think this need to be broken down into multiple steps:

1. a refactoring PR that creates the class structure as you have in your branch, but does no functional changes (or additions). So it would only implement the local chunks cache as we have it now. as that new structure seems cleaner / easier, I guess we could do that in any case. and it should be pretty easy to review and get merged.

2. we need to agree about csize (discussing it in detail, all the places where it occurs, all the usages it has, and what to do with it), see the new (c)size ticket #2357, and implement the csize change, if any

3. implement your adhoc chunks cache

[ThomasWaldmann]

btw, i guess it would be useful if you do a DONTMERGE PR with your current branch, just to be able to get detailled feedback.

[enkore]

(moved to #2357 (comment) )

[ThomasWaldmann]

#2313 (comment) I'ld put a bounty on 1. immediately and after discussion of 2. maybe another one on 3. - but needs separate tickets first.

For 3. first check existing bounties, there might be already one or two...

[ThomasWaldmann]

Talked with @textshell and we both thought that this maybe should be shifted to after 1.1.x due to unclear compatibility / effort / scope.

It's a bit of a pity, it is a super interesting feature, but otoh, we should not delay 1.1 too much.

@enkore what do you think?

[enkore]

It doesn't matter, because planning on a per-issue level doesn't work with the dynamics of an open source project.
If it works out, good, otherwise, good. There is no bad outcome here.

---

Re. comment No. 3

(1) It's clear now that splitting the cache up into chunks and files is not a meaningful change. Implementing #2487 makes a lot more sense and will give way to have two different Cache implementations, which is meaningful.

Superficially it looks like chunks and files caches are orthogonal and should/could be split up, but they're not (this is not obvious, because the dependency between the two are outside the Cache: you can't make use of the data in the files cache without a proper chunks cache).

Meanwhile, SecurityManager and it's dependencies in the Cache implementation and the Cache are, in fact, orthogonal, and should be split up.
(2) The solution space is pretty much completely defined by the problem, most importantly compatibility. See the other ticket (#2357).
(3) Implementing things is easy and not the problem here.

[enkore]

I'll prepare a patch this evening.