Add metadata search index for more responsive FUSE #182
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds the construction of a metadata index during archive creation,
which can be used to narrow down the location of particular entries
within the items list. The FUSE mount uses this index to fetch only
those chunks that are relevant to the specific operation instead of
fetching the metadata of the entire archive.
As a result, using FUSE mounts of large archives is consirably more
reponsive. And more importantly, the performance isn't indirectly
proportional to the the archive size anymore. Any bulk operations that
require the full metadata tree anyways (such as running "find" on the
entire archive) are not negatively impacted.
For this to work, the filesystem traversal order had to be changed from
depth-first to breadth-first, which introduces the new metadata version
number 2. Any otherwise unrelated parts of the code and tests that
relied on the previous behavior are adjusted accordingly.