Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File cache is not persistent across remounts #2137

Open
nbdd0121 opened this issue Jul 9, 2024 · 1 comment
Open

File cache is not persistent across remounts #2137

nbdd0121 opened this issue Jul 9, 2024 · 1 comment
Labels
doc Doc: problem with incorrect or missing documentation. feature request Feature request: request to add new features or functionality p1 P1 question Customer Issue: question about how to use tool

Comments

@nbdd0121
Copy link

nbdd0121 commented Jul 9, 2024

Describe the issue

When GCSFuse unmounts and remounts, the file cache populated doesn't seem to be accessed.

n https://cloud.google.com/storage/docs/gcsfuse-cache#persistence says that the file cache should be reused when the metadata cache has been populated, but it doesn't seem to be the case.

A brief grep through the codebase suggests that contentcache have the code that handles persistence of the cache, but this functionality is not present in the filecache code. The contentcache seems completely unused currently so the functionality is lost.

Reproduction

The following experiment is conducted:

  • Have a 1.7G file on a GCS bucket
  • Mount with the following options gcsfuse -o ro --file-mode=755 --config-file=gcs.yaml BUCKET_NAME /mnt with the config file below:
file-cache:
  max-size-mb: -1
  cache-file-for-range-read: true
metadata-cache:
  stat-cache-max-size-mb:  32
  ttl-secs: 3600
  type-cache-max-size-mb: 4
cache-dir: ~/.cache/gcsfuse
  • vmtouch -t /mnt/large_file takes ~18 seconds.
  • Subsequent vmtouch calls are also instant to complete.
  • umount /mnt and remount.
  • vmtouch -t /mnt/large_file still takes ~18 seconds.

System (please complete the following information):

  • OS: NixOS 24.05
  • Platform: Bare-metal
  • Version: 2.3.1

Additional context
Add any other context about the problem here.

SLO:
We strive to respond to all bug reports within 24 business hours.

@nbdd0121 nbdd0121 added p1 P1 question Customer Issue: question about how to use tool labels Jul 9, 2024
@charith87
Copy link
Collaborator

Thanks @nbdd0121 for reaching out.
This is a known limitation at this point that the file cache is not persistent across mounts.

The documentation could have been more clearer here. It actually tries to say the file cache will be re-used for subsequent operations once the metadata cache is filled. It assumes that file cache is also filled in the current session.
The lines together with the previous line is what might be misleading. Will work with the documentation team to change this appropriately.

The reason for the limitation right now is the file cache doesn't hold a marker of the generation(version) of the object on the disk ( which the metadata cache holds). There's no way to reconcile if the local file (cached already) and the new metadata (filled in the new mount session) are the same without additional changes. This is a roadmap item in the future but no timelines at this point.

cc @marcoa6

@charith87 charith87 added feature request Feature request: request to add new features or functionality doc Doc: problem with incorrect or missing documentation. labels Jul 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc Doc: problem with incorrect or missing documentation. feature request Feature request: request to add new features or functionality p1 P1 question Customer Issue: question about how to use tool
Projects
None yet
Development

No branches or pull requests

2 participants