Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose API for stat and type cache #1246

Open
lesnitsky opened this issue Jul 27, 2023 · 4 comments
Open

Expose API for stat and type cache #1246

lesnitsky opened this issue Jul 27, 2023 · 4 comments
Labels
Eng-Backlog feature request Feature request: request to add new features or functionality p2 P2

Comments

@lesnitsky
Copy link

lesnitsky commented Jul 27, 2023

I've tried GCSFuse on Cloud Run both with cache enabled and disabled and I'm seeing a significant difference in performance. I understand where consistency limitations are coming from when cache is enabled, however I still want performance benefits of cached stats. Our system has a full control over storage buckets, so updating a local cache is as simple as subscribing to a pubsub topic. Is there any way to manipulate the cache from outside and if not – would you consider exposing such an API?

As far as I understand reading through the docs, stat cache is both positive and negative, however, new objects becoming available over time (due to other actors creating those objects) is quite a common scenario, so maybe it makes sense to make negative caching opt-in.

@lesnitsky lesnitsky added p1 P1 question Customer Issue: question about how to use tool labels Jul 27, 2023
@ashmeenkaur ashmeenkaur added the feature request Feature request: request to add new features or functionality label Jul 28, 2023
@ashmeenkaur
Copy link
Collaborator

Hi @lesnitsky,

We won't be able to expose an API to modify GCSFuse cache as it is an internal resource, and we don't want to allow external updates to it.
However, I do think that your suggestion to make negative cache opt-in is an interesting use case. I've passed your suggestion along to our team, and we'll be evaluating it internally. We'll get back to you as soon as we have a decision.

In the meantime, I would suggest you to evaluate use of --stat-cache-ttl flag to figure out if setting a lower time-to-live helps your use case. (Detailed documentation here)

Thanks,
Ashmeen

@ashmeenkaur ashmeenkaur added p2 P2 and removed p1 P1 question Customer Issue: question about how to use tool labels Jul 28, 2023
@lesnitsky
Copy link
Author

lesnitsky commented Jul 28, 2023

lower ttl won't help, our system is pretty much "if the object is there, it's there forever" so caching makes a lot of sense (and I'd rather increase default values), but also new objects should be available right away – so negative caching is a blocker here.

@christopher-kwan-ai
Copy link

Having a similar constraint too so I would be interested on being able to separately configure positive/negative results.

@gargnitingoogle
Copy link
Collaborator

Created internal FR id 333354655 .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Eng-Backlog feature request Feature request: request to add new features or functionality p2 P2
Projects
None yet
Development

No branches or pull requests

5 participants