-
-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Have garbage collection leave store paths until they've been dead for a given amount of time #7572
Comments
I've really wanted an LRU option for GC that would age things out of the store. |
We were just talking about this today.
That is doable for leaf dead objects, but not so easy for dead objects that are referenced by other dead objects. But I think this is OK. Polling as part of "auto gc" should be fine. |
I've been playing with some ideas around this over at https://github.com/risicle/nix-heuristic-gc |
I was just thinking about this again, and I realized it's actually pretty simple to implement. I had been thinking in terms of tracking the "time since death" of store paths, which sounds hard, but you only actually need to track the "time since death" of gcroots. Here's an outline of how it could work:
|
Leaving until dead for a while would be nice, but I would also be pretty happy with just a minimum age before deletion. I can imagine This would give me a pretty great setup Run Maybe as a bonus |
Is your feature request related to a problem? Please describe.
It's hard to decide how often to schedule garbage collection.
If you set it too long, too much builds up in your nix store.
If you set it too short, you lose caching on transient shells too often and find yourself downloading the same flake commits constantly.
In fact, no matter how long you set it, it sometimes deletes what you were just using 5 minutes ago.
Describe the solution you'd like
I'd like nix to keep track of how much time has passed since a store path "died" (lost all gcroots keeping it in the store), and be capable of garbage collecting only store paths for which that amount of time is over a user supplied value. Ideally this would not be an approximation based on polling, but exact, and kept up to date by the nix daemon, at least in multi-user setups. However, if it is based on polling, transient commands (such as
nix run
) should also record the momentary "liveness" directly even if the polling window doesn't hit them.With that, you could set nix to garbage collect daily, but only delete objects that have been consistently dead for more than, say, 2 weeks. Dev shells and other transient phenomenon that you use often would never be garbage collected, but outdated objects would still disappear in a reasonable amount of time.
It would be somewhat unintuitive for old generations to actually still be in the nix store for quite a while after their profile symlinks disappear, but this could also be fixed by having 2 classes of gcroots. One of which incurs the wait period before deletion, the other of which does not. It's debatable whether that's worthwhile, however.
Describe alternatives you've considered
#2793 is more easily implemented, but wouldn't match what users really want from a cache expiry system as well.
Additional context
Would help significantly with #4250
I brought it up earlier here: https://discourse.nixos.org/t/what-would-you-like-to-see-improved-in-nix-cli-experience/24012/15
Priorities
Add 👍 to issues you find important.
The text was updated successfully, but these errors were encountered: