Have garbage collection leave store paths until they've been dead for a given amount of time

[tejing1]

Is your feature request related to a problem? Please describe.
It's hard to decide how often to schedule garbage collection.

If you set it too long, too much builds up in your nix store.

If you set it too short, you lose caching on transient shells too often and find yourself downloading the same flake commits constantly.

In fact, no matter how long you set it, it sometimes deletes what you were just using 5 minutes ago.

Describe the solution you'd like
I'd like nix to keep track of how much time has passed since a store path "died" (lost all gcroots keeping it in the store), and be capable of garbage collecting only store paths for which that amount of time is over a user supplied value. Ideally this would not be an approximation based on polling, but exact, and kept up to date by the nix daemon, at least in multi-user setups. However, if it is based on polling, transient commands (such as nix run) should also record the momentary "liveness" directly even if the polling window doesn't hit them.

With that, you could set nix to garbage collect daily, but only delete objects that have been consistently dead for more than, say, 2 weeks. Dev shells and other transient phenomenon that you use often would never be garbage collected, but outdated objects would still disappear in a reasonable amount of time.

It would be somewhat unintuitive for old generations to actually still be in the nix store for quite a while after their profile symlinks disappear, but this could also be fixed by having 2 classes of gcroots. One of which incurs the wait period before deletion, the other of which does not. It's debatable whether that's worthwhile, however.

Describe alternatives you've considered
#2793 is more easily implemented, but wouldn't match what users really want from a cache expiry system as well.

Additional context
Would help significantly with #4250
I brought it up earlier here: https://discourse.nixos.org/t/what-would-you-like-to-see-improved-in-nix-cli-experience/24012/15

Priorities

Add 👍 to issues you find important.

[kjeremy]

I've really wanted an LRU option for GC that would age things out of the store.

[Ericson2314]

We were just talking about this today.

Ideally this would not be an approximation based on polling

That is doable for leaf dead objects, but not so easy for dead objects that are referenced by other dead objects. But I think this is OK. Polling as part of "auto gc" should be fine.

[risicle]

I've been playing with some ideas around this over at https://github.com/risicle/nix-heuristic-gc

[tejing1]

I was just thinking about this again, and I realized it's actually pretty simple to implement. I had been thinking in terms of tracking the "time since death" of store paths, which sounds hard, but you only actually need to track the "time since death" of gcroots.

Here's an outline of how it could work:

• Add 2 directories, /nix/var/nix/rootexpiry/current and /nix/var/nix/rootexpiry/old
  • rootexpiry/old contains symlinks to store paths
  • The symlinks in rootexpiry/old are named after the time at which the roots pointing to those store paths disappeared. (represented as seconds since the epoch, say)
  • rootexpiry/current contains symlinks that point to gcroots (sort of like how gcroots/auto has symlinks pointing to result symlinks out in the filesystem)
  • The symlinks in rootexpiry/current are named after the store paths the gcroots point to. (so that we still have that information when the gcroot itself disappears or changes)
• The nix daemon watches (with inotify on linux, say) the rootexpiry/current directory, as well as the targets of all the symlinks inside.
  • When one of the symlink targets disappears or changes target, the nix daemon deletes the symlink in rootexpiry/current and creates a corresponding entry in rootexpiry/old named after the current time. (actually, to avoid race conditions, do it in the other order)
  • In single-user installs, nix commands need to check and update rootexpiry/current whenever they happen to get run, since there's no daemon to watch it. This means the apparent time of expiry is pushed forward to the next nix command run, at worst, and is exactly correct, still, so long as nix commands are used to manipulate gcroots.
• Transient commands such as nix run register their own entries in rootexpiry/old automatically.
• Garbage collection can delete symlinks in rootexpiry/old whose names encode a time older than the timeframe specified by the user and then treat any remaining entries as gcroots, keeping them present when it does collection. It also needs to treat the store paths encoded in the names of symlinks in rootexpiry/current as gcroots, to avoid any race conditions.
• To better capture the timing information of all store objects, nix would also need to add entries in rootexpiry/old for the build dependencies whenever it builds a derivation (as opposed to substituting it), successfully or not. This ensures that build dependencies that are not runtime dependencies stick around for a sufficient amount of time.
• Old generations of profiles can be exempt from tracking in rootexpiry/current, making them also exempt from the wait time after their deletion.

[kevincox]

Leaving until dead for a while would be nice, but I would also be pretty happy with just a minimum age before deletion. I can imagine nix-collect-garbage --min-path-age 30d that only deletes paths that were added more than 30d ago. /nix/var/nix/db/db.sqlite already has ValidPaths.registrationTime which should be enough to implement this. Just skip deleting paths that are newer than the configured age.

This would give me a pretty great setup Run nix-collect-garbage --delete-older-than 30d --min-path-age 30d on a schedule to avoid collecting too much old data over time and configure min-free + max-free to do a more eager collection when space is limited.

Maybe as a bonus --max-freed and max-free could order deletions by the registrationTime as well.