-
-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Virtual path semantics that are deterministic, lazy and suitable for performing deduplication of values by a path value key. #10689
Comments
The non-determinism issue can be solved in the common case by deriving the virtual path from the input fingerprint. This wouldn't work for impure inputs, but I suppose if you have impure inputs, it's to be expected that you might end up with non-determinism in your evaluation. It's also not obvious how to come up with a fingerprint for accessors like those returned by Another solution might be to restrict what you can do with a virtual path substring, e.g. make it illegal to call |
For stability, it'd be best to expose the fingerprints as little as possible, e.g. only their ordering but not their contents, but even exposing the ordering is a liability that we don't currently have.
We might want to consider locking them (in memory) anyway, if we don't already do that.
We could evaluate the set of all included paths and then fingerprint the set and the underlying source. It's eager at the language level, but does not require any copying.
Or fall back to copying. This could be done by adding string-specific thunk constructors (e.g. |
This issue has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/2024-08-07-nix-team-meeting-minutes-167/50287/1 |
Is your feature request related to a problem? Please describe.
This issue discusses the solution space for a crucial step to make NixOS evaluation not copy nixpkgs to the store.
#6530 solves the problem of lazy fetching while using the module system, by introducing another problem, which is new observable nondeterminism in the form of virtual path string placeholders (
/nix/store/virtual0000<n>
).Lazy paths (#10252) also add laziness, but the module system's use of paths causes it to have to copy nonetheless.
Describe the solution you'd like
Some possible solutions
This is an unsolved problem. Lazy trees will leak non-deterministic virtual path names in this case, which in my view is the primary reason why we can't just adopt that solution.
Possible solutions I'm aware of
nondeterministicComparePathRoot
function?nondeterministicPathRootIdString
? A new "path map" type?)Additional context
Discussed in meeting before, links tbd.
Also came up in
/
to mean path append; alternative topath + "/" + "foo"
footgun #7301 (comment)Priorities
Add 👍 to issues you find important.
The text was updated successfully, but these errors were encountered: