Broken NIX_PATH tracking issue

[roberth]

Describe the bug

NIX_PATH environment variable does not work when nix.conf has a nix-path setting.
This is highly unexpected and not intentional.
Multiple issues and PRs exist, but solving this problem has proven difficult within the established abstractions for settings handling.

Issues

• nix-env --query --available does not follow NIX_PATH #8784
• nix-shell fails to parse NIX_PATH #8890

PRs

• Restore NIX_PATH taking precedence over nix-path #7871
• Make the NIX_PATH environment variable take precedence over config files #8902

Priorities

Add 👍 to issues you find important.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/disabling-channels-breaks-nix-path-resolution/34825/3

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nix-path-is-not-recognized/38404/6

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-04-22-nix-team-meeting-minutes-141/44083/1

[colonelpanic8]

@roberth @fricklerhandwerk I'm happy to take another swing at this one.

Seems like https://github.com/NixOS/nix/pull/9066/files should probably be used as a starting point for testing.

w.r.t. your comment here #8902 (comment)

it seems like you are saying that there is probably some issue with what I did in this pr, w.r.t. the interaction between NIX_PATH and restrict-eval, but its not quite clear to me what exactly the issue is, and whether you and @thufschmitt ever did sit down and figure out a resolution or not. Just want to make sure that I'm not missing any details that you guys worked out or decided on before I spend more time on this.

Happy to jump on a call or discuss over text chat or something as well.

Thanks!

[colonelpanic8]

Would also be happy to use #7871 as a starting point if we feel like that is a better route.

I don't have a super strong recollection of what I even did in my PR, but I do vaguely recall that it did have the advantage of having a more "correct" version of the behavior w.r.t. precedence of the command line arguments.

[fricklerhandwerk]

@colonelpanic8 thanks a lot, I appreciate your support! Feel free to join the maintainer meetings if you'd like to talk.

We did sit down, which resulted in the top post of #9066 that describes the expected behavior, which is currently violated. It should be tested exhaustively and then fixed. We discussed last time (and the details aren't written down because there were too many) that the test setup as it is stands in the way to do this cleanly, and we'd like to first see how to approach that.

[colonelpanic8]

@fricklerhandwerk oh, wait so it seems like you just happened to pick this up again recently? I had assumed that since so much time had passed since the creation of #9066 that no one was actively working on this and that there may not even be any plans to fix the issue.

I'm definitely eager to help out here (this issue is kind of a major annoyance for me) but if you are already actively working on fixing the issue (and it sounds like maybe you have plans to pair on it with eelco), I wonder if I might just be stepping on toes by involving myself.

I'd love to hear your honest opinion on whether my involvement would be helpful at this stage. I'm definitely happy to attend the meeting, and work on any parts of this (even if they are tedious as with the testing).

[SuperSandro2000]

I have the suspicion that the search.nixos.org import broke because of this https://github.com/NixOS/nixos-search/actions/runs/8851738555/job/24308951178

Discussion in matrix https://matrix.to/#/!RROtHmAaQIkiJzJZZE:nixos.org/$CW01HdjRSMSyxXZlVw5cHassVIidQBjYDN8sAqDcR54?via=nixos.org&via=matrix.org&via=nixos.dev

[fricklerhandwerk]

already actively working on fixing the issue

This would be an exaggeration. It's on my mind and I dabbled at it, but I didn't make progress. Talking it through and walking through the code with someone would definitely help with that. But don't rely on me. Feel free to make your own investigations and discuss them here or as pull requests.

[piegamesde]

While this is definitely a Nix bug, I'd argue that the NixOS module is broken as well: The nix.nixPath option sets environment.sessionVariables.NIX_PATH instead of settings.nix-path, which is surprising.

However, even with both of these fixed, I'd argue that the general behavior and interaction between these two would be suboptimal and surprising: If one configures the Nix path through nix.conf and then a script goes along and wants to append to NIX_PATH, it will effectively override the existing path instead of appending, which is clearly not intended. (The alternative of appending/merging both is equally problematic for other reasons. Maybe the two simply should not coexist …)

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/24-05-add-flake-to-nix-path/46310/11

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-06-05-nix-team-meeting-minutes-150/46583/1

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/why-is-an-older-version-of-an-app-getting-installed-here/49413/7