Broken symlink when trying to create a fixed output derivation using flakes

[rgrunbla]

Describe the bug

When trying to build the derivation at https://github.com/sisyphe-re/RIOT with flakes using nix build, a broken symlink to the store is created. The derivation is a fixed output derivation.

Steps To Reproduce:

[remy@typhoon:~/TMP]$ nix --version
nix (Nix) 2.4pre20210503_6d2553a

[remy@typhoon:~/TMP]$ nix build github:sisyphe-re/RIOT#RIOT

[remy@typhoon:~/TMP]$ ls -alh
total 32K
drwxr-xr-x  2 remy users 4.0K May 30 22:35 .
drwx------ 85 remy users  20K May 30 22:34 ..
lrwxrwxrwx  1 remy users   60 May 30 22:35 result -> /nix/store/h818mqmf8n7bp8cg6y10flmnfsr0rdl2-RIOT-2021.04-RC3

[remy@typhoon:~/TMP]$ ls -alh /nix/store/h818mqmf8n7bp8cg6y10flmnfsr0rdl2-RIOT-2021.04-RC3
ls: cannot access '/nix/store/h818mqmf8n7bp8cg6y10flmnfsr0rdl2-RIOT-2021.04-RC3': No such file or directory

[remy@typhoon:~/TMP]$

Expected behavior

The result should probably not be a broken symlink, and I expect to have access to the output of the fixed output derivation.

nix-env --version output : nix (Nix) 2.4pre20210503_6d2553a

[rgrunbla]

A simpler repository reproducing the problem: https://github.com/rgrunbla/hello-broken-flakes

[remy@typhoon:~/]$ nix build github:rgrunbla/hello-broken-flakes

[remy@typhoon:~/]$ ls -alh result
lrwxrwxrwx 1 remy users 49 31 mai   10:13 result -> /nix/store/cah002sv267nggm967gaj9vlcqw0zpvq-hello

[remy@typhoon:~/]$ file result
result: broken symbolic link to /nix/store/cah002sv267nggm967gaj9vlcqw0zpvq-hello

[remy@typhoon:~/]$

[thufschmitt]

I can’t reproduce this (with Nix 2.4pre20210519_af4ff64). In both cases, the result link points to an existing store path. Do you mind sharing the output of nix show-config to see whether there’s something of interest there?

[rgrunbla]

Here is the config, @regnat : https://0x0.st/-2wf.txt

I tried on a single user machine with nix (Nix) 2.4pre20201205_a5d85d0 and it was working well, then I updated to nix (Nix) 2.4pre20210503_6d2553a and it is not working anymore. Is your nix-daemon also at 2.4pre20210519_af4ff64 ?

Using your nix version:

remy@medina:~/nix/TMP$ ../result/bin/nix build github:rgrunbla/hello-broken-flakes
remy@medina:~/nix/TMP$ ls -alh result
lrwxrwxrwx 1 remy debian 49 May 31 09:59 result -> /nix/store/cah002sv267nggm967gaj9vlcqw0zpvq-hello
remy@medina:~/nix/TMP$ file result
result: broken symbolic link to /nix/store/cah002sv267nggm967gaj9vlcqw0zpvq-hello
remy@medina:~/nix/TMP$ ../result/bin/nix --version
nix (Nix) 2.4pre20210519_af4ff64

[rgrunbla]

I bisected the problem, it seems it has been introduced by either 58cdab6 or 3ac9d74 from #4330

[thufschmitt]

Ah I managed to reproduce it and found (part) of the issue. The thing that makes these derivations special is that they have a self-reference.

(I’m actually quite surprised that this ever worked at all, because references in ca derivations isn’t supposed to be allowed unless the ca-derivations experimental feature is enabled. Looks like there’s a bug in the way we check for these)

But the thing is that somehow having these references makes Nix hash the output path twice (once modulo-references and once not), causing the result symlink to point at something different than the actual output path.

“amusingly”, enabling the ca-derivations experimental feature bypasses the issue because in that case the result symlink will be computed from the hash that’s stored in the db rather than the one in the derivation file.

[ncfavier]

What self-reference are you talking about?

Changing src = self to src = ./. doesn't seem to make a difference.

[thufschmitt]

What self-reference are you talking about?

Changing src = self to src = ./. doesn't seem to make a difference.

I mean self-references at the level of store paths (the ones mentioned in https://www.tweag.io/blog/2020-11-18-nix-cas-self-references/ ). They are annoying for content-addressed derivations (FO or not) because they need to point to the the store path under which the result will be stored, but that path is built from the content of the output, including the reference (so you’re kind-of screwed if you try to do things directly that way)

[ncfavier]

Right, but what part of this flake introduces a self-reference?

Running strings on the single binary output, the only store path I find is glibc.

[thufschmitt]

Running strings on the single binary output, the only store path I find is glibc.

Uh you’re right, I spoke too fast.

I still think this is due to the reference. Looks like there’s something weird going on around https://github.com/NixOS/nix/blob/master/src/libstore/store-api.cc#L183-L186 (And interestingly, if you tweak the derivation to use another hash method than sha256, https://github.com/NixOS/nix/blob/master/src/libstore/store-api.cc#L186 will trigger an assertion failure)

[thufschmitt]

After a bit more investigation, the discrepancy comes from

nix/src/libexpr/primops.cc

Line 1257 in 7f31c08

auto outPath = state.store->makeFixedOutputPath(method, h, drvName);

which computes the output path as it is stored in the derivation. This call doesn’t take into account the references of the output path (which is quite logical given that it happens before the build).
Otoh, the actual output path is computed here and does take into account these references. So the two differ, hence the issue.

I’m not sure how we could properly solve this without breaking backwards-compatibility because there’s no way we can know the final output path at evaluation time

EDIT: Replaced the links by updated permanent links

[thufschmitt]

After a second thought, there,s an “obvious” (though not trivial to implement) solution, which is to pre-declare the references for CA derivations. So having an extra argument outputReferences to builtins.derivation that would contain the list of paths that the output should refer to (and that would be empty by default). So for example, the RIOT derivation in the issue would have outputReferences = [ stdenv.glibc ].

That way, the eval-time computation of the output path can match the build-time one (and we can fail if the list of declared references doesn’t match the actual one), and everyone’s happy.

[rgrunbla]

Hello.

I'm pretty sure I encountered this bug again, trying to do something totally different than the previous time. Are you working on this, @regnat ? (not pressuring you to do anything, just asking)

Thanks,

[thufschmitt]

Are you working on this, @regnat ?

Nah sorry, didn’t have time to put more work into this unfortunately

[rgrunbla]

Hello, using nix (Nix) 2.5pre20211126_55275fc, it seems the steps below are now working. Is the problem being worked-on or is it just some happy coincidence ?

A simpler repository reproducing the problem: https://github.com/rgrunbla/hello-broken-flakes

[remy@typhoon:~/]$ nix build github:rgrunbla/hello-broken-flakes

[remy@typhoon:~/]$ ls -alh result
lrwxrwxrwx 1 remy users 49 31 mai   10:13 result -> /nix/store/cah002sv267nggm967gaj9vlcqw0zpvq-hello

[remy@typhoon:~/]$ file result
result: broken symbolic link to /nix/store/cah002sv267nggm967gaj9vlcqw0zpvq-hello

[remy@typhoon:~/]$

[thufschmitt]

Hello, using nix (Nix) 2.5pre20211126_55275fc, it seems the steps below are now working. Is the problem being worked-on or is it just some happy coincidence ?

Unfortunately, that’s mostly a coincidence, and it still doesn’t really work because although it’s now possible to build a FO derivation with references, it’s not possible to depend on it. For example building github:regnat/hello-broken-flakes will fail

[rgrunbla]

It works here, it seems:

[remy@typhoon:~]$ nix build github:regnat/hello-broken-flakes

[remy@typhoon:~]$ ./result/bin/hello
Hello World
[remy@typhoon:~]$ nix --version
nix (Nix) 2.5pre20211126_55275fc

[thufschmitt]

That’s utterly strange… is your daemon also using the same Nix version?

[rgrunbla]

Sorry for the delay, but yes, the version is the same.

[stale]

I marked this as stale due to inactivity. → More info

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/differences-in-evaulation-with-fixed-output-dependencies-under-darwin-and-linux/20592/5