Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GHC >= 9.6 TemplateHaskell doesn't work in pkgsStatic #275304

Open
bgamari opened this issue Dec 19, 2023 · 40 comments · Fixed by #275609
Open

GHC >= 9.6 TemplateHaskell doesn't work in pkgsStatic #275304

bgamari opened this issue Dec 19, 2023 · 40 comments · Fixed by #275609
Labels
0.kind: regression Something that worked before working no longer 6.topic: haskell 6.topic: static

Comments

@bgamari
Copy link
Contributor

bgamari commented Dec 19, 2023

Describe the bug

Haskell packages in nixpkgs.pkgsStatic.haskell.packages.ghc98 are unable to be built.

Steps To Reproduce

Steps to reproduce the behavior:

  1. nix build nixpkgs#legacyPackages.x86_64-linux.pkgsStatic.haskell.packages.ghc98.Diff

Expected behavior

Diff is built, linking against musl.

Observed behavior

nix-repl> :b legacyPackages.x86_64-linux.pkgsStatic.haskell.packages.ghc98.Diff                                                                                                                                               error: build of '/nix/store/ickzn6az4asiwrh1wq20blm0dvcr5w17-Diff-static-x86_64-unknown-linux-musl-0.4.1.drv' on 'ssh://ben@maurer.local' failed: builder for '/nix/store/ickzn6az4asiwrh1wq20blm0dvcr5w17-Diff-static-x86_64-unknown-linux-musl-0.4.1.drv' failed with exit code 1;
       last 10 log lines:
       >    | ^^^^^^^^^^^^^^^^^^^^^^^^^^
       >
       > src/Data/Algorithm/Diff.hs:29:1: error: [GHC-47808]
       >     Failed to load dynamic interface file for Data.Array:
       >       Exception when reading interface file  /nix/store/zb7g1q1vza1x0fmb8qk8cv9y23b9w81g-x86_64-unknown-linux-musl-ghc-9.8.1/lib/x86_64-unknown-linux-musl-ghc-9.8.1/lib/../lib/x86_64-linux-ghc-9.8.1/array-0.5.6.0-inplace/Data/Array.dyn_hi
       >         /nix/store/zb7g1q1vza1x0fmb8qk8cv9y23b9w81g-x86_64-unknown-linux-musl-ghc-9.8.1/lib/x86_64-unknown-linux-musl-ghc-9.8.1/lib/../lib/x86_64-linux-ghc-9.8.1/array-0.5.6.0-inplace/Data/Array.dyn_hi: withBinaryFile: does not exist (No such file or directory)
       >    |
       > 29 | import Data.Array (listArray, (!))
       >    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
       > load' failed
       For full logs, run 'nix log /nix/store/ickzn6az4asiwrh1wq20blm0dvcr5w17-Diff-static-x86_64-unknown-linux-musl-0.4.1.drv'.
error: builder for '/nix/store/ickzn6az4asiwrh1wq20blm0dvcr5w17-Diff-static-x86_64-unknown-linux-musl-0.4.1.drv' failed with exit code 1;
       last 10 log lines:
       >    | ^^^^^^^^^^^^^^^^^^^^^^^^^^
       >
       > src/Data/Algorithm/Diff.hs:29:1: error: [GHC-47808]
       >     Failed to load dynamic interface file for Data.Array:
       >       Exception when reading interface file  /nix/store/zb7g1q1vza1x0fmb8qk8cv9y23b9w81g-x86_64-unknown-linux-musl-ghc-9.8.1/lib/x86_64-unknown-linux-musl-ghc-9.8.1/lib/../lib/x86_64-linux-ghc-9.8.1/array-0.5.6.0-inplace/Data/Array.dyn_hi
       >         /nix/store/zb7g1q1vza1x0fmb8qk8cv9y23b9w81g-x86_64-unknown-linux-musl-ghc-9.8.1/lib/x86_64-unknown-linux-musl-ghc-9.8.1/lib/../lib/x86_64-linux-ghc-9.8.1/array-0.5.6.0-inplace/Data/Array.dyn_hi: withBinaryFile: does not exist (No such file or directory)
       >    |
       > 29 | import Data.Array (listArray, (!))
       >    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
       > load' failed
       For full logs, run 'nix log /nix/store/ickzn6az4asiwrh1wq20blm0dvcr5w17-Diff-static-x86_64-unknown-linux-musl-0.4.1.drv'.

Additional context

The problem here appears to manifest during building of Haddock documentation. For instance,

$ nix repl
nix-repl> :lf nixpkgs#
nix-repl> :b legacyPackages.x86_64-linux.haskell.lib.dontHaddock (legacyPackages.x86_64-linux.pkgsStatic.haskell.packages.ghc98.Diff)

This derivation produced the following outputs:
  out -> /nix/store/5096rq1664b9qq2f2g9ih0sv6is48d8z-Diff-static-x86_64-unknown-linux-musl-0.4.1

Notify maintainers

@nh2

@bgamari bgamari added the 0.kind: bug Something is broken label Dec 19, 2023
@bgamari
Copy link
Contributor Author

bgamari commented Dec 19, 2023

My suspicion here is that this is either a Cabal or Haddock bug, although I'm not yet sure which.

@angerman
Copy link
Contributor

Of note: something assumes the existence of dynamic files, while there are none. Not quite sure why GHC would try to load dynamic files.
Trying to read

/nix/store/s13v3xsi60z627ic821fm70mlw43a3za-x86_64-unknown-linux-musl-ghc-9.8.1/lib/x86_64-unknown-linux-musl-ghc-9.8.1/lib/../lib/x86_64-linux-ghc-9.8.1/array-0.5.6.0-inplace/Data/Array.dyn_hi

however

/nix/store/s13v3xsi60z627ic821fm70mlw43a3za-x86_64-unknown-linux-musl-ghc-9.8.1/lib/x86_64-unknown-linux-musl-ghc-9.8.1/lib/../lib/x86_64-linux-ghc-9.8.1/array-0.5.6.0-inplace/Data
total 27K
dr-xr-xr-x 3 root root    5 Jan  1  1970 .
dr-xr-xr-x 3 root root    7 Jan  1  1970 ..
dr-xr-xr-x 6 root root   22 Jan  1  1970 Array
-r--r--r-- 2 root root 2.9K Jan  1  1970 Array.hi
-r--r--r-- 2 root root 2.9K Jan  1  1970 Array.p_hi

Maybe someone has an idea where the dyn_hi load comes from.

@rnhmjoj
Copy link
Contributor

rnhmjoj commented Dec 19, 2023

ping: @NixOS/static

@sternenseemann
Copy link
Member

Good to know that the hadrian regression from #208959 has been fixed, so we can at least build GHC now.

@sternenseemann
Copy link
Member

My diagnosis is the following:

  • Hadrian silently disables building the haddock executable, presumably because docs are disabled (so maybe building core lib docs and building the haddock executable is still the same flag?)
  • This is not known to haskellPackages.mkDerivation, since I dropped the enableHaddockProgram flag when I ported the GHC expression to Hadrian, presumably either because cross was completely broken initially or I assumed it got fixed.
  • When Cabal gets told to build docs, it uses the only haddock in scope, the one from the build->build compiler which doesn't work of course.

I can fix that by just disabling haddock in the same way as we do for GHC < 9.6. I'll try doing that later.

@bgamari @angerman The question is of course, and you can answer that better than me, has anything changed w.r.t. hadddock and cross with Hadrian?

@bgamari
Copy link
Contributor Author

bgamari commented Dec 19, 2023

Thanks @sternenseemann! Your hypothesis does sound plausible.

Recently we did rework Haddock to take documentation from Haskell interface (.hi) files. I can't help but wonder whether this logic may be culpable: https://gitlab.haskell.org/ghc/haddock/-/blob/b0b0e0366457c9aefebcc94df74e5de4d00e17b7/haddock-api/src/Haddock.hs#L170. This was apparently introduced due to haskell/haddock#256.

@sternenseemann
Copy link
Member

sternenseemann commented Dec 19, 2023

Seems plausible. I'm personally not too fussed that this change means that haddock is not “retargetable”, i.e. you always need to use the precise haddock bundled with the GHC you are using to compile the documented code. In fact, we probably should explicitly tell Cabal which haddock to use, so this kind of issue doesn't happen or is easier to diagnose.

I'll need to investigate, though, under which circumstances we can build haddock with hadrian now.

@sternenseemann
Copy link
Member

The problems seems to be that the haddock package is only built using the stage1 compiler (so as part of stage2) which we necessarily never reach in the case of cross compilation. Presumably we can work around this in UserSettings somehow (although IME you are quite limited if your solution is to be maintainable), but I feel like this is a genuine gap and there ought to be a better way to build a cross-compiler with hadrian…

@angerman
Copy link
Contributor

I've just skimmed the code, but why do we do this:

  -- Inject dynamic-too into ghc options if the ghc we are using was built with
  -- dynamic linking
  flags'' <- ghc flags $ do
        df <- getDynFlags
        case lookup "GHC Dynamic" (compilerInfo df) of
          Just "YES" -> return $ Flag_OptGhc "-dynamic-too" : flags
          _ -> return flags

what's the rational for adding -dynamic-too here? I can somewhat extract the rational from haskell/haddock#256, but the comment above this is rather poor. Also it does not provide any way to pass to haddock to prevent this automagic.

I guess the proper thing here is to just disable haddocks for cross, and rely on native compilers haddocks.

@sternenseemann
Copy link
Member

I guess the proper thing here is to just disable haddocks for cross, and rely on native compilers haddocks.

Do you mean the native compiler's haddock executable or re-using the documentation built natively? The former currently happens (unintentionally) and seems to be the source of the problem…

sternenseemann added a commit to sternenseemann/nixpkgs that referenced this issue Dec 20, 2023
In this situation, haddock would not be built by hadrian, as there is no
stage0:exe:haddock target by default. (We should eventually try adding
one.) If haddock is enabled and the build->host haddock missing, Cabal
tries using the build->build haddock which may fail to load the
documentation from the interface files produced by the build->host
GHC (e.g. due to a mismatch between dynamic and static linking).

Add regression tests to haskell-updates jobset.

Resolves NixOS#275304.
@sternenseemann sternenseemann moved this from Regression to In Progress in Haskell infrastructure in nixpkgs enhancements Dec 20, 2023
@sternenseemann sternenseemann linked a pull request Dec 20, 2023 that will close this issue
13 tasks
@angerman
Copy link
Contributor

@sternenseemann

re-using the documentation built natively

This :D

@domenkozar
Copy link
Member

domenkozar commented Jan 24, 2024

@domenkozar domenkozar reopened this Jan 24, 2024
@domenkozar
Copy link
Member

Even easier reproducer:

nix-build -A pkgsStatic.haskell.packages.ghc98.th-orphans

error: builder for '/nix/store/dibiy3qjbg2l5ahlqf28axfqz5xw91xn-th-orphans-static-x86_64-unknown-linux-musl-0.13.14.drv' failed with exit code 1;
       last 10 log lines:
       > /nix/store/20rsi77ny2i4i1rbd63h4392a245j5dz-gnutar-1.35/bin/tar
       > No uhc found
       > Running phase: buildPhase
       > Preprocessing library for th-orphans-0.13.14..
       > Building library for th-orphans-0.13.14..
       > [1 of 2] Compiling Language.Haskell.TH.Instances.Internal ( src/Language/Haskell/TH/Instances/Internal.hs, dist/build/Language/Haskell/TH/Instances/Internal.o )
       > [2 of 2] Compiling Language.Haskell.TH.Instances ( src/Language/Haskell/TH/Instances.hs, dist/build/Language/Haskell/TH/Instances.o )
       >
       > <no location info>: error:
       >     Couldn't find a target code interpreter. Try with -fexternal-interpreter
       For full logs, run 'nix log /nix/store/dibiy3qjbg2l5ahlqf28axfqz5xw91xn-th-orphans-static-x86_64-unknown-linux-musl-0.13.14.drv'.

@angerman
Copy link
Contributor

Even easier reproducer:

nix-build -A pkgsStatic.haskell.packages.ghc98.th-orphans

error: builder for '/nix/store/dibiy3qjbg2l5ahlqf28axfqz5xw91xn-th-orphans-static-x86_64-unknown-linux-musl-0.13.14.drv' failed with exit code 1;
       last 10 log lines:
       > /nix/store/20rsi77ny2i4i1rbd63h4392a245j5dz-gnutar-1.35/bin/tar
       > No uhc found
       > Running phase: buildPhase
       > Preprocessing library for th-orphans-0.13.14..
       > Building library for th-orphans-0.13.14..
       > [1 of 2] Compiling Language.Haskell.TH.Instances.Internal ( src/Language/Haskell/TH/Instances/Internal.hs, dist/build/Language/Haskell/TH/Instances/Internal.o )
       > [2 of 2] Compiling Language.Haskell.TH.Instances ( src/Language/Haskell/TH/Instances.hs, dist/build/Language/Haskell/TH/Instances.o )
       >
       > <no location info>: error:
       >     Couldn't find a target code interpreter. Try with -fexternal-interpreter
       For full logs, run 'nix log /nix/store/dibiy3qjbg2l5ahlqf28axfqz5xw91xn-th-orphans-static-x86_64-unknown-linux-musl-0.13.14.drv'.

That suggests that the GHC was not built as stage2 compiler, or some of the new cross target logic prohibits native codegen now as well.

@wolfgangwalther
Copy link
Contributor

Just confirmed this is still a problem with GHC 9.10.1.

To be precise: I tested pkgsStatic.haskell.packages.ghc9101.th-orphans still fails with the above external interpreter error message.

I did not test, at least yet, the self-bootstrapping approach I tried earlier with GHC 9.8. That might still be worth a try.

@sternenseemann sternenseemann changed the title Static linking with GHC 9.8.1 is broken GHC 9.8 TemplateHaskell doesn't work in pkgsStatic Jun 3, 2024
@NorfairKing
Copy link
Contributor

Good to know that the hadrian regression from #208959 has been fixed, so we can at least build GHC now.

Where/when was it fixed? I'd like to patch the ghc I'm using so I'd need to know which commit fixed this.

@NorfairKing
Copy link
Contributor

NorfairKing commented Sep 25, 2024

EDIT: Mistake, ignore.

@wolfgangwalther
Copy link
Contributor

wolfgangwalther commented Sep 25, 2024

With #208959 fixed, I now get the same external-interpreter error when building th-orphans for both GHC 9.6 and 9.8. This makes sense, because it's because of the hadrian build, which both use. So this issue really applies to both now.

Edit: And as mentioned above for GHC 9.10 as well. So basically for GHC 9.6 up.

@wolfgangwalther
Copy link
Contributor

Just confirmed this is still a problem with GHC 9.10.1.

To be precise: I tested pkgsStatic.haskell.packages.ghc9101.th-orphans still fails with the above external interpreter error message.

I did not test, at least yet, the self-bootstrapping approach I tried earlier with GHC 9.8. That might still be worth a try.

I was able to successfully bootstrap GHC 9.10.1 for pkgsStatic from GHC 9.10.1 itself this time.

It still doesn't solve the problem at hand, though:

<no location info>: error:
    Couldn't find a target code interpreter. Try with -fexternal-interpreter

All still the same.

@peterbecich
Copy link
Contributor

The original case in this issue #275304 (comment)

nix build nixpkgs#legacyPackages.x86_64-linux.pkgsStatic.haskell.packages.ghc98.Diff

succeeds for me, now.

@peterbecich
Copy link
Contributor

Agree with #275304 (comment);

nix build nixpkgs#legacyPackages.x86_64-linux.pkgsStatic.haskell.packages.ghc9101.th-orphans                     

produces the error.

Additional example:

nix build nixpkgs#legacyPackages.x86_64-linux.pkgsCross.aarch64-multiplatform.haskell.packages.ghc9101.th-orphans

produces the error

@nixos-discourse
Copy link

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/build-aarch64-docker-image-on-amd64-machine/55052/5

@peterbecich
Copy link
Contributor

peterbecich commented Nov 1, 2024

These also produce the error
9.6

nix build nixpkgs/haskell-updates#legacyPackages.x86_64-linux.pkgsStatic.haskell.packages.ghc96.th-orphans_0_13_15

9.8

nix build nixpkgs/haskell-updates#legacyPackages.x86_64-linux.pkgsStatic.haskell.packages.ghc98.th-orphans_0_13_15

@peterbecich
Copy link
Contributor

peterbecich commented Nov 1, 2024

@bgamari, can we change the title to "GHC 9.6, 9.8, 9.10 TemplateHaskell doesn't work in pkgsStatic, pkgsCross"?

dhess added a commit to hackworthltd/primer that referenced this issue Nov 3, 2024
Namely:

* Bump `ghc-wasm` flake to latest, which includes support for Template
Haskell and macOS. Note that we now need to pass
`-fno-external-interpreter` to GHC; this appears to be related to
NixOS/nixpkgs#275304.

* Run the Wasm build on macOS in CI, in addition to Linux.

* Don't bother building Haddocks in the Wasm build.

* Don't use `-fwrite-ide-info` in Wasm build.

* `allow-newer` for `time` package.

Signed-off-by: Drew Hess <src@drewhess.com>
@sternenseemann sternenseemann changed the title GHC 9.8 TemplateHaskell doesn't work in pkgsStatic GHC >= 9.8 TemplateHaskell doesn't work in pkgsStatic Nov 19, 2024
@wolfgangwalther wolfgangwalther changed the title GHC >= 9.8 TemplateHaskell doesn't work in pkgsStatic GHC >= 9.6 TemplateHaskell doesn't work in pkgsStatic Jan 19, 2025
@bgamari bgamari changed the title GHC >= 9.6 TemplateHaskell doesn't work in pkgsStatic GHC 9.6, 9.8, 9.10 TemplateHaskell doesn't work in pkgsStatic Feb 7, 2025
@sternenseemann
Copy link
Member

sternenseemann commented Feb 10, 2025

@wolfgangwalther GHC 9.12.1 (at least) does build and install ghc-iserv and friends. The advantage pkgsStatic has is that GHC's configure script never detects that we are, in fact, cross compiling.

@sternenseemann
Copy link
Member

The haskell.nix builder for GHC work around this by sidestepping hadrians build and install process and doing it a bit more explicit.

https://github.com/input-output-hk/haskell.nix/blob/6eaafcdf04bab7be745d1aa4f74d2cc85700042b/compiler/ghc/default.nix#L787

I'm not even sure Hadrian can (or should be fixed). The proper solution seems to just bin it outright and build GHC with cabal only.

@angerman I don't understand how this works on the haskell.nix side. Even if you manually force the build of iserv and install it, isn't ghc still built without -finternal-interpreter and won't ever attempt to use the internal target code interpreter?

@wolfgangwalther
Copy link
Contributor

@bgamari changed the title GHC >= 9.6 TemplateHaskell doesn't work in pkgsStatic GHC 9.6, 9.8, 9.10 TemplateHaskell doesn't work in pkgsStatic

Was this just as a reaction to #275304 (comment) or did you intentionally take 9.12 out of this, because it should do better?

We currently still get the same error for pkgsStatic.haskell.packages.ghc912.th-orphans:

       > <no location info>: error:
       >     Couldn't find a target code interpreter. Try with -fexternal-interpreter

Thus, changing back to "GHC >= 9.6".

If there is anything that was changed about this in 9.12, please let us know.

@wolfgangwalther wolfgangwalther changed the title GHC 9.6, 9.8, 9.10 TemplateHaskell doesn't work in pkgsStatic GHC >= 9.6 TemplateHaskell doesn't work in pkgsStatic Feb 15, 2025
@angerman
Copy link
Contributor

@angerman I don't understand how this works on the haskell.nix side. Even if you manually force the build of iserv and install it, isn't ghc still built without -finternal-interpreter and won't ever attempt to use the internal target code interpreter?

@sternenseemann I'm a bit confused. GHC should always be able to accept -fexternal-interpreter. I think it defaults to the internal if that is available, and -fexternal-interpreter isn't passed. Hence you can always pass -fexternal-interpreter for cross compilers (stage 1 or stage 2). Now for musl, where we basically build a ghc with an internal interpreter, yes, we shouldn't need iserv. haskell.nix though targets many more cross targets, and as such musl is just a very special case of cross for us. And we primarily tried to work around the insanity of Hadrian.

@sternenseemann
Copy link
Member

So does that mean that the iserv hadrian package and the internal-interpreter flag of e.g. the ghc hadrian package are independent of each other?
That would be kind of good since technically we only need the internal interpreter to deal with this regression since nixpkgs doesn't do anything with iserv yet (#248979).

I've been able to clear up my question how haskell.nix achieves this since – it just disables the hadrian cross logic altogether.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.kind: regression Something that worked before working no longer 6.topic: haskell 6.topic: static
Development

Successfully merging a pull request may close this issue.

9 participants