pkgsMusl.haskell.compiler.ghc883 is broken (bootstrap with ghc865-binary fails) #85924

monacoremo · 2020-04-24T08:50:49Z

Describe the bug

Trying to build GHC 8.8.3 on pkgsMusl (in order to build static Haskell executables) fails with:

...
Installing library in /nix/store/1pqmyrp2f4y3kwnvp2isc09jff91f1bw-ghc-8.6.5-binary/lib/ghc-8.6.5/ghc-8.6.5
"/nix/store/1pqmyrp2f4y3kwnvp2isc09jff91f1bw-ghc-8.6.5-binary/lib/ghc-8.6.5/bin/ghc-pkg" --force --global-package-db "/nix/store/1pqmyrp2f4y3kwnvp2isc09jff91f1bw-ghc-8.6.5-binary/lib/ghc-8.6.5/package.conf.d" update rts/dist/package.conf.install
/nix/store/1pqmyrp2f4y3kwnvp2isc09jff91f1bw-ghc-8.6.5-binary/lib/ghc-8.6.5/bin/ghc-pkg: /nix/store/1i32nv0sm71qr3qak4jg9bqmlq4pq1la-ncurses-6.2-abi5-compat/lib/libtinfo.so.5: no version information available (required by /nix/store/1pqmyrp2f4y3kwnvp2isc09jff91f1bw-ghc-8.6.5-binary/lib/ghc-8.6.5/bin/ghc-pkg)
/nix/store/1pqmyrp2f4y3kwnvp2isc09jff91f1bw-ghc-8.6.5-binary/lib/ghc-8.6.5/bin/ghc-pkg: /nix/store/1i32nv0sm71qr3qak4jg9bqmlq4pq1la-ncurses-6.2-abi5-compat/lib/libtinfo.so.5: no version information available (required by /nix/store/1pqmyrp2f4y3kwnvp2isc09jff91f1bw-ghc-8.6.5-binary/lib/ghc-8.6.5/bin/../terminfo-0.4.1.2/libHSterminfo-0.4.1.2-ghc8.6.5.so)
Inconsistency detected by ld.so: dl-lookup.c: 111: check_match: Assertion `version->filename == NULL || ! _dl_name_match_p (version->filename, map)' failed!
make[1]: *** [ghc.mk:991: install_packages] Error 127
make: *** [Makefile:51: install] Error 2
builder for '/nix/store/ywiy47vvr8b9blq32kw2zcqqqpqrs7kc-ghc-8.6.5-binary.drv' failed with exit code 2
cannot build derivation '/nix/store/0aivgs76j9xrf9flbbs5mhlfc0gfqyk9-ghc-8.8.3.drv': 1 dependencies couldn't be built

To Reproduce

On a recent pin of nixpkgs-unstable (rev 10100a9):

nix-build -A pkgsMusl.haskell.compiler.ghc883

Expected behavior

GHC 8.8.3 is built based on Musl

Notify maintainers

@MarcWeber @andres @peti

Metadata

system: "x86_64-linux"
host os: Linux 5.0.0-38-generic, Ubuntu, 19.04 (Disco Dingo)
multi-user?: no
sandbox: yes
version: nix-env (Nix) 2.3.1
channels(remo): "nixos-20.03pre204199.3140fa89c51, nixpkgs-20.03pre204216.cc6cf0a96a6"
nixpkgs: /home/remo/.nix-defexpr/channels/nixpkgs

The text was updated successfully, but these errors were encountered:

cdepillabout · 2020-04-24T12:53:33Z

@monacoremo Thanks for creating this issue.

Unfortunately, we don't always check to make sure the musl builds succeed before merging stuff in.

If you wanted to fix this issue, we'd happily accept a PR.

Also, @nh2 is doing a lot with the musl builds, especially in https://github.com/nh2/static-haskell-nix. If you absolutely need a musl'd GHC, I'd suggest to use https://github.com/nh2/static-haskell-nix.

nh2 · 2020-04-24T13:43:22Z

To add to that: I pin a nixpkgs that works well in a submodule:

and the same in the nixpkgs.file:

https://github.com/nh2/static-haskell-nix/blob/acaffd5fb807aa78f106eb3b4a99b05d3a1ec043/nixpkgs.nix#L27-L28

On the issue: We've seen build failures on latest nixpkgs master with musl recently, hopefulyy they will be fixed soon!

peti · 2020-04-24T19:27:40Z

Just for the record, current versions of master (and haskell-updates) still have this issue. We'll try to build the package with 3c7ef6b reverted to check whether the update of the boot compiler caused this problem.

monacoremo · 2020-04-25T08:36:00Z

Many thanks all for your help and quick responses!

As @peti suggested I reverted 3c7ef6b and pkgsMusl.haskell.compiler.ghc883 is currently building, getting much further than on ghc865-binary - looking good! I'll update once the build is done, takes quite a bit of time...

Edit: Confirmed, successfully built /nix/store/x03kyh5mfrapm1mjrh5a5v4ygly6rzwz-ghc-8.8.3 !

I would like to contribute a fix if possible, but I'm not sure what direction would be best to take - GHC 8.8.3 binaries are now available, would it make sense to bootstrap from there?

Edit2: Maybe it just needs s/ncurses5/ncurses6/ in https://github.com/nixos/nixpkgs/blob/master/pkgs/development/compilers/ghc/8.6.5-binary.nix? Trying that out right now. Would that be likely to break anything else?

Edit3: Couldn't get the libtinfo.so dependency from ncurses to work for 8.6.5-binary on pkgsMusl... @dtzWill , I saw that you worked on related things, do you have an idea maybe how to solve this?

nh2 · 2020-06-07T18:45:48Z

CC @bgamari

bgamari · 2020-06-07T18:58:15Z

It looks to me like this is a bug in the ncurses-abi-compat derivation. In particular, the libtinfo it exposes has the incorrect soname. This is likely because is isn't libtinfo at all; rather, it's simply a symlink to libncursesw. In general I don't see how this could work.

bgamari · 2020-06-07T19:00:42Z

We work around this in mpickering/ghc-artefact-nix via https://github.com/mpickering/ghc-artefact-nix/blob/99fcf89d36d306beb243acccc87d1f6e153ad50f/artifact.nix#L104.

nh2 · 2020-06-07T19:35:41Z

Collecting some info for myself:

The ncurses-abi-compat derivation @bgamari refers to (called rather ncurses-*-abi5-compat in the issue description) is

nixpkgs/pkgs/development/libraries/ncurses/default.nix

Line 18 in 09b137a

name = "ncurses-${version}" + lib.optionalString (abiVersion == "5") "-abi5-compat";

Question:

@bgamari Do you know what a fix could look like? Instead of a symlink, should it be a copy of the .so file, on which patchelf --replace-needed is subsequently run, as shown in your link? If yes, should that happen unconditionally or only for abiVersion == "5"?

I don't know anything about the ncurses 5->6 switch, so I have no idea why this sophisticated mechanism is there in the first place.

bgamari · 2020-06-07T20:43:49Z

The ideal solution would be to rather use a GHC binary distribution which expects ncurses ABI 6. For instance, this patch appears to work around the issue locally:

diff --git a/pkgs/development/compilers/ghc/8.6.5-binary.nix b/pkgs/development/compilers/ghc/8.6.5-binary.nix
index 97793d91289..188b12f5dad 100644
--- a/pkgs/development/compilers/ghc/8.6.5-binary.nix
+++ b/pkgs/development/compilers/ghc/8.6.5-binary.nix
@@ -1,6 +1,6 @@
 { stdenv
 , fetchurl, perl, gcc
-, ncurses5, gmp, glibc, libiconv
+, ncurses6, gmp, glibc, libiconv
 , llvmPackages
 }:
 
@@ -11,7 +11,7 @@ let
   useLLVM = !stdenv.targetPlatform.isx86;
 
   libPath = stdenv.lib.makeLibraryPath ([
-    ncurses5 gmp
+    ncurses6 gmp
   ] ++ stdenv.lib.optional (stdenv.hostPlatform.isDarwin) libiconv);
 
   libEnvVar = stdenv.lib.optionalString stdenv.hostPlatform.isDarwin "DY"
@@ -38,8 +38,8 @@ stdenv.mkDerivation rec {
       sha256 = "1p2h29qghql19ajk755xa0yxkn85slbds8m9n5196ris743vkp8w";
     };
     x86_64-linux = {
-      url = "http://haskell.org/ghc/dist/${version}/ghc-${version}-x86_64-deb9-linux.tar.xz";
-      sha256 = "1pqlx6rdjs2110g0y1i9f8x18lmdizibjqd15f5xahcz39hgaxdw";
+      url = "http://haskell.org/ghc/dist/${version}/ghc-${version}-x86_64-fedora27-linux.tar.xz";
+      sha256 = "18dlqm5d028fqh6ghzn7pgjspr5smw030jjzl3kq6q1kmwzbay6g";
     };
     aarch64-linux = {
       url = "http://haskell.org/ghc/dist/${version}/ghc-${version}-aarch64-ubuntu18.04-linux.tar.xz";
@@ -100,9 +100,9 @@ stdenv.mkDerivation rec {
     '' +
     # Rename needed libraries and binaries, fix interpreter
     stdenv.lib.optionalString stdenv.isLinux ''
-      find . -type f -perm -0100 -exec patchelf \
+      find . -type f -perm -0100 \
+          -exec patchelf \
           --replace-needed libncurses${stdenv.lib.optionalString stdenv.is64bit "w"}.so.5 libncurses.so \
-          --replace-needed libtinfo.so libtinfo.so.5 \
           --interpreter ${glibcDynLinker} {} \;
 
       sed -i "s|/usr/bin/perl|perl\x00        |" ghc-${version}/ghc/stage2/build/tmp/ghc-stage2

bgamari · 2020-06-07T20:45:07Z

@nh2, I have opened #89769 to track the symlink issue.

* include workaround for NixOS/nixpkgs#85924 * musl 1.2.0 (#98 (comment)) * workaround for fontforge execution failure due to wrong RPATH (NixOS/nixpkgs#94126) * ilmbase musl compilation error

* include workaround for NixOS/nixpkgs#85924 * musl 1.2.0 (#98 (comment)) * workaround for fontforge execution failure due to wrong RPATH (NixOS/nixpkgs#94126) * ilmbase musl compilation error (NixOS/nixpkgs#94205) * mesa: Fix `-Werror=int-conversion` build error on musl fixup! nixpkgs: Update submodule:

* include workaround for NixOS/nixpkgs#85924 * musl 1.2.0 (#98 (comment)) * workaround for fontforge execution failure due to wrong RPATH (NixOS/nixpkgs#94126) * ilmbase musl compilation error (NixOS/nixpkgs#94205) * mesa: Fix `-Werror=int-conversion` build error on musl (NixOS/nixpkgs#94207)

This is a revamp of our use of [fully static Haskell executables](NixOS/nixpkgs#43795). The main changes are: * We can now use a newer revision of Nixpkgs The current revision we're using corresponds to Nixpkgs 20.03 * We can now more easily update Nixpkgs The logic in this repository more closely tracks how `static-haskell-nix` works, so whenever that repository updates Nixpkgs we can pick up the change here, too. Note that we cannot currently use Nixpkgs `master` due to [this issue](NixOS/nixpkgs#85924), but once that is fixed when we can track Nixpkgs `master` more freely. * We no longer need to keep track of separate Nixpkgs revisions for static vs. non-static builds The two builds can also share dependencies, too. Only the top-level derivations differ now. The reason that this doesn't use `static-haskell-nix` directly is because of [`cabal2nix` evaluation failures](nh2/static-haskell-nix#73), so this change works by lifting just the minimum logic that we need from the `static-haskell-nix` repository until we can finally use that repository (or Nixpkgs directly when it's upstreamed).

expipiplus1 · 2020-11-08T07:48:26Z

Is it feasible to add pkgsMusl.haskell.compiler.ghc* to hydra to prevent this from happening?

expipiplus1 · 2020-11-08T09:36:06Z

FWIW this also happens with the ghc-8.10.2 binary used to bootstrap ghc901:

$ nb -A pkgsMusl.haskell.compiler.ghc901
...
Inconsistency detected by ld.so: dl-lookup.c: 105: check_match: Assertion `version->filename == NULL || ! _dl_name_match_p (version->filename, map)' failed!
make[1]: *** [ghc.mk:973: install_packages] Error 127
make: *** [Makefile:51: install] Error 2
builder for '/nix/store/d6a8i0q7yiv97rsi9yz38p7f8ykhn7c6-ghc-8.10.2-binary.drv' failed with exit code 2
cannot build derivation '/nix/store/z3kk34qqqnncsc132v899yscxanfrpb6-ghc-9.0.0.20200925.drv': 1 dependencies couldn't be built
error: build of '/nix/store/z3kk34qqqnncsc132v899yscxanfrpb6-ghc-9.0.0.20200925.drv' failed

expipiplus1 · 2020-11-08T12:46:55Z

I built things with @bgamari's patch above and it all went very smoothly, is there a reason against including it in nixpkgs, even if just for musl builds?

monacoremo added the 0.kind: bug Something is broken label Apr 24, 2020

monacoremo mentioned this issue Apr 24, 2020

PostgREST static build nh2/static-haskell-nix#92

Closed

6 tasks

cdepillabout added the 6.topic: haskell label Apr 24, 2020

veprbl added the 6.topic: static label Apr 25, 2020

monacoremo mentioned this issue Apr 25, 2020

Add static build with Nix (fixes #1295) PostgREST/postgrest#1494

Merged

2 tasks

This was referenced May 20, 2020

Build failure on pkgsMusl.ghc865Binary breaking all GHC versions #88160

Closed

Possible to have pre-built binaries tweag/ormolu#543

Closed

bgamari mentioned this issue Jun 7, 2020

ncurses-5's symlink approach to providing libtinfo.so is broken #89769

Open

nh2 mentioned this issue Jul 2, 2020

Fully static Haskell executables - overview issue #43795

Open

60 tasks

nh2 mentioned this issue Jul 27, 2020

Support for GHC 8.10.1 nh2/static-haskell-nix#90

Closed

nh2 added a commit to nh2/static-haskell-nix that referenced this issue Jul 27, 2020

nixpkgs: Update submodule to include workaround for NixOS/nixpkgs#85924

3a0521e

nh2 mentioned this issue Jul 29, 2020

Tracking issue for GHC bootstrapping compilers being broken with musl in nixpkgs nh2/static-haskell-nix#99

Closed

Gabriella439 mentioned this issue Aug 16, 2020

Update static Nix build dhall-lang/dhall-haskell#2001

Merged

monacoremo mentioned this issue Oct 6, 2020

GHC 8.10 compatibility nh2/static-haskell-nix#98

Merged

5 tasks

expipiplus1 mentioned this issue Nov 9, 2020

Fix GHC bootstrap in pkgsMusl and include patch for binutils/16177 #103183

Merged

10 tasks

nomeata mentioned this issue Nov 11, 2020

Static cross-compilation: requires dynamic R_X86_64_32 reloc against '__TMC_END__' which may overflow at runtime; recompile with -fPIC input-output-hk/haskell.nix#914

Closed

FRidh closed this as completed in 15b3bc3 Dec 6, 2020

nh2 mentioned this issue Jul 4, 2021

GHC does not build with musl (ZuriHac 2021 edition) #129247

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pkgsMusl.haskell.compiler.ghc883 is broken (bootstrap with ghc865-binary fails) #85924

pkgsMusl.haskell.compiler.ghc883 is broken (bootstrap with ghc865-binary fails) #85924

monacoremo commented Apr 24, 2020

cdepillabout commented Apr 24, 2020

nh2 commented Apr 24, 2020

peti commented Apr 24, 2020

monacoremo commented Apr 25, 2020 •

edited

Loading

nh2 commented Jun 7, 2020

bgamari commented Jun 7, 2020

bgamari commented Jun 7, 2020

nh2 commented Jun 7, 2020

bgamari commented Jun 7, 2020

bgamari commented Jun 7, 2020

expipiplus1 commented Nov 8, 2020

expipiplus1 commented Nov 8, 2020

expipiplus1 commented Nov 8, 2020

pkgsMusl.haskell.compiler.ghc883 is broken (bootstrap with ghc865-binary fails) #85924

pkgsMusl.haskell.compiler.ghc883 is broken (bootstrap with ghc865-binary fails) #85924

Comments

monacoremo commented Apr 24, 2020

cdepillabout commented Apr 24, 2020

nh2 commented Apr 24, 2020

peti commented Apr 24, 2020

monacoremo commented Apr 25, 2020 • edited Loading

nh2 commented Jun 7, 2020

bgamari commented Jun 7, 2020

bgamari commented Jun 7, 2020

nh2 commented Jun 7, 2020

bgamari commented Jun 7, 2020

bgamari commented Jun 7, 2020

expipiplus1 commented Nov 8, 2020

expipiplus1 commented Nov 8, 2020

expipiplus1 commented Nov 8, 2020

monacoremo commented Apr 25, 2020 •

edited

Loading