ghc-{8.2.2,8.4.2}: fixups mostly for 2-stage cross, musl

[dtzWill]

• grab patches from gentoo to fix various 2-stage cross build system
  bugs
• explicitly set CrossCompilePrefix to the expected targetPrefix
  -- ensures everything has expected name and location
  -- fixes lack of prefix'ing when doing glibc -> musl "cross"
• Stage1Only: only set if doing "true" cross
• don't try to specify include/lib dirs for ncurses on cross
  -- only used by terminfo which actually removed the include option,
  and the lib option doesn't seem to do anything other than
  confuse the situation re:cross.

Fixes #37522

• Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Fits CONTRIBUTING.md.

---

Tested native x86_64, aarch64, cross-to-musl on those platforms,
and 'hello' :).

[dtzWill]

cc @Ericson2314 @bgamari

[dtzWill]

Since I put this together for someone on IRC, here's a command for turning hours of CPU time into a musl-based haskell "Hello World":

$ NIX_PATH=nixpkgs=https://github.com/dtzWill/nixpkgs/archive/feature/ghc-cross-musl.tar.gz nix-build '<nixpkgs>' --arg crossSystem '{config="x86_64-unknown-linux-musl";}' -A haskell.packages.ghc841.hello 

😁

[Ericson2314]

CC @angerman

[angerman]

The BuildFlavour fix is not working because it misses the crucial import. That's part of the fixes in #37254. However, I still believe we might actually rather want to do

sed -E "s/^#BuildFlavour[ ]+= ${BuildFlavour}\$/BuildFlavour = ${BuildFlavour}/" < mk/build.mk.sample > mk/build.mk

That in turn should also set the Stage1Only values right.

[Ericson2314]

If you two (and @bgamari too, maybe) could collaborate, that would be great!

[Ericson2314]

Historically, I've been able to cross compile fine without the gentoo patches, but I'll look through them to see what they offer.

[Ericson2314]

Per haskell/cabal#4941 (comment) @23Skidoo would be interested in us producing official cabal-install binaries statically linked with musl. LMK what I can do to help with these PRs.

[nh2]

Since I put this together for someone on IRC, here's a command for turning hours of CPU time into a musl-based haskell "Hello World":

@dtzWill Fantastic, this worked!

I have uploaded a built nix closure of the GHC built in that process here, so that people who don't want to wait hours to compile GHC can use my binary build:

https://github.com/nh2/nixpkgs/releases/tag/ghc-cross-musl-works

[nh2]

The stuff in here allowed me to build some fully static Haskell executables:

https://github.com/nh2/static-haskell-nix

[nh2]

@dtzWill Which of these branches is more up-to-date / should I use?

• https://github.com/dtzWill/nixpkgs/commits/ghc-cross-musl-works
• https://github.com/dtzWill/nixpkgs/commits/feature/ghc-cross-musl

[dtzWill]

@dtzWill Which of these branches is more up-to-date / should I use?

haha I honestly don't know. :/ sorry.

There were other deeper problems that made cross-compiling (musl or no) with GHC problematic,
I think the best solution for static builds would be to build musl-native (localSystem instead of crossSystem),
which requires having a "bootstrap" GHC that's also musl-native.
I was hoping to get this from these cross efforts, but I'm not sure how to create such a "bootstrap" package appropriately (or at all).

Or maybe cross-compilation with GHC works better now than it did, not sure.
Still building "native" seems less likely to be problematic if we can clear the bootstrap hurdle :).

[Ericson2314]

Ah I finally understand this now. This stuff is for 2-stage cross, which wasn't attempted before. It's a somewhat odd use-case, but seems at least harmless to everything else. I'm happy merging this if you all want; just got rid of conflicts.

[Ericson2314]

Actually @angerman I think recently submitted a better version of the last patch. I'm going to gate them all on the two-stage case.

[nh2]

@Ericson2314 The rebase seems to make GHC 8.2.2 unhappy: #37522 (comment)

[dtzWill]

@Ericson2314 thanks! What do you mean "2-stage cross", and why is it odd? (not challenging, just .. I don't remember this at all and am curious!)

[Ericson2314]

Normally GHC just builds a single cross compiler and that's this. Here GHC builds a cross compiler which builds a native compiler, a process that is only possible because the target platform is very close to the build platform (i.e. can run musl binaries on linux with glibc for most things). General compilation is limited to a single-stage build.

[angerman]

@Ericson2314, which last patch are you referring to?

[Ericson2314]

@angerman https://raw.githubusercontent.com/gentoo/gentoo/08a41d2dff99645af6ac5a7bb4774f5f193b6f20/dev-lang/ghc/files/ghc-8.2.1_rc1-hp2ps-cross.patch

[angerman]

I don’t think i’ve Done anything to hp2ps or unlit. Only to hsc2hs. I’m afraid you must be misremembering, @Ericson2314.

[nh2]

With the version before the rebase, I managed to build a Haskell package with lots of dependencies (stack) fully statically: https://github.com/nh2/static-haskell-nix/tree/master/static-stack

Now we just have to get it working on the latest nixpkgs, and figure out the haddock problems (see here).

[Ericson2314]

@angerman Ah I was thinking of haskell/hsc2hs@738f366, which was inspired by those patches.

[dtzWill]

(!!!)

~~it's happening!!~~

[Ericson2314]

Oh we need 8.4.3 now too.

[dtzWill]

FWIW build path for me seems to go through sqlalchemy, which is currently failing to build (failed test) w/musl but is fixed by bumping its version, see #42180 . At least ran into this trying to build 8.2.2.

[dtzWill]

@Ericson2314 any idea what the problem with 8.4.3 is? It builds (and seems to work when I manually invoke it on a trivial program), see #42455 (comment) for a bit more details.

Looks like the output doesn't have the right prefixes-- trying a quick fix that I don't expect to work but might do the trick... otherwise thoughts/suggestions are welcome :).