haskell-generic-builder: disable static PIE

[rnhmjoj]

Motivation for this change

This fix the error

ld: -r and -pie may not be used together

when building static haskell packages using the pkgsStatic overlay.

Things done

• Tested using sandboxing (nix.useSandbox on NixOS)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via pkgsStatic.haskellPackages.random
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

[cdepillabout]

Can you turn ghc/ghc#19580 into an actual link to the GHC issue tracker?

---

Also, I don't quite understand the combination of the comment here with the code.

The comment says: Static libraries fails to build if Nix forcibly adds -pie to the linker flags. But then the code does appear to add pie to hardeningDisable?

[rnhmjoj]

It seems that by default Nix adds -pie to the linker flags, while GHC adds -r -no-pie which have the opposite meaning and so the linking fails. Adding hardeningDisable = [ "pie" ] prevent Nix from adding -pie: the option has a negative meaning.

[rnhmjoj]

The comment should be a bit clearer, now.

[cdepillabout]

@rnhmjoj Could you show an example of using this PR to build a Haskell package statically?

[rnhmjoj]

@rnhmjoj Could you show an example of using this PR to build a Haskell package statically?

Simply try nix-build -A pkgsStatic.haskellPackages.random: it should fail with the error I mentioned above.
I'm not sure what's causing the bug to appear, because not every haskell package is affected, but most are, though.

[cdepillabout]

nix-build -A pkgsStatic.haskellPackages.random

Ah, thanks, I didn't realize it was this easy.

I tried this on the current haskell-updates branch (at commit 00cca09):

$ nix-build -A pkgsStatic.haskellPackages.random
...
Preprocessing library for splitmix-0.1.0.3..
Building library for splitmix-0.1.0.3..
[1 of 4] Compiling Data.Bits.Compat ( src-compat/Data/Bits/Compat.hs, dist/build/Data/Bits/Compat.o )
[2 of 4] Compiling System.Random.SplitMix.Init ( src/System/Random/SplitMix/Init.hs, dist/build/System/Random/SplitMix/Init.o )
/nix/store/xwdw8df7hv4xgkvzjjkk1d26pg8fp8zd-x86_64-unknown-linux-musl-binutils-2.35.1/bin/x86_64-unknown-linux-musl-ld: -r and -pie may not be used together
`x86_64-unknown-linux-musl-ld' failed in phase `Merge objects'. (Exit code: 1)
builder for '/nix/store/38p7f7wcij9mkkcxm46wqgixw2smii4q-splitmix-0.1.0.3-x86_64-unknown-linux-musl.drv' failed with exit code 1
cannot build derivation '/nix/store/pckzibrv6lxbgvz37ryi99vvjxyi4dqm-random-1.2.0-x86_64-unknown-linux-musl.drv': 1 dependencies couldn't be built
error: build of '/nix/store/pckzibrv6lxbgvz37ryi99vvjxyi4dqm-random-1.2.0-x86_64-unknown-linux-musl.drv' failed

I appeared to get the same error you were seeing.

I then applied the patch from this PR, and re-ran the above Nix command:

$ nix-build -A pkgsStatic.haskellPackages.random
...
/nix/store/fvdzhyccg3ikgvb8r82lxch9xbvgb7lf-random-1.2.0-x86_64-unknown-linux-musl

Looks like this worked, and it is linked statically!

$ ls /nix/store/fvdzhyccg3ikgvb8r82lxch9xbvgb7lf-random-1.2.0-x86_64-unknown-linux-musl/lib/ghc-8.10.4/x86_64-linux-ghc-8.10.4/random-1.2.0-6zIdUIrmHInBiqbyqpDYsu/*.a
libHSrandom-1.2.0-6zIdUIrmHInBiqbyqpDYsu.a
libHSrandom-1.2.0-6zIdUIrmHInBiqbyqpDYsu_p.a

[cdepillabout]

I cherry-picked this into #123682 as 830ef64.

You can watch the evaluation on Hydra if you're interested: https://hydra.nixos.org/eval/1671982. Note that this also includes merging master into haskell-updates, so it is possible that build failure may be from that as well. I imagine it is unlikely that any build failures would come from this static-linking fix.

Thanks for looking into this!

[rnhmjoj]

Thank you!

[Mic92]

I am also looking into static-pie right now. It should be possible: #123989

[cdepillabout]

@rnhmjoj

In the Haskell infrastructure, we have a file that defines a bunch of packages that are built by Hydra for the haskell-updates branch:

https://github.com/NixOS/nixpkgs/blob/d45bcb549ec7146148ea498a93178f48b9a1cfd6/pkgs/top-level/release-haskell.nix

If you look on Hydra you can see the builds for this jobset: https://hydra.nixos.org/jobset/nixpkgs/haskell-updates

We've been debating adding some sort of pkgsStatic.haskellPackages.FOO job to make sure that statically linking Haskell executables works at least somewhat. Would you be willing to be added as a maintainer for a job like this?

If you're a maintainer, you'll be pinged on PRs like the above when the job breaks. For example: #123682 (comment). Here, you can see that the taffybar derivation is broken, and rvl has been pinged to fix it.

Is this something you'd be willing to do?

(Our guess is that static linking probably won't fail that often, so in practice this probably wouldn't require too much work.)

---

Anyone else is of course welcome to volunteer to be a maintainer for this as well.

cc @maralorn @sternenseemann @nh2

[rnhmjoj]

I am also looking into static-pie right now. It should be possible: #123989

@Mic92 In general yes, but GHC can't produce this kind of executables, yet.

We've been debating adding some sort of pkgsStatic.haskellPackages.FOO job to make sure that statically linking Haskell executables works at least somewhat. Would you be willing to be added as a maintainer for a job like this?

@cdepillabout I can't guarantee I'll be able to fix issues in a timely manner, but I'm trying to keep the static builds infrastructure working: I'm distributing static binaries for a couple of haskell programs I wrote, so whenever I make a new release I have to do this anyway. If this is ok for you then yes, you can add me as a maintainer.

@nh2 is also maintaining static builds, though I think their approach is to use pkgsMusl rather than pkgsStatic due to issues like TemplateHaskell.

[sternenseemann]

@nh2 is also maintaining static builds, though I think their approach is to use pkgsMusl rather than pkgsStatic due to issues like TemplateHaskell.

How does that work pkgsMusl is very similar to pkgsStatic — except it doesn't link statically.

Also if musl is ever a problem, we have glibc.static by now, something to investigate as well at some point.

[rnhmjoj]

How does that work pkgsMusl is very similar to pkgsStatic — except it doesn't link statically.

Correct, static linking is then enable manually. You can see an example here: https://github.com/nh2/static-haskell-nix/blob/master/default.nix

Also if musl is ever a problem, we have glibc.static by now, something to investigate as well at some point.

Indeed, IIRC musl was chosen for pkgsStatic only because glibc was hard to statically link.

[nh2]

@nh2 is also maintaining static builds, though I think their approach is to use pkgsMusl rather than pkgsStatic due to issues like TemplateHaskell.

That is correct, TH execution loads .so files by default so we need to have both .so and .a files around. pkgsStatic disables .so files where it can.

Indeed, IIRC musl was chosen for pkgsStatic only because glibc was hard to statically link.

Yes, glibc maintainers do not really like / care about static linking, and it shows in some places, for example networking functionality.

[sternenseemann]

Yes, glibc maintainers do not really like / care about static linking, and it shows in some places, for example networking functionality.

We have glibc.static in nixpkgs nowadays, although it does not seem to see much use. Changing pkgsStatic to use that by default could be an interesting experiment.