Pass `ld-options` through to GHC #5451

nh2 · 2018-07-20T23:11:46Z

Tentative. Fixes #4925

Based on #5446; that one should be merged first and then this one rebased.

To review, only review the top commits (this change is much smaller than it looks here).

Please include the following checklist in your PR:

Patches conform to the coding conventions.
Any changes that could be relevant to users have been recorded in the changelog.
The documentation has been updated, if necessary.
If the change is docs-only, [ci skip] is used to avoid triggering the build bots.

Please also shortly describe how you tested your change. Bonus points for added tests!

I'll be testing it by using a cabal init mini project and checking that it "fails the right way" (linker errors) when --ld-option=-static is given.

nh2 · 2018-07-20T23:11:57Z

Please let me know if there are docs I should update for this.

nh2 · 2018-07-21T01:04:36Z

Hmm, this isn't working yet. The flags are passed to the linker, but they appear too early in the cc invocation; I intended them to appear at the very end but they appear before -o and before all the Haskell libraries.

Since with linker flags later -l flags fill in "holes" of symbols left by earlier -l flags, this isn't good yet.

I want them to appear at the end. Any hints?

Is this because of the position inside GHC flags as given in

cabal/Cabal/Distribution/Simple/Program/GHC.hs

Line 602 in d484bc4

, [ "-optl" ++ opt | opt <- ghcOptLinkOptions opts]

or is this a decision of GHC itself?

nh2 · 2018-07-21T01:10:42Z

Is this because of the position inside GHC flags as given in ... or is this a decision of GHC itself?

I tried permuting the flags directly in the GHC invocation cabal makes, but the results are the same. So it seems like ghc sticks -optl options at the very front of the linker invocation (as the 5th argument for me), before it passes -lHS... flags.

nh2 · 2018-07-21T01:23:59Z

I found a very hacky workaround: Pass -optl-Wl,--start-group to GHC first, without closing it with --end-group.

This makes the order unimportant: See https://linux.die.net/man/1/ld

--start-group archives --end-group
The archives should be a list of archive files. They may be either explicit file names, or -l options.

The specified archives are searched repeatedly until no new undefined references are created. Normally, an archive is searched only once in the order that it is specified on the command line. If a symbol in that archive is needed to resolve an undefined symbol referred to by an object in an archive that appears later on the command line, the linker would not be able to resolve that reference. By grouping the archives, they all be searched repeatedly until all possible references are resolved.

Using this option has a significant performance cost. It is best to use it only when there are unavoidable circular references between two or more archives.

I don't want to keep this as the final solution though, as it can make linking slower (and you lose the ability to specify which symbols exactly should be picked by exploiting the order; but this ability you lose anyway if GHC gives you only one location to put your symbols).

At least this workaround is independent from Cabal itself: It works with the current PR without modifications, only users would have to use it.

nh2 · 2018-07-21T01:38:31Z

Urgh, trouble. You cannot pass --ld-option=-Wl,--start-group to cabal configure:

Cabal will pass that flag directly to ld like this:

/nix/store/3a74d03pq6d0cg3crkfw965k5s58hbn4-binutils-2.30/bin/ar -r dist/build/objs-27195/libHSdarcs-2.14.1-ARuU0cypkJyGiHVcj2sKnE.a '@dist/build/objs-27195/ar27195-1.rsp'
/nix/store/3a74d03pq6d0cg3crkfw965k5s58hbn4-binutils-2.30/bin/ar: creating dist/build/objs-27195/libHSdarcs-2.14.1-ARuU0cypkJyGiHVcj2sKnE.a
/nix/store/pgsjpn6hp8cgcbrvvabk6vnza2i7l7sg-binutils-wrapper-2.30/bin/ld -r -o dist/build/HSdarcs-2.14.1-ARuU0cypkJyGiHVcj2sKnE.o '@dist/build/ld27195-2.rsp' -Wl,--start-group -lgssapi_krb5 -lcom_err -lkrb5support -lkrb5 -lkrb5support -lk5crypto -lssl -lcrypto -lnghttp2 -lssh2
/nix/store/3a74d03pq6d0cg3crkfw965k5s58hbn4-binutils-2.30/bin/ld: unrecognized option '-Wl,--start-group'

But you also can't pass --ld-option=--start-group, because then at a later stage Cabal will pass -optl--start-group to gcc (which it uses as the linker), which doesn't understand --start-group and wants -Wl,--start-group instead.

This problem is created by the fact that GHC uses gcc for linking instead of ld.

Why is that? From #ghc IRC:

geekosaur: because the C scaffolding is needed for libc to work, and it's not always clear how to set it up right; the C compiler knows how

nh2 · 2018-07-21T01:52:22Z

I have researched how the various linkers do it:

GNU `ld`

Implements --start-group and the workaround works conceptually (I have tried it with ghc directly).

GNU `gold`

Implements --start-group; I have not tried it.

LLVM `lld`

(See https://stackoverflow.com/questions/11893996/why-does-the-order-of-l-option-in-gcc-matter#comment89874319_11894098)

The lld linker of the LLVM project does not implement the POSIX behaviour (see R..'s comment):

Here is how LLD approaches the problem. Instead of memorizing only undefined symbols, we program LLD so that it memorizes all symbols. When it sees an undefined symbol that can be resolved by extracting an object file from an archive file it previously visited, it immediately extracts the file and link it. It is doable because LLD does not forget symbols it have seen in archive files.

nh2 · 2018-12-07T22:35:08Z

Is this PR good?

l29ah · 2018-12-17T08:12:26Z

How can i test it without messing up my working ghc installation?

nh2 · 2018-12-17T12:38:41Z

How can i test it without messing up my working ghc installation?

You should be able to test it by just cloning from git, checking out the branch, and building it with cabal or stack. You don't need to install it globally.

jkachmar · 2019-01-28T22:00:56Z

Is there anything that an interested passerby like myself could do to help get this merged?

Statically linked Haskell executables are something I'm very much interested, so I'd like to help unblock anything if it's at all within my capability 😄

sboosali · 2019-02-14T00:45:26Z

any update?

jkachmar · 2019-04-06T23:28:25Z

Ping. #5446 has been merged, so it would be really nice if this could be merged in as well.

nh2 · 2019-08-04T12:49:25Z

I've rebased (there was a formatting-to-80-line-limit change apparently).

This is ready to be merged from my side.

Also answering my own question from before:

I want them to appear at the end. Any hints?

Is this because of the position inside GHC flags as given in

cabal/Cabal/Distribution/Simple/Program/GHC.hs

Line 602 in d484bc4

, [ "-optl" ++ opt | opt <- ghcOptLinkOptions opts]

or is this a decision of GHC itself?

I found that this is a decision of GHC itself, not Cabal.

Using

  , ["-l" ++ lib     | lib <- ghcOptLinkLibs opts]
  , ["-L" ++ dir     | dir <- flags ghcOptLinkLibPath ]
  , [ "-optl" ++ opt | opt <- ghcOptLinkOptions opts]

instead of

  , [ "-optl" ++ opt | opt <- ghcOptLinkOptions opts]
  , ["-l" ++ lib     | lib <- ghcOptLinkLibs opts]
  , ["-L" ++ dir     | dir <- flags ghcOptLinkLibPath ]

doesn't help with the fact that the -optl flags appear before the

-lHStext-1.2.3.1
-lHSbinary-0.8.6.0
-lHScontainers-0.6.0.1

flags that GHC adds at the very end.

If we want to change that, we probably have to do that in GHC itself.

nh2 · 2019-08-04T13:02:17Z

Also I would very very very much like if this could be cherry-picked on 3.0 (where it applies cleanly), so that 8.8 will have a good story for static building.

Cabal/Distribution/Simple/GHC.hs

phadej · 2019-08-04T13:23:03Z

This fixes only GHC, what about GHCJS? Wouldn't similar problem we there too?

phadej · 2019-08-04T13:27:01Z

Also I would very very very much like if this could be cherry-picked on 3.0

This is completely @bgamari's call. This shouldn't break anything, but who knows...

I'd like to see similar change for GHCJS, and tests. Without those, I wouldn't merge this.

phadej · 2019-08-04T13:27:40Z

... i.e. IMHO 3.0 train has left the station.

jkachmar · 2019-08-04T14:43:25Z

I’d GHCJS supposed to be a first-class-citizen for Cabal support? I was under the impression that GHC was the primary support target for Cabal.

If this is the case it would more than make sense for this to be pulled into the 3.0 release train as this is a significant feature for anyone building “relocatable” Haskell executables.

nh2 · 2019-08-04T15:28:36Z

My goals with this are for statically linking Linux executables.

I would prefer to leave ghcjs related changes to those familiar with that toolchain; I'm not even sure to what program ld-options would be eventually given there in the end (probably some nodejs stuff, but certainly not a system linker), or what flags people might want to give there.

nh2 · 2019-08-08T23:17:57Z

Repeating a side investigation from a review question, so that it's visible here (quoting myself):

The programOverrideArgs for ldProgram is what's given with the --ld-options CLI flag to e.g. cabal build or cabal configure.

Interestingly they are duplicated 2x [...] I haven't figured out yet why that is, as that's a separate issue.

I have figured that out now. It happens for the new ld-options just as much as for the already-existing gcc-options. I've found the reason for that and file #6183 for it.

This addresses the last mystery I've encountered while implementing this.

nh2 · 2019-08-08T23:20:27Z

Ready to merge from my side.

Please remember that this PR adds only 2 lines of code, and the 2 are identical.

Correspondingly, this is a 1-line PR with pretty obvious effect that's been waiting for > 1 year.

It would be great to get it landed.

23Skidoo · 2019-08-09T00:24:52Z

Merged, thanks!

Pass `ld-options` through to GHC (cherry picked from commit 853414b)

23Skidoo · 2019-08-09T00:30:11Z

Also cherry-picked into 3.0.

23Skidoo · 2019-08-09T02:26:06Z

Cabal-3.0.0.0 has been released on Hackage with this change.

nh2 · 2019-08-09T23:36:30Z

Now that this is done, who should we ask that's familiar with the cabal parts of ghcjs whether a similar change would make sense for ghcjs?

phadej · 2019-08-10T16:56:47Z

Looks like GHCJS accepts -optl arguments, but doesn't really do anything with them.

Given some Main.hs,

% ghcjs -fforce-recomp -v -optl=BIG_CAPS_IS_EASY_TO_SPOT Main.hs
...
*** Deleting temp dirs:
Deleting: /tmp/ghc10471_0

and still produces Main.jsexe, where GHC barks

gcc: error: BIG_CAPS_IS_EASY_TO_SPOT: No such file or directory
...
Deleting: /tmp/ghc10650_0
`gcc' failed in phase `Linker'. (Exit code: 1)

There is however linking step in GHCJS compilation process too

link: linkables are ...
LinkableM (2019-08-10 16:54:34.309359487 UTC) Main
   [DotO lens-maybe.js_o]
Linking lens-maybe.jsexe (Main)

but I'm not sure if there's anything to configure, and whether it's just ghcjs-options then.

nh2 · 2019-08-17T15:48:20Z

... flags that GHC adds at the very end.

If we want to change that, we probably have to do that in GHC itself.

I've filed a GHC bug for that now: https://gitlab.haskell.org/ghc/ghc/issues/17071

nh2 · 2019-08-18T04:11:41Z

I've filed a GHC bug for that now

I've made a tentative PR for that now: https://gitlab.haskell.org/ghc/ghc/merge_requests/1589

23Skidoo · 2019-08-18T17:55:57Z

Now that this is done, who should we ask that's familiar with the cabal parts of ghcjs whether a similar change would make sense for ghcjs?

/cc @luite @JoshMeredith

luite · 2019-08-25T08:33:57Z

GHCJS has an internal linker and doesn't pass the options on to a separate program at the moment.

In general it ignores unsupported options. It should be safe to pass -optl arguments, but they are not really used yet.

Until now, giving `-optl` linker flags to `ghc` on the command line placed them in the wrong place in the `ld` command line: They were given before all the Haskell libararies, when they should appear after. Background: Most linkers like `ld.bfd` and `ld.gold`, but not the newer LLVM `lld`, work in a way where the order of `-l` flags given matters; earlier `-lmylib1` flags are supposed to create "holes" for linker symbols that are to be filled with later `lmylib2` flags that "fill the holes" for these symbols. As discovered in haskell/cabal#5451 (comment), the `-optl` flags appeared before e.g. the -lHStext-1.2.3.1 -lHSbinary-0.8.6.0 -lHScontainers-0.6.0.1 flags that GHC added at the very end. Haskell libraries typically depend on C libraries, so `-lHS*` flags will create holes for the C libraries to fill in, but that only works when those libraries' `-l` flags are given **after** the `-lHS*` flags; until now they were given before, which was wrong. This meant that Cabal's `--ld-options` flag and `ld-options` `.cabal` file field were pretty ineffective, unless you used the `--ld-option=--start-group` hack as (haskell/cabal#5451 (comment)) that convinces the classical linkers to not be dependent on the order of linker flags given. This commit fixes the problem by simply flipping the order, putting `-optl` flags at the end, after Haskell libraries. The code change is effectively only `args1 ++ args` -> `args ++ args1` but the commit also renames the variables for improved clarity. Simple way to test it: ghc --make Main.hs -fforce-recomp -v -optl-s on a `Main.hs` like: import qualified Data.Set as Set main = print $ Set.fromList "hello"

nh2 · 2019-09-20T12:42:53Z

I've made a tentative PR for that now: https://gitlab.haskell.org/ghc/ghc/merge_requests/1589

My GHC PR was merged, so 8.10 should fix the remaining problems here.

This was referenced Jul 20, 2018

ld-options is not taken into account when passed by command line or cabal.project.local #4925

Closed

Fully static Haskell executables - overview issue NixOS/nixpkgs#43795

Open

nh2 force-pushed the issue-4925-pass-ld-options-to-ghc branch from 6840cdf to eeb4aeb Compare October 16, 2018 21:28

Profpatsch mentioned this pull request Jan 21, 2019

Fully static binaries tweag/rules_haskell#379

Closed

jneira mentioned this pull request Apr 7, 2019

How to link a static lib (zlib) using cli options instead extra-libraries #5805

Closed

nh2 force-pushed the issue-4925-pass-ld-options-to-ghc branch from eeb4aeb to 1916762 Compare August 4, 2019 11:32

nh2 added 2 commits August 4, 2019 14:05

Pass ld-options through to GHC. Fixes haskell#4925

7e0a7b8

Changelog: Add entry for ld-options being passed to ghc

26f5b34

nh2 force-pushed the issue-4925-pass-ld-options-to-ghc branch from 1916762 to 26f5b34 Compare August 4, 2019 12:05

phadej reviewed Aug 4, 2019

View reviewed changes

Cabal/Distribution/Simple/GHC.hs Show resolved Hide resolved

23Skidoo merged commit 853414b into haskell:master Aug 9, 2019

23Skidoo added a commit that referenced this pull request Aug 9, 2019

Merge pull request #5451 from nh2/issue-4925-pass-ld-options-to-ghc

af7de53

Pass `ld-options` through to GHC (cherry picked from commit 853414b)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pass `ld-options` through to GHC #5451

Pass `ld-options` through to GHC #5451

nh2 commented Jul 20, 2018 •

edited

Loading

nh2 commented Jul 20, 2018

nh2 commented Jul 21, 2018 •

edited

Loading

nh2 commented Jul 21, 2018

nh2 commented Jul 21, 2018 •

edited

Loading

nh2 commented Jul 21, 2018

nh2 commented Jul 21, 2018

nh2 commented Dec 7, 2018

l29ah commented Dec 17, 2018

nh2 commented Dec 17, 2018

jkachmar commented Jan 28, 2019

sboosali commented Feb 14, 2019

jkachmar commented Apr 6, 2019

nh2 commented Aug 4, 2019 •

edited

Loading

nh2 commented Aug 4, 2019

phadej commented Aug 4, 2019

phadej commented Aug 4, 2019

phadej commented Aug 4, 2019

jkachmar commented Aug 4, 2019

nh2 commented Aug 4, 2019

nh2 commented Aug 8, 2019

nh2 commented Aug 8, 2019

23Skidoo commented Aug 9, 2019

23Skidoo commented Aug 9, 2019 •

edited

Loading

23Skidoo commented Aug 9, 2019

nh2 commented Aug 9, 2019

phadej commented Aug 10, 2019

nh2 commented Aug 17, 2019

nh2 commented Aug 18, 2019

23Skidoo commented Aug 18, 2019

luite commented Aug 25, 2019

nh2 commented Sep 20, 2019

Pass ld-options through to GHC #5451

Pass ld-options through to GHC #5451

Conversation

nh2 commented Jul 20, 2018 • edited Loading

nh2 commented Jul 20, 2018

nh2 commented Jul 21, 2018 • edited Loading

nh2 commented Jul 21, 2018

nh2 commented Jul 21, 2018 • edited Loading

nh2 commented Jul 21, 2018

nh2 commented Jul 21, 2018

GNU ld

GNU gold

LLVM lld

nh2 commented Dec 7, 2018

l29ah commented Dec 17, 2018

nh2 commented Dec 17, 2018

jkachmar commented Jan 28, 2019

sboosali commented Feb 14, 2019

jkachmar commented Apr 6, 2019

nh2 commented Aug 4, 2019 • edited Loading

nh2 commented Aug 4, 2019

phadej commented Aug 4, 2019

phadej commented Aug 4, 2019

phadej commented Aug 4, 2019

jkachmar commented Aug 4, 2019

nh2 commented Aug 4, 2019

nh2 commented Aug 8, 2019

nh2 commented Aug 8, 2019

23Skidoo commented Aug 9, 2019

23Skidoo commented Aug 9, 2019 • edited Loading

23Skidoo commented Aug 9, 2019

nh2 commented Aug 9, 2019

phadej commented Aug 10, 2019

nh2 commented Aug 17, 2019

nh2 commented Aug 18, 2019

23Skidoo commented Aug 18, 2019

luite commented Aug 25, 2019

nh2 commented Sep 20, 2019

Pass `ld-options` through to GHC #5451

Pass `ld-options` through to GHC #5451

nh2 commented Jul 20, 2018 •

edited

Loading

nh2 commented Jul 21, 2018 •

edited

Loading

nh2 commented Jul 21, 2018 •

edited

Loading

GNU `ld`

GNU `gold`

LLVM `lld`

nh2 commented Aug 4, 2019 •

edited

Loading

23Skidoo commented Aug 9, 2019 •

edited

Loading