statically linked GHC libraries on darwin still depend on haskell libraries #21200
Assignees
Labels
Comments
|
As a note, this causes ghc bug 12479 where the OS X sierra linker no longer accepts mach o binaries with greater than 32k of linked libraries to run: If you run otool on the binary and add up the path lengths for each library, it adds up to the 40960 length. Its likely this will also affect any other haskell binary on sierra that uses a lot of libraries. |
|
Use ghc802 and it should work :) |
|
This issue might be caused by the fact that the generic Haskell builder cannot use |
|
@peti good call. GHC 8.0.2 does fix split-obj for darwin again, so I'll give that a go. |
|
Just for reference: the relevant bit in the generic builder is at https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/generic-builder.nix#L23. |
|
Compiling the GHCs chain to test this. Will report once it's done. |
|
I doubt split objs has anything to do with the rpaths/library search paths in the stack binary to be honest. The 8.0.1 compiled stack on sierra yields me: I'm about 99.9% sure its due to enableSharedExecutables evaluating to true. I hard set it to false and got this: Vs So I would presume either cabal is getting -dynamic or -optl=-shared passed in by default here: |
No, split-objs has nothing to do with rpaths compiled into the stack binary. It it does have an effect on the store paths that Nix might consider a run-time dependency (as opposed to a build-time dependency). |
|
Looks to be !isCross evaluating to true from this commit, reverting the enableSharedExecutables line to before restores static haskell libraries being linked into executables. |
|
@mitchty I'm not following, which line exactly? |
|
http://hydra.nixos.org/job/nixpkgs/trunk/stack.x86_64-darwin#tabs-charts says it always had 2.5TB of closure size. |
|
@domenkozar 64ec4dd#diff-360bed088faac7580cee137eeffb39e0L20 Presumably all the other lines with isCross are also evaluating not as expected in this case, but that one alone gets me binaries with ghc libraries statically linked. Stack fails to build with it fixed, but other things, ghc-mod as my prior example, build as expected by forcing it to false or removing !isCross. The stack build error with it set to false (only thing that seems to fail to build I have hit yet): The libghc dylib name looks suspicious for 8.0.1, so have to negate its not my setup somehow. |
|
@mitchty that's a bug in GHC with Sierra, you have to use 8.0.2-rc2: The issue with 8.0.2 is that packages won't compile yet :) |
|
I have the following patch but it fails with |
|
|
Yeah I can't actually get stack to build/run on sierra at all. I've given up and just installed it from their release page to get stuff done. |
|
This might be the clue: |
|
Did some research and clang has different ways of stripping dead code: results into: @copumpkin do we really need following at runtime? |
|
|
Seems like we should just depend on:
|
|
I'll add my fixes to #21589 which brings closure down to ~700MB, but if we want stack closure to be down to ~150MB on darwin we'll have to wait for #15718 (comment) |
No, I'm really sad that OSXPrivateSDK is used anywhere and I don't know how it got introduced. A huge chunk of my stdenv work was to avoid bringing that in. Also, no clue why it would be a runtime dependency, but that makes even less sense than using it in the first place 😄 More broadly, how does this tie to the SDK being too old? Not saying it doesn't, but I don't follow the argument that it does. |
|
Dunno, I was just asking if upgrading would make it better :) |
|
Oh, I doubt it. I'd just use that new utility function to strip all references to OSXPrivateSDK for now and file an issue that it's being used. Is that the last big part of the closure? I'll need to track down where and how it appeared. |
|
Back to the issue, this is clearly a bug somewhere. Same problem with The whole closure is 3GB, including haskell dependencies. It's linked correctly, as expected, just system libs. Let's inspect Nix runtime graph: Aha, somehow it still depends on Binary references the I tried using clang dead code elimination, but it fails to work: |
domenkozar
changed the title
This was referenced Mar 22, 2017
|
I mentioned this on IRC but the warp references are all inside the |
|
@Ericson2314 is this somewhat familiar with you? :) |
|
I don't know the details, but static linking + data dirs causing problems is not at all surprising to me. |
|
Still happening with latest purescript: It does seem like it's part of some data file. |
|
where would be the best place to put an override for purescript?, its currently defined in hackage-packages.nix which is 7.8mb and auto-generated? this override is enough to remove GHC from the closure: https://gist.github.com/cleverca22/af72df113ec7c49ac2377753db819c1a#file-gistfile1-nix-L15-L19 further investigation points towards configuration-common.nix |
I sometimes add these kind of "clean up overrides" for end-user packages into |
|
So on darwin the major culprit are things like yesodweb/wai@3e1ffd1 and we're back to #4504 (comment) |
7 tasks
|
stack closure size: http://hydra.nixos.org/job/nixpkgs/trunk/stack.x86_64-darwin#tabs-charts |
|
cabal-install closure should be addressed in 56be4be |
|
I'll close this catch-all issue, summary:
|
|
This still seems to happen (with |
|
|
|
Oops, didn't mean to reopen! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Linux:
Darwin:
Can something be done about it? I didn't see any notes around stack and executable sharing on darwin.
It's 2.7G that needs to be fetched for CI to build projects with stack.
cc @peti @copumpkin
The text was updated successfully, but these errors were encountered: