Fully static binaries #379
Comments
|
nixpkgs has made major progress in this territory, meta issue: NixOS/nixpkgs#43795 Here’s an example how dhall uses that feature: dhall-lang/dhall-haskell@3e305ac They add support to cabal to make this possible:
We don’t use cabal, but we can infer the GHC flags we need from these patches. |
|
Is there anything that interested parties like myself can help contribute here? Fully statically linked executables (in addition to everything else that's part of the |
|
It's worth noting that since #587 rules_haskell links binaries "mostly static" in the sense that both It seems that what's missing to achieve full parity with the cc rules is the |
|
Yes but a fully static mode would enable trivially deployable Haskell binaries, which is what I'm assuming @jkachmar is shooting for. I'd say the way to go about this is to enable |
|
Just so, @mboes. The added wrinkle is that it would also entail something like what @nh2 has done in the linked NixOS thread. Special care would need to be taken to address some of the requirements that statically linked Haskell applications bring (e.g. GHC w/ |
|
Right. |
Does |
|
Afaik only licensing (You can only link against LGPL dynamically from non-copyleft). |
That's a question I would have asked you. ;) |
|
@nh2 I'm not aware of any functional issues, I'm just concerned about it from the perspective of a commercial user not wanting to run afoul of any OSS licenses. I just raised it as a point here because I imagine some of this would need to be "baked in" to |
|
@jkachmar Makes total sense, I wasn't questioning your proposal. I just want to be very aware of anything that technically breaks under static linking, so that I don't link it in even in my fully open-source projects. Also note there's |
|
So, after a long hiatus, I've had some time to play with this, and I think I've got something working. At https://github.com/lunaris/minirepo I've got a minimal repository that sets up:
I've tested:
Tricks I've played:
I think this is "OK" (I'm not doing anything I shouldn't be) but fully suspect it isn't (I'm definitely doing something I shouldn't be). Take a look and let me know. It's definitely worth saying -- this is all thanks to the works of others (@nh2, DAML, Tweag) and I'm hugely excited by it. Thanks so much for getting us to this point! P.S. I'm aware that I'm still (I believe) using |
|
Very cool. I'll take a closer look at your repo and see if there is anything we can do upstream to make this nicer. |
|
Of course I spoke too soon -- now trying to get something meatier ( |
|
@lunaris That's cool!
Not sure if you've seen it already, but the use of my
Do you mean something like this? |
|
@nh2 Alas no -- I am using your |
I'd like to try reproducing this, do you have a commit with the error? |
|
Sure -- try https://github.com/lunaris/minirepo/tree/pg and then: $ bazel run //example-service:implYou should get something like: among the noise. |
|
@hanshoglund Not sure where you're at but from what I can tell:
Again, still not sure why GHC wants to see the dynamic libraries when being told |
|
This appears to be an issue with the toolchain (i.e. what is being pulled from Nix) as opposed to rules_haskell. At least, if I switch out the |
|
Doing some more investigation on this, it seems that the As it is for now, I'll see about putting together something in which we revert to using Nix for the Hackage package set, which is not ideal but would still be an improvement for us (or anyone who values static binaries more, perhaps). |
Indeed, that was an issue that was resolved recently, see #1130.
What issues are you encountering? In the past we had issues where pulling in I'm not terribly familiar with the |
|
@aherrmann Thanks for your reply -- I've tried a number of things but I don't think it's the issue you report. Bumping my Nixpkgs and rules_haskell appears to have made some things better -- Any thoughts? I suspect I'm doing something silly here. |
|
Yes, I was being silly -- in practice it's even more specific than that -- you only want |
|
Huzzah! I think it all works -- check out https://github.com/lunaris/minirepo/tree/b0c0ca440bcbb73e76afbbdb640b23c11cf28b00 In short, and documenting many pitfalls in the hope that Google indexing helps those who come later (links and comments in the repo too):
Now I have something approaching featureful, I plan to try and integrate it into our (much larger) repository and see how it goes. But perhaps this offers a route to fully-static binaries in |
|
This is wonderful news, thank you @lunaris for pushing this forward and documenting your progress and the required steps! I tried out It's very reassuring to see that this works with a recent but otherwise unpatched rules_haskell. Though, there seem to be quite a few pitfalls to set this up correctly. I think a good next step to improve support in rules_haskell would be to add setup instructions to the use-cases guide and to add an example to rules_haskell CI to ensure that this keeps working. |
|
For sure -- happy to help with documentation and the like. That said, I don't think we're quite there yet. Some more gotchas I've encountered since:
diff --git a/haskell/private/cc_wrapper.py.tpl b/haskell/private/cc_wrapper.py.tpl
index 7540ddf8..d188f04d 100644
--- a/haskell/private/cc_wrapper.py.tpl
+++ b/haskell/private/cc_wrapper.py.tpl
@@ -239,8 +239,9 @@ class Args:
if consumed:
# Remember the required libraries.
- self.libraries.append(library)
- out.append("-l{}".format(library))
+ if library != "ffi" and library != "gmp":
+ self.libraries.append(library)
+ out.append("-l{}".format(library))
which surely shouldn't happen if your GHC is using musl. This could be because my configuration of Any thoughts on all this welcome and as I said, as we get to the bottom of this for sure sign me up for documentation -- it's sorely needed and would love to pitch in. |
|
OK, so a less hacky way of fixing the I can't think of a condition which we'd check for this -- Of course this proposal in general might be horrible -- feedback welcome. Example diff to be clear: diff --git a/haskell/private/cc_wrapper.py.tpl b/haskell/private/cc_wrapper.py.tpl
index 7540ddf8..6e24d3eb 100644
--- a/haskell/private/cc_wrapper.py.tpl
+++ b/haskell/private/cc_wrapper.py.tpl
@@ -138,6 +138,19 @@ class Args:
# not speficy the required runpaths on the command-line in the
# context of Bazel.
self.rpaths.extend(self.library_paths)
+
+ # hsc2hs will call us (its configured C compiler) when generating
+ # binaries which output Haskell code. Such binaries end in the
+ # suffix "hsc_make". When we are building in a static configuration
+ # (as indicated by TODO_WHAT_DO_WE_CHECK), we want these binaries
+ # too to be statically linked, since the dynamic libraries they
+ # attempt to load won't be available at runtime when we're done
+ # building. It should be that everything we are linking together is
+ # statically linked (e.g. GHC-compiled Haskell libraries, etc.; the
+ # Haskell toolchain), and so we just need to pass `-static` to the
+ # real C compiler to produce the final binary.
+ if self.output.endswith("hsc_make"):
+ self.args.extend(["-static"])
else:
# We don't expect rpath arguments if not linking, however, just in
# case, forward them if we don't mean to modify them.( I will continue working on the other two points. |
|
What a journey this has been. Yet again, I think I'm "there" but I'm sure I'll be proven wrong in the coming days. Update for now for those following/landing here from a Google search. Disclaimer: all diffs/hacks are WIPs -- lots of cleaning up and design discussions needed before these become bonafide patches ContextYou need to end up at #970 and #728 at some point, which both talk about pieces of the puzzle somewhat separately. In short -- GHCi and Template Haskell are going to do their best to load dynamic libraries, and you need to pull several tricks to stop this happening. These two threads talk about several of them. Thanks so much to those who wrote them! I'm focussing on a GHC-from-Nixpkgs use case and (luckily) am only concerned with Linux right now. Bindists are likely still an open problem/I've no idea about them or Windows/macOS. In my scenario at least, there are broadly three "groups" of problems:
Bundled libraries from Nixpkgs
Note that here I'm overriding the GHC from @nh2's Libraries from Hackage/Stackage -- built with Cabal/
|
|
PR #1390 is a first stab at implementing all these things cleanly. It's just the bare bones so we can discuss whether or not we want to support all these hacks/this use case; if it's a goer we can look at tests and docs (though I've tried to make a start with some fairly heavy comments). |
|
I've also updated https://github.com/lunaris/minirepo with a bigger set of working examples that tests GHCi, Template Haskell, GHCIDE support, Haddocks, etc. Additionally, our large (several tens of binaries, hundreds of libraries, plenty of C-linking Hackage packages) work repository appears to all build just fine (and the executables and Docker images actually work) but it'll take a bit more time to make sure we've tested all the cases before it lands in production 😄 |
|
@lunaris Thanks a lot for documenting all the steps in detail! Sorry for the late reply, I've been meaning to look into this in more depth but haven't gotten around to it, yet. I can give comments/answers to some of the points raised. Building GHC with a static RTS has worked well in Regarding the Regarding the |
|
That is weird -- it does seem to work without the |
🎉 That's great news!
Yes, that sounds reasonable. I'm actually surprised that we don't need this in
This might also be a good thing to integrate into the |
|
I will give the |
Thanks for the pointer. That doesn't look right. With static linking each library archive only contains symbols introduced by that library, symbols from transitive dependencies are not pulled in. Only when statically linking the final executable will all required symbols be bundled together in that executable. I.e. all transitive static archive dependencies need to be available. (If this wasn't the case it would lead to lots of duplicate symbols and unnecessarily inflate the size of intermediate archives.) In the context of
That may well be by chance. The REPL is invoked via A more precise test would be to run |
|
Thanks for the pointer -- I'll run these tests and report back. That said, even if the artifacts were lying around on disk (believable), how would GHCi know to try and pick them up without appropriate |
|
Indeed, this is completely broken. The question then, is how to solve this error (from the example in Does GHCi work in DAML without any issues? I know DAML is using zlib; does it use other libraries? Note that I don't think this is |
If they are dependencies of some
Yes, GHCi works in DAML. There is a static I haven't reproduced the issue, yet (building GHC takes a while). But, I have a suspicion. GHC lists two search paths that seem relevant for ssl: The latter would only contain the dynamic library. The former would contain the static library, but that path is relative to the execroot and won't work relative to the repository root. However, |
|
Implemented in #1390 🎉 |
GHC by default creates "static" libraries. As explained in #378, this really means "mostly static" in Bazel terms, because some dependencies are still dynamically linked. It is possible, however, to create fully static binaries, provided there are static versions of all external libraries available. We should implement this link mode as well, and make it configurable per-target. We should probably stick to a similar API for this to what the CC rules already provide.
cc @lunaris
The text was updated successfully, but these errors were encountered: