Add support for cross-compilation with Nix #44

georgefst · 2022-08-12T12:14:48Z

The aim here is to build fully statically-linked binaries for Linux and Windows. This will make distribution easier. I particularly have in mind Steam games which distribute Monpad (EDIT: this is a lot less important now with dump-html mode - Ewephoria uses that and doesn't ship Monpad at all).

Various potentially-useful links about building static Haskell binaries:

We should also look at the setups of projects that do this (GHCup, HLS, Dhall...). Also note that we can't legally link GMP statically. Fortunately support for native Haskell bignums was recently added to Haskell.nix.

Some of the commits here are hacks (partly to get Ewephoria) out of the door, and shouldn't be merged. I'll probably make a new branch once the main hurdles have been overcome.

Largely taken from https://input-output-hk.github.io/haskell.nix/tutorials/getting-started-flakes.html. Builds from x86 Linux, for x86 Musl Linux or Windows.

georgefst · 2022-08-12T12:21:03Z

flake.nix

@@ -0,0 +1,25 @@
+{
+  description = "monpad";


I had hoped to make this totally generic (remove ./haskell directory specifier, release flag etc.), and use it across other projects. Build.hs would then just pull it down when necessary from some repository (with an integrity check). But it seems that flake files have to be checked in, so that's a no-go.

Anyway, once we upload to Hackage (#20), the package will be picked up by Haskell.nix, all assets will be in the sdist, and we won't need to do anything special in Nix at all. Although perhaps it would be useful to retain the ability to deploy binaries without a corresponding Hackage version?

georgefst · 2022-08-12T12:41:58Z

flake.nix

+                src = ./haskell;
+                compiler-nix-name = "ghc924";
+                shell.tools = { cabal = { }; };
+                shell = { inherit crossPlatforms; };


If we want to support nix build, rather than just *-cabal within nix develop (more on that elsewhere), we need to add configureArgs = "-frelease"; here if we don't want the built monpad to rely on finding the rsc folder. This is annoying because:

It's duplication between this file and the Shake build script.

It doesn't work! Presumably this is something to do with the asset files not being tracked by Nix. But I thought that Haskell.nix would handle that for us. I'm sure I even read somewhere about it having special support specifically for file-embed. nix build .#monpad:exe:monpad fails with:
src/Embed.hs:32:27: error: • Exception when trying to run compile-time code: rsc/common.css: withBinaryFile: does not exist (No such file or direc> Code: (embedFile $ "rsc" </> "common.css") • In the untyped splice: $(embedFile $ "rsc" </> "common.css") | 32 | commonCSS () = GET_FILE("rsc" </> "common.css") | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

It doesn't work!

~~It does now! I think what changed is that I'm now working from a clean checkout. Somehow a dirty git working dir gets in the way.~~ EDIT: got too excited - as stated above this currently builds a version with no embedded files

Hacked a solution in 814b4cc, but I should probably make those dependencies tracked by Nix, or something. I also have a no-symlinks stash for Build.hs modifications (although I'm not sure that's the best approach either, given assets need to be known to Git and I don't want duplicates).

Taken from https://input-output-hk.github.io/haskell.nix/tutorials/cross-compilation.html#static-executables-with-musl-libc.

georgefst · 2022-08-12T12:53:31Z

README.md

+
+If we're on x86-64 Linux, we can use `nix develop` to enter a shell from which we can cross-compile:
+```sh
+~/.ghcup/bin/runghc Build.hs --target=x86_64-w64-mingw32


Weirdly, we have to remove the Windows target from crossPlatforms in order for plain ~/.ghcup/bin/runghc Build.hs to work, i.e. to build for the host platform. This seems like a Haskell.nix bug. It's not a huge issue though, as we don't have much reason to use Nix at all for such a build. It would be nice to get cached dependencies though.

This command actually fails quickly for TH reasons. It may be salvageable, but the fact it doesn't "just work" seems a limitation of the shell-based approach, as opposed to nix build.

If we just use nix build .#x86_64-w64-mingw32:monpad:exe:monpad, we would need to add some Nix config for compiling the non-Haskell parts of the project.

Perhaps it could just shell out to runghc Build.hs assets.

georgefst · 2022-08-12T12:56:45Z

README.md

+If we're on x86-64 Linux, we can use `nix develop` to enter a shell from which we can cross-compile:
+```sh
+~/.ghcup/bin/runghc Build.hs --target=x86_64-w64-mingw32
+~/.ghcup/bin/runghc Build.hs --target=x86_64-unknown-linux-musl


This works, but isn't actually useful without passing some extra flags, since ldd shows just as many dependencies as the GCC version.

Again, this is a drawback of the shell-based approach.

On the other hand nix build .#x86_64-unknown-linux-musl:monpad:exe:monpad fails building libevdev, whereas interestingly the shell+cabal approach does not.

I assume the issue is getting hold of a statically-linked libevdev. I've asked about this on r/haskell.

If this is a real sticking point, we could somehow (separate branch, or Nix-based override) compile without --system-device support for now. It isn't actually necessary for my immediate use case. It's enough (though hacky) to replace the fields under os(linux) with just hs-source-dirs: windows.

It's possible that my Haskell evdev library is in some way not totally Musl compatible. I'd hope it would be, but it's untested. I recall the evdev-rs needing changes to explicitly support it. Otherwise maybe Haskell.nix just needs to be told about the native dependency.

We have static musl builds working now, after providing a static libevdev. cabal build doesn't produce a static exe because the magic is in nix build somewhere. Another killer for nix develop+cabal approach I originally desired.

georgefst · 2022-08-12T13:01:57Z

README.md

+~/.ghcup/bin/runghc Build.hs --target=x86_64-unknown-linux-musl
+~/.ghcup/bin/runghc Build.hs --target=aarch64-unknown-linux-gnu
+```
+


We could do with a section on deploying the resulting artefacts.

Ideally we'll manage to fully statically link everything, meaning we'll just have a single executable file to deploy.

If we are stuck with shared libraries, we'll need commands like:

TARGET_DIR=... sh -c 'rsync result/ $TARGET_DIR/ -a --copy-links && rsync dist/rsc/ $TARGET_DIR/bin/rsc/ -a --copy-links'

TARGET_NAME=... sh -c 'nix copy --to ssh://$TARGET_NAME .#aarch64-unknown-linux-gnu:monpad:exe:monpad && scp ./result/bin/monpad $TARGET_NAME:tmp/monpad-cross-nix'

We now have static Linux builds, but Windows needs a load of DLLs (weirdly this is an inversion of the usual non-Nix state of things). Can we compile statically with MinGW?

flake.nix

georgefst · 2022-08-12T13:24:23Z

README.md

+```sh
+~/.ghcup/bin/runghc Build.hs --target=x86_64-w64-mingw32
+~/.ghcup/bin/runghc Build.hs --target=x86_64-unknown-linux-musl
+~/.ghcup/bin/runghc Build.hs --target=aarch64-unknown-linux-gnu


I've removed aarch64-multiplatform from crossPlatforms because this doesn't actually work. It causes mysterious evaluation-time failures (nix develop or nix build).

georgefst · 2022-08-12T13:28:53Z

README.md

+~/.ghcup/bin/runghc Build.hs --target=x86_64-w64-mingw32
+~/.ghcup/bin/runghc Build.hs --target=x86_64-unknown-linux-musl
+~/.ghcup/bin/runghc Build.hs --target=aarch64-unknown-linux-gnu
+```


Using the system GHC here is pretty odd. Can we just use cabal run (or even ./Build.hs)?

georgefst · 2022-08-12T16:27:13Z

flake.nix

+                src = ./haskell;
+                compiler-nix-name = "ghc924";
+                shell.tools = { cabal = { }; };
+                shell = { inherit crossPlatforms; };


This crossPlatforms is only necessary for nix build, which we were hoping not to use (the other being for nix develop, which gives us access to the Shake build script).

But right now, that's working and the other isn't.

georgefst · 2022-08-22T23:57:44Z

flake.nix

+          haskellNix.overlay
+          (final: prev: prev.lib.optionalAttrs prev.stdenv.hostPlatform.isMusl {
+            libevdev = prev.libevdev.overrideAttrs (_: { dontDisableStatic = true; });
+          })


I should upstream this patch somehow, as initially suggested on Reddit.

georgefst · 2022-09-20T18:30:09Z

This looks interesting: https://gitlab.com/macaroni.dev/macaroni.nix.

It turns out files under `extra-source-files` are not found at compile time (`file-embed`) unless they're in `rsc` (and not as symlinks), and known to Git (staging makes a difference, which confused me for a while, e.g. in georgefst/hello-hs@fa41589).

This is primarily so that we can remove a dependency on the GPL-licensed libgmp. See https://github.com/input-output-hk/haskell.nix/pull/1784/commits.

georgefst · 2022-12-20T00:47:42Z

The commit "Work around Musl getAddrInfo issue" may be misplaced. Even on a native build on master, getAddrInfo returns Nothing on my main Linux machine, since getAddrInfo isn't returning any useful addresses (EDIT: nope - sometimes it does work, maybe when I'm connected directly to main router instead of an extender). I'm pretty sure this has broken in the past few months, and seems likely to be down to the OS rather than Haskell libraries or anything...

(prompted by seeing someone else using very similar code, and being curious about the slight differences)

Add basic generic Nix flake config

71c6555

Largely taken from https://input-output-hk.github.io/haskell.nix/tutorials/getting-started-flakes.html. Builds from x86 Linux, for x86 Musl Linux or Windows.

georgefst commented Aug 12, 2022

View reviewed changes

georgefst added 2 commits August 12, 2022 13:49

Add basic Nix instructions

3820899

Build fully statically with musl

2c205d4

Taken from https://input-output-hk.github.io/haskell.nix/tutorials/cross-compilation.html#static-executables-with-musl-libc.

georgefst commented Aug 12, 2022

View reviewed changes

Formatting

f3995bd

georgefst commented Aug 12, 2022

View reviewed changes

flake.nix Outdated Show resolved Hide resolved

georgefst commented Aug 12, 2022

View reviewed changes

georgefst added 2 commits August 23, 2022 00:47

Remove unnecessary Musl configure args

f32b194

Fix musl build by providing static libevdev

08fa95c

georgefst commented Aug 22, 2022

View reviewed changes

georgefst mentioned this pull request Sep 3, 2022

Nix packages broken georgefst/evdev#25

Closed

georgefst added 4 commits November 22, 2022 15:57

Work around Musl getAddrInfo issue

2725c68

Embed asset files in Nix build

814b4cc

It turns out files under `extra-source-files` are not found at compile time (`file-embed`) unless they're in `rsc` (and not as symlinks), and known to Git (staging makes a difference, which confused me for a while, e.g. in georgefst/hello-hs@fa41589).

Minor refactor

51ce5b4

Bump flake deps

fb64207

This is primarily so that we can remove a dependency on the GPL-licensed libgmp. See https://github.com/input-output-hk/haskell.nix/pull/1784/commits.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for cross-compilation with Nix #44

Add support for cross-compilation with Nix #44

georgefst commented Aug 12, 2022 •

edited

Loading

georgefst Aug 12, 2022

georgefst Aug 12, 2022

georgefst Aug 12, 2022

georgefst Aug 12, 2022 •

edited

Loading

georgefst Nov 28, 2022

georgefst Aug 12, 2022

georgefst Aug 12, 2022

georgefst Aug 12, 2022

georgefst Aug 12, 2022

georgefst Aug 12, 2022 •

edited

Loading

georgefst Aug 23, 2022

georgefst Aug 12, 2022

georgefst Aug 12, 2022 •

edited

Loading

georgefst Aug 23, 2022

georgefst Aug 12, 2022

georgefst Aug 12, 2022

georgefst Aug 12, 2022

georgefst Aug 22, 2022

georgefst commented Sep 20, 2022

georgefst commented Dec 20, 2022 •

edited

Loading

Add support for cross-compilation with Nix #44

Are you sure you want to change the base?

Add support for cross-compilation with Nix #44

Conversation

georgefst commented Aug 12, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

georgefst Aug 12, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

georgefst Aug 12, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

georgefst Aug 12, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

georgefst commented Sep 20, 2022

georgefst commented Dec 20, 2022 • edited Loading

georgefst commented Aug 12, 2022 •

edited

Loading

georgefst Aug 12, 2022 •

edited

Loading

georgefst Aug 12, 2022 •

edited

Loading

georgefst Aug 12, 2022 •

edited

Loading

georgefst commented Dec 20, 2022 •

edited

Loading