Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding pkgsStatic: a fully static overlay #48803

Merged
merged 8 commits into from
Dec 5, 2018

Conversation

matthewbauer
Copy link
Member

@matthewbauer matthewbauer commented Oct 21, 2018

Motivation for this change

This uses the cross compilation infrastructure to build a statically linked Musl-based package set. Some basic things seem to build okay... Working on getting things working as much as possible.

You can access them from the pkgsStatic attribute set. Basic things like pkgsStatic.hello seem to work okay.

As part of the implementation this adds a new type of overlay called "crossOverlays". These are similar to ordinary overlays except for one critical difference. They only apply to the /target/ package set. To accomplish this, we use the cross stdenv to clearly separate out build, host, and target.

In addition, I have added a custom "static" overlay that will enable static by modifying the stdenv and modifying other packages as necessary. Additional packages can be enabled later on here.

/cc @nh2 @dtzWill

@GrahamcOfBorg GrahamcOfBorg added 6.topic: stdenv Standard environment 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels Oct 21, 2018
@dtzWill
Copy link
Member

dtzWill commented Oct 21, 2018 via email

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: ncurses, zlib

Partial log (click to expand)

/nix/store/iirmif7qgp7pgbv80z5x1sj3hbay893d-ncurses-6.1
/nix/store/di0d17idlw6fxf3qwpfyxciw94ywi664-zlib-1.2.11

@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: ncurses, zlib

Partial log (click to expand)

/nix/store/snmcl9nfjj49h2j0s1qpvmsajhm91141-ncurses-6.1
/nix/store/v8w9k5zzfidw4ik5rjxbpf9cfvbrj9s3-zlib-1.2.11

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: ncurses, zlib

Partial log (click to expand)

/nix/store/wydzkhfm25cgbr1nfq28nqvgk0mk7518-ncurses-6.1
/nix/store/x0aj2fsd3hskn1fckk3w4b8bl6mx8m1s-zlib-1.2.11

@nh2
Copy link
Contributor

nh2 commented Oct 25, 2018

@matthewbauer @dtzWill Are any of you at NixCon by any chance?

@dtzWill
Copy link
Member

dtzWill commented Oct 25, 2018 via email

@matthewbauer
Copy link
Member Author

Nope but @Ericson2314 should be

@Ericson2314
Copy link
Member

Ericson2314 commented Oct 26, 2018

Actually I won't be :(. I was too slow, and no tdpace left.

@Ericson2314
Copy link
Member

Maybe next year I'll finally learn not to procrastinate.

@@ -28,7 +29,7 @@ stdenv.mkDerivation rec {
setOutputFlags = false; # some aren't supported

configureFlags = [
"--with-shared"
(if enableShared then "--with-shared" else "--without-shared")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can use lib.withFeature for this

@GrahamcOfBorg GrahamcOfBorg added the 2.status: merge conflict This PR has merge conflicts with the target branch label Oct 26, 2018
@GrahamcOfBorg GrahamcOfBorg removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Oct 29, 2018
@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: ncurses, zlib

Partial log (click to expand)

/nix/store/wydzkhfm25cgbr1nfq28nqvgk0mk7518-ncurses-6.1
/nix/store/x0aj2fsd3hskn1fckk3w4b8bl6mx8m1s-zlib-1.2.11

@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: ncurses, zlib

Partial log (click to expand)

/nix/store/snmcl9nfjj49h2j0s1qpvmsajhm91141-ncurses-6.1
/nix/store/v8w9k5zzfidw4ik5rjxbpf9cfvbrj9s3-zlib-1.2.11

@matthewbauer
Copy link
Member Author

@GrahamcOfBorg build pkgsStatic.hello

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: ncurses, zlib

Partial log (click to expand)

/nix/store/iirmif7qgp7pgbv80z5x1sj3hbay893d-ncurses-6.1
/nix/store/di0d17idlw6fxf3qwpfyxciw94ywi664-zlib-1.2.11

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: pkgsStatic.hello

Partial log (click to expand)

post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/2zxr3jgljngkcmv3dkxcja3698pjrx1s-hello-2.10-aarch64-unknown-linux-musl
shrinking /nix/store/2zxr3jgljngkcmv3dkxcja3698pjrx1s-hello-2.10-aarch64-unknown-linux-musl/bin/hello
cannot find section .dynamic
gzipping man pages under /nix/store/2zxr3jgljngkcmv3dkxcja3698pjrx1s-hello-2.10-aarch64-unknown-linux-musl/share/man/
aarch64-unknown-linux-musl-strip is /nix/store/bqvzvb5b9c7rl4k5zara50gwnm5mmasr-aarch64-unknown-linux-musl-binutils-2.30/bin/aarch64-unknown-linux-musl-strip
stripping (with command aarch64-unknown-linux-musl-strip and flags -S) in /nix/store/2zxr3jgljngkcmv3dkxcja3698pjrx1s-hello-2.10-aarch64-unknown-linux-musl/lib  /nix/store/2zxr3jgljngkcmv3dkxcja3698pjrx1s-hello-2.10-aarch64-unknown-linux-musl/bin
patching script interpreter paths in /nix/store/2zxr3jgljngkcmv3dkxcja3698pjrx1s-hello-2.10-aarch64-unknown-linux-musl
checking for references to /build in /nix/store/2zxr3jgljngkcmv3dkxcja3698pjrx1s-hello-2.10-aarch64-unknown-linux-musl...
cannot find section .dynamic

@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: pkgsStatic.hello

Partial log (click to expand)

 /nix/store/acnfbicd84bnya2l2dq868b5l482qihw-coreutils-8.30/bin/mkdir -p '/nix/store/fvdzx3bmx2mxcavg4m6lja21h4r6bv09-hello-2.10/share/man/man1'
 /nix/store/acnfbicd84bnya2l2dq868b5l482qihw-coreutils-8.30/bin/install -c -m 644 hello.1 '/nix/store/fvdzx3bmx2mxcavg4m6lja21h4r6bv09-hello-2.10/share/man/man1'
make[3]: Leaving directory '/private/tmp/nix-build-hello-2.10.drv-0/hello-2.10'
make[2]: Leaving directory '/private/tmp/nix-build-hello-2.10.drv-0/hello-2.10'
make[1]: Leaving directory '/private/tmp/nix-build-hello-2.10.drv-0/hello-2.10'
post-installation fixup
gzipping man pages under /nix/store/fvdzx3bmx2mxcavg4m6lja21h4r6bv09-hello-2.10/share/man/
strip is /nix/store/g5r4apl0za012ffs6ladinwa5w0m1l3k-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/fvdzx3bmx2mxcavg4m6lja21h4r6bv09-hello-2.10/bin
patching script interpreter paths in /nix/store/fvdzx3bmx2mxcavg4m6lja21h4r6bv09-hello-2.10

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: pkgsStatic.hello

Partial log (click to expand)

post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/7fmc0g5wkc3z0rxrr36fag5hgg5bmry5-hello-2.10-x86_64-unknown-linux-musl
shrinking /nix/store/7fmc0g5wkc3z0rxrr36fag5hgg5bmry5-hello-2.10-x86_64-unknown-linux-musl/bin/hello
cannot find section .dynamic
gzipping man pages under /nix/store/7fmc0g5wkc3z0rxrr36fag5hgg5bmry5-hello-2.10-x86_64-unknown-linux-musl/share/man/
x86_64-unknown-linux-musl-strip is /nix/store/ygcd42hl4maam2h96svkay90p20070ij-x86_64-unknown-linux-musl-binutils-2.30/bin/x86_64-unknown-linux-musl-strip
stripping (with command x86_64-unknown-linux-musl-strip and flags -S) in /nix/store/7fmc0g5wkc3z0rxrr36fag5hgg5bmry5-hello-2.10-x86_64-unknown-linux-musl/lib  /nix/store/7fmc0g5wkc3z0rxrr36fag5hgg5bmry5-hello-2.10-x86_64-unknown-linux-musl/bin
patching script interpreter paths in /nix/store/7fmc0g5wkc3z0rxrr36fag5hgg5bmry5-hello-2.10-x86_64-unknown-linux-musl
checking for references to /build in /nix/store/7fmc0g5wkc3z0rxrr36fag5hgg5bmry5-hello-2.10-x86_64-unknown-linux-musl...
cannot find section .dynamic

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: ncurses, zlib

Partial log (click to expand)

/nix/store/iirmif7qgp7pgbv80z5x1sj3hbay893d-ncurses-6.1
/nix/store/di0d17idlw6fxf3qwpfyxciw94ywi664-zlib-1.2.11

@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: ncurses, zlib

Partial log (click to expand)

/nix/store/snmcl9nfjj49h2j0s1qpvmsajhm91141-ncurses-6.1
/nix/store/v8w9k5zzfidw4ik5rjxbpf9cfvbrj9s3-zlib-1.2.11

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: ncurses, zlib

Partial log (click to expand)

/nix/store/wydzkhfm25cgbr1nfq28nqvgk0mk7518-ncurses-6.1
/nix/store/x0aj2fsd3hskn1fckk3w4b8bl6mx8m1s-zlib-1.2.11

@matthewbauer matthewbauer changed the title Experiments in pkgsStatic: a fully static overlay Adding pkgsStatic: a fully static overlay Oct 30, 2018
@GrahamcOfBorg GrahamcOfBorg added the 2.status: merge conflict This PR has merge conflicts with the target branch label Dec 5, 2018
@matthewbauer matthewbauer force-pushed the crossOverlays branch 3 times, most recently from a0e0d77 to 51ddaf2 Compare December 5, 2018 02:08
@GrahamcOfBorg GrahamcOfBorg removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Dec 5, 2018
These aren’t needed with dontDisableStatic
crossOverlays only apply to the packages being built, not the build
packages. It is useful when you don’t care what is used to build your
packages, just what is being built. The idea relies heavily on the
cross compiling infrastructure. Using this implies that we need to
create a cross stdenv.
Nixpkgs is meant to link everything dynamically. We don’t want to
expose static packages at the top level. If some package needs
statically built binaries, it should use a custom override.
- makeStaticBinaries don’t work on Darwin (no stable ABI!)
- Need to make sure NIX_CFLAGS_LINK appends
- isStatic is not used anymore
The conditional was incorrect - postInstall script should only hsppen
when enableShared = true.
This is kind of a mess, but basically:

- static=true, shared=true means to build statically but move it to
  the static output
- static=true, shared=false means to build statically and leave it in
  the main output
- static=false, shared=true means to not build static at all

Confusingly, the old default was static=true, shared=true even though
static=false? Still can’t figure out what was meant by that.
when enableShared = false, we set --without-shared flag.
Adds the static overlay that can be used to build Nixpkgs statically.
Can be used like:

  nix build pkgsStatic.hello

Not all packages build, as some rely on dynamic linking.

, # Allow a configuration attribute set to be passed in as an argument.
config ? {}

, # List of overlays layers used to extend Nixpkgs.
overlays ? []

, # List of overlays to apply to target packages only.
crossOverlays ? []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a great improvement. As @Ericson2314 mentioned, this brings us much closer to build and host terminology. So what about changing this to hostOverlays, because then it can be applied to non-cross as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With non cross, the build and host packages are the same, so it's not possible to just override in one case. crossOverrides actually will turn on cross if no crossSystem (to be renamed :)) was given to make things work.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One reason that crossOverlays makes sense is that you will be using the "cross" stdenv here. This could be a little surprising, for instance, if you use crossOverlays on macOS, you will get CC=gcc instead of CC=clang.

Copy link
Member

@Ericson2314 Ericson2314 Dec 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah we should add LLVM-based cross. That is the easier part of iOS host-is-darwin cross, which is context in which I last worked on that. (i.e. I should have been worked on the lower hanging fruit but didn't.)

Copy link
Member

@roberth roberth Dec 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@matthewbauer so that's the host or run CC, right? So why not hostOverlays or runOverlays? I'd say "cross" is a property of the entire package set: was it cross-compiled? yes or no, boolean. When describing which architecture some variable applies to, it should be build or host/run, because that is precise.

Now I might be entirely wrong about this, but this, crossOverlays, seems very similar to the way in which I accidentally wrote pkgs.extend to be a bit broken.

I'm trying to do something at the edge of my understanding of Nixpkgs internals here on a limited time budget. My initial goal was to achieve greater consistency, mainly between the Nixpkgs function and NixOS, so that it will be as simple as possible to understand the interface and work with it. I hope that a new-ish perspective is useful; if not, I should focus on other things.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Ericson2314 I seem to have lost the original conversation - but I think there is a way to disable the Clang dependency in cctools. You just have to define DISABLE_CLANG_AS here:

https://github.com/tpoechtrager/cctools-port/blob/f53c6186393915ba9bff287f18769f9e14ec02e6/cctools/as/driver.c#L285

The builtin assembler should work fine then.

Copy link
Member Author

@matthewbauer matthewbauer Dec 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@robeth That looks like okay logic. We definitely want to unify the build, host, target & local, cross, eventually. But I still am thinking crossOverlays is good here because it mirrors crossSystem. Right now, this is how I see it-

  • {build,host,target}Platform - used internally in Nixpkgs as the "immediate" values of build, host, target. These are given to you, not set directly. Sliding windows algorithm handles the details...
  • {local,cross}System - external arguments of Nixpkgs describing where you are and where you want to go. This is what you want to set to get from point a to point b.

If cross-compiling is a graph, then each node is a set of packages. We activate a kind of greedy pathfinding algorithm in moving from localSystem to crossSystem. Outside of Nixpkgs, we shouldn't care about the exact path that is taken, just that we end up with a package built by "localSystem", for "crossSystem". In the same way, when you set "crossOverlays", you are saying to give me a package set that is statically built, but using my localSystem as is.

@dylex
Copy link
Contributor

dylex commented Jan 8, 2019

Since this change, I get an error about passing an unknown crossOverlays argument to stdenv when using a custom stdenv. 2ee9522a6c76e28949b6237928d2cc74fb9edfc2 seems to fix it fine as far as I can tell, similar to what was done to the other stdenvs. Does this seem right?

@matthewbauer
Copy link
Member Author

Thanks! I forgot to do some of the extra stdenv's. We could also just use ... because we aren't using crossOverlays. Merged in 04beae0.

if stdenv.hostPlatform.isDarwin
then throw "Cannot build fully static binaries on Darwin/macOS"
else stdenv.mkDerivation (args // {
NIX_CFLAGS_LINK = toString (args.NIX_CFLAGS_LINK or "") + "-static";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@matthewbauer "-static" -> " -static", right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah probably. The original version left out the space so I just left it as is

@jb55
Copy link
Contributor

jb55 commented Feb 18, 2019 via email

@matthewbauer
Copy link
Member Author

Yeah meson looks to be broken right now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: stdenv Standard environment 8.has: clean-up 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants