Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding pkgsStatic: a fully static overlay #48803

Merged
merged 8 commits into from
Dec 5, 2018
Merged

Adding pkgsStatic: a fully static overlay #48803

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
9d8de9f
c-ares: don’t set configure flags
matthewbauer Nov 8, 2018
a3a6ad7
stdenv: implement crossOverlays
matthewbauer Dec 5, 2018
b966d3c
treewide: remove static packages
matthewbauer Dec 5, 2018
8726f6a
stdenv/adapters.nix: fixup makeStaticBinaries
matthewbauer Dec 5, 2018
5e06294
darwin/libiconv: fix static/shared logic
matthewbauer Dec 5, 2018
e999def
zlib: clean up static/shared distincion
matthewbauer Dec 5, 2018
53a6789
ncurses: make shared linking optinoal
matthewbauer Dec 5, 2018
6d90a8b
top-level/stage.nix: add static overlay
matthewbauer Dec 5, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions pkgs/development/libraries/c-ares/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,6 @@ stdenv.mkDerivation rec {
sha256 = "0lk8knip4xk6qzksdkn7085mmgm4ixfczdyyjw656c193y3rgnvc";
};

configureFlags = stdenv.lib.optionals stdenv.hostPlatform.isWindows [ "--disable-shared" "--enable-static" ];

meta = with stdenv.lib; {
description = "A C library for asynchronous DNS requests";
homepage = https://c-ares.haxx.se;
Expand Down
13 changes: 9 additions & 4 deletions pkgs/stdenv/cross/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
{ lib
, localSystem, crossSystem, config, overlays
, localSystem, crossSystem, config, overlays, crossOverlays ? []
}:

let
bootStages = import ../. {
inherit lib localSystem overlays;
crossSystem = null;

crossSystem = localSystem;
crossOverlays = [];

# Ignore custom stdenvs when cross compiling for compatability
config = builtins.removeAttrs config [ "replaceStdenv" ];
};
Expand Down Expand Up @@ -33,7 +36,8 @@ in lib.init bootStages ++ [

# Run Packages
(buildPackages: {
inherit config overlays;
inherit config;
overlays = overlays ++ crossOverlays;
selfBuild = false;
stdenv = buildPackages.stdenv.override (old: rec {
buildPlatform = localSystem;
Expand All @@ -48,14 +52,15 @@ in lib.init bootStages ++ [

cc = if crossSystem.useiOSPrebuilt or false
then buildPackages.darwin.iosSdkPkgs.clang
else if crossSystem.useAndroidPrebuilt
else if crossSystem.useAndroidPrebuilt or false
then buildPackages.androidenv."androidndkPkgs_${crossSystem.ndkVer}".gcc
else buildPackages.gcc;

extraNativeBuildInputs = old.extraNativeBuildInputs
++ lib.optionals
(hostPlatform.isLinux && !buildPlatform.isLinux)
[ buildPackages.patchelf buildPackages.paxctl ]
++ lib.optional hostPlatform.isDarwin buildPackages.clang
++ lib.optional
(let f = p: !p.isx86 || p.libc == "musl"; in f hostPlatform && !(f buildPlatform))
buildPackages.updateAutotoolsGnuConfigScriptsHook
Expand Down
2 changes: 1 addition & 1 deletion pkgs/stdenv/custom/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
, localSystem, crossSystem, config, overlays
}:

assert crossSystem == null;
assert crossSystem == localSystem;
Copy link
Member

@Ericson2314 Ericson2314 Nov 28, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're getting closer to changing these to buildPlatform and hostPlatform :). Good discovery!


let
bootStages = import ../. {
Expand Down
4 changes: 2 additions & 2 deletions pkgs/stdenv/darwin/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{ lib
, localSystem, crossSystem, config, overlays
, localSystem, crossSystem, config, overlays, crossOverlays ? []

# Allow passing in bootstrap files directly so we can test the stdenv bootstrap process when changing the bootstrap tools
, bootstrapFiles ? let
Expand All @@ -16,7 +16,7 @@
}
}:

assert crossSystem == null;
assert crossSystem == localSystem;

let
inherit (localSystem) system platform;
Expand Down
4 changes: 2 additions & 2 deletions pkgs/stdenv/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
{ # Args just for stdenvs' usage
lib
# Args to pass on to the pkgset builder, too
, localSystem, crossSystem, config, overlays
, localSystem, crossSystem, config, overlays, crossOverlays ? []
} @ args:

let
Expand Down Expand Up @@ -36,7 +36,7 @@ let

# Select the appropriate stages for the platform `system'.
in
if crossSystem != null then stagesCross
if crossSystem != localSystem || crossOverlays != [] then stagesCross
else if config ? replaceStdenv then stagesCustom
else { # switch
"i686-linux" = stagesLinux;
Expand Down
2 changes: 1 addition & 1 deletion pkgs/stdenv/freebsd/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
, localSystem, crossSystem, config, overlays
}:

assert crossSystem == null;
assert crossSystem == localSystem;
let inherit (localSystem) system; in


Expand Down
4 changes: 2 additions & 2 deletions pkgs/stdenv/linux/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# compiler and linker that do not search in default locations,
# ensuring purity of components produced by it.
{ lib
, localSystem, crossSystem, config, overlays
, localSystem, crossSystem, config, overlays, crossOverlays ? []

, bootstrapFiles ?
let table = {
Expand Down Expand Up @@ -32,7 +32,7 @@
in files
}:

assert crossSystem == null;
assert crossSystem == localSystem;

let
inherit (localSystem) system platform;
Expand Down
2 changes: 1 addition & 1 deletion pkgs/stdenv/native/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
, localSystem, crossSystem, config, overlays
}:

assert crossSystem == null;
assert crossSystem == localSystem;

let
inherit (localSystem) system;
Expand Down
4 changes: 2 additions & 2 deletions pkgs/stdenv/nix/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
{ lib
, crossSystem, config, overlays
, crossSystem, localSystem, config, overlays
, bootStages
, ...
}:

assert crossSystem == null;
assert crossSystem == localSystem;

bootStages ++ [
(prevStage: {
Expand Down
13 changes: 8 additions & 5 deletions pkgs/top-level/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,16 +22,18 @@
# `*Platform`s.
localSystem

, # The system packages will ultimately be run on. Null if the two should be the
# same.
crossSystem ? null
, # The system packages will ultimately be run on.
crossSystem ? localSystem

, # Allow a configuration attribute set to be passed in as an argument.
config ? {}

, # List of overlays layers used to extend Nixpkgs.
overlays ? []

, # List of overlays to apply to target packages only.
crossOverlays ? []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a great improvement. As @Ericson2314 mentioned, this brings us much closer to build and host terminology. So what about changing this to hostOverlays, because then it can be applied to non-cross as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With non cross, the build and host packages are the same, so it's not possible to just override in one case. crossOverrides actually will turn on cross if no crossSystem (to be renamed :)) was given to make things work.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One reason that crossOverlays makes sense is that you will be using the "cross" stdenv here. This could be a little surprising, for instance, if you use crossOverlays on macOS, you will get CC=gcc instead of CC=clang.

Copy link
Member

@Ericson2314 Ericson2314 Dec 5, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah we should add LLVM-based cross. That is the easier part of iOS host-is-darwin cross, which is context in which I last worked on that. (i.e. I should have been worked on the lower hanging fruit but didn't.)

Copy link
Member

@roberth roberth Dec 5, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@matthewbauer so that's the host or run CC, right? So why not hostOverlays or runOverlays? I'd say "cross" is a property of the entire package set: was it cross-compiled? yes or no, boolean. When describing which architecture some variable applies to, it should be build or host/run, because that is precise.

Now I might be entirely wrong about this, but this, crossOverlays, seems very similar to the way in which I accidentally wrote pkgs.extend to be a bit broken.

I'm trying to do something at the edge of my understanding of Nixpkgs internals here on a limited time budget. My initial goal was to achieve greater consistency, mainly between the Nixpkgs function and NixOS, so that it will be as simple as possible to understand the interface and work with it. I hope that a new-ish perspective is useful; if not, I should focus on other things.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Ericson2314 I seem to have lost the original conversation - but I think there is a way to disable the Clang dependency in cctools. You just have to define DISABLE_CLANG_AS here:

https://github.com/tpoechtrager/cctools-port/blob/f53c6186393915ba9bff287f18769f9e14ec02e6/cctools/as/driver.c#L285

The builtin assembler should work fine then.

Copy link
Member Author

@matthewbauer matthewbauer Dec 5, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@robeth That looks like okay logic. We definitely want to unify the build, host, target & local, cross, eventually. But I still am thinking crossOverlays is good here because it mirrors crossSystem. Right now, this is how I see it-

  • {build,host,target}Platform - used internally in Nixpkgs as the "immediate" values of build, host, target. These are given to you, not set directly. Sliding windows algorithm handles the details...
  • {local,cross}System - external arguments of Nixpkgs describing where you are and where you want to go. This is what you want to set to get from point a to point b.

If cross-compiling is a graph, then each node is a set of packages. We activate a kind of greedy pathfinding algorithm in moving from localSystem to crossSystem. Outside of Nixpkgs, we shouldn't care about the exact path that is taken, just that we end up with a package built by "localSystem", for "crossSystem". In the same way, when you set "crossOverlays", you are saying to give me a package set that is statically built, but using my localSystem as is.


, # A function booting the final package set for a specific standard
# environment. See below for the arguments given to that function, the type of
# list it returns.
Expand Down Expand Up @@ -61,7 +63,8 @@ in let
builtins.intersectAttrs { platform = null; } config
// args.localSystem);

crossSystem = lib.mapNullable lib.systems.elaborate crossSystem0;
crossSystem = if crossSystem0 == null then localSystem
else lib.systems.elaborate crossSystem0;

# A few packages make a new package set to draw their dependencies from.
# (Currently to get a cross tool chain, or forced-i686 package.) Rather than
Expand Down Expand Up @@ -91,7 +94,7 @@ in let
boot = import ../stdenv/booter.nix { inherit lib allPackages; };

stages = stdenvStages {
inherit lib localSystem crossSystem config overlays;
inherit lib localSystem crossSystem config overlays crossOverlays;
};

pkgs = boot stages;
Expand Down