stdenvBootstrapTools: native aarch64-darwin build #202347

stephank · 2022-11-22T13:12:48Z

Description of changes

Following discussion in:

This removes cross compilation from the aarch64-darwin bootstrap tools, and instead builds them with a nativeaarch64-darwin toolchain.

This should fix the currently failing Hydra job. The last successful run was in april 2022, but it was disabled temporarily. Since re-enabling it, some fixes were done, but it now fails on Python, which does not support cross-compiling on Darwin at all. (Perhaps Python was added by accident, but cross-compiling from Intel to ARM will be a problem down the road any way.)

The following builds succeed for me:

nix-build pkgs/top-level/release.nix -A stdenvBootstrapTools.aarch64-darwin.test
nix-build pkgs/top-level/release.nix -A stdenvBootstrapTools.x86_64-darwin.test

And I also have a follow-up commit prepared that actually swaps in the new bootstrap-tools: stephank@7be022c. This successfully builds stdenv on both x86_64-darwin and aarch64-darwin. This will eventually be a second PR once Hydra has completed a build, because the URLs currently point to manual builds hosted on my server.

Things done

stephank · 2023-01-03T15:14:15Z

pkgs/stdenv/darwin/make-bootstrap-tools.nix

      # Copy coreutils, bash, etc.
      cp ${coreutils_}/bin/* $out/bin
      (cd $out/bin && rm vdir dir sha*sum pinky factor pathchk runcon shuf who whoami shred users)

      cp ${bash}/bin/bash $out/bin
+      ln -s bash $out/bin/sh


bash and bzip2 links were previously created by the unpack script, but I moved them here. I feel like we should avoid altering the package at unpack-time if we can, but the remainder (wrappers, install names) seem unavoidable.

stephank · 2023-01-03T15:15:17Z

pkgs/stdenv/darwin/make-bootstrap-tools.nix

-      ''}
+      # copy sigtool
+      cp -d ${pkgs.darwin.sigtool}/bin/sigtool $out/bin
+      cp -d ${pkgs.darwin.sigtool}/bin/codesign $out/bin


I removed the conditions around codesigning to keep both archs more in sync. They don't harm intel, and only affect the bootstrap.

stephank · 2023-01-03T15:16:21Z

pkgs/stdenv/darwin/make-bootstrap-tools.nix

-        cp -rd ${pkgs.darwin.CF}/Library $out
+      cp -rd ${pkgs.darwin.CF}/Library $out
+      ${lib.optionalString stdenv.targetPlatform.isAarch64 ''
+        cp -rd ${pkgs.darwin.libobjc}/lib/* $out/lib/


Both CoreFoundation and libobjc are tbds only on aarch64, but having these allows the CF test to pass.

stephank · 2023-01-03T15:17:38Z

pkgs/stdenv/darwin/make-bootstrap-tools.nix

+        sed -i -e 's|/nix/store/.*/libobjc.A.dylib|@executable_path/../libobjc.A.dylib|g' \
+          $out/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation.tbd


This is a bit of a hack. I'm not sure how to properly implement the rpath logic here for tbds. But it seems to work fine, and only affects the bootstrap.

pkgs/stdenv/darwin/make-bootstrap-tools.nix

stephank · 2023-01-03T15:32:31Z

pkgs/stdenv/darwin/make-bootstrap-tools.nix

-          echo patching $i
-          install_name_tool -add_rpath $out/lib $i || true
-        fi
-      done


This step seems unnecessary, because we already add @executable_path/../lib during the build. Regardless, I based the new script on the ARM variant, which doesn't have it, and that appears to work nicely for Intel too.

stephank · 2023-01-03T15:33:42Z

pkgs/stdenv/darwin/make-bootstrap-tools.nix


      export CPP="clang -E $flags"
-      export CC="clang $flags -Wl,-rpath,${unpack}/lib -Wl,-v -Wl,-sdk_version,10.10"
-      export CXX="clang++ $flags --stdlib=libc++ -lc++abi -isystem${unpack}/include/c++/v1 -Wl,-rpath,${unpack}/lib -Wl,-v -Wl,-sdk_version,10.10"


The -sdk_version appears unnecessary, and conflicts with -platform_version that the Clang driver already passes to ld. (It warns about -sdk_version being ignored.)

The -rpath is something Clang also accepts, no -Wl necessary.

stephank · 2023-01-03T15:35:39Z

pkgs/tools/archivers/dumpnar/default.nix

+  meta = with lib; {
+    homepage = "https://github.com/stephank/dumpnar";
+    description = "Minimal tool to produce a Nix NAR archive.";
+    license = licenses.lgpl2Plus;
+    platforms = platforms.all;
+    maintainers = [ maintainers.stephank ];
+  };


This is code extracted from Nix itself. Trying to use Nix itself here was trouble. It significantly increases the built closure, plus it apparently looks for config causing errors even for simple things like creating a NAR. I didn't investigate beyond that point and simply extracted the code. (I had another use for it myself.)

stephank · 2023-01-03T15:36:08Z

pkgs/top-level/release.nix

+              inherit (bootstrap) dist test;
+              # Test a full stdenv bootstrap from the bootstrap tools definition
+              # TODO: Re-enable once the new bootstrap-tools are in place.
+              #inherit (bootstrap.test-pkgs) stdenv;


This will be re-enabled in the follow-up PR that swaps in the new tools.

The change in bootstrapFiles means it's unfortunately not possible to test a stdenv build in this PR via this attribute. Instead, pull in the follow-up commit from the PR description, then just do nix-build -A stdenv (optionally setting system).

pkgs/top-level/release.nix

nixos-discourse · 2023-02-07T17:50:46Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/1802

pkgs/stdenv/darwin/make-bootstrap-tools.nix

pkgs/tools/archivers/dumpnar/default.nix

SuperSandro2000 · 2023-02-07T21:57:43Z

pkgs/top-level/release.nix

-            inherit
-              (import ../stdenv/linux/make-bootstrap-tools.nix {
+        genAttrs systemsWithAnySupport (system:
+          if hasSuffix "-linux" system then


Can we use stdenv.isLinux?

No, this is iterating over systemsWithAnySupport.

stephank · 2023-02-12T13:49:39Z

Thanks for the review. I updated the PR and believe I addressed all comments.

Et7f3 · 2023-04-23T15:09:14Z

pkgs/stdenv/darwin/make-bootstrap-tools.nix

    '';
  };

  bootstrapLlvmVersion = llvmPackages.llvm.version;

  bootstrapFiles = {
-    sh      = "${build}/on-server/sh";


nix-repl> :p legacyPackages.x86_64-darwin.freshBootstrapTools.test-pkgs.CoinMP error: attribute 'sh' missing at /nix/store/8ycgcgzaskmw3wcw0kn7v0v1ggwnvbzz-source/pkgs/stdenv/darwin/default.nix:72:15: 71| name = "bootstrap-tools"; 72| builder = bootstrapFiles.sh; # Not a filename! Attribute 'sh' on bootstrapFiles | ^ 73| args = if localSystem.isAarch64 then [ ./unpack-bootstrap-tools-aarch64.sh ] else [ ./unpack-bootstrap-tools.sh ]; «derivation

reverting this line show bzip2 is also required. So this PR might be incomplete.

See: https://github.com/NixOS/nixpkgs/pull/202347/files#r1060701648

There's a tracking PR for new bootstrap tools: #222717

Ah ok. I just track eval errors. I would have appreciated a throw or assert (in that case the :p can continue)

Artturin · 2023-09-07T01:12:33Z

pkgs/stdenv/darwin/make-bootstrap-tools.nix

-      # Set the ELF interpreter / RPATH in the bootstrap binaries.
-      echo Patching the tools...
+  bootstrapTools = derivation {
+    inherit system;


This breaks nix build -f pkgs/top-level/release.nix unstable

error: undefined variable 'system' at /home/artturin/nixgits/my-nixpkgs/.worktree/1/pkgs/stdenv/darwin/make-bootstrap-tools.nix:213:32: 212| 213| bootstrapTools = derivation { | ^ 214| inherit system;

it should probably be

diff --git a/pkgs/stdenv/darwin/make-bootstrap-tools.nix b/pkgs/stdenv/darwin/make-bootstrap-tools.nix index 94c61e396b65..46ba25f8603b 100644 --- a/pkgs/stdenv/darwin/make-bootstrap-tools.nix +++ b/pkgs/stdenv/darwin/make-bootstrap-tools.nix @@ -211,7 +211,7 @@ in rec { }; bootstrapTools = derivation { - inherit system; + inherit (localSystem) system; name = "bootstrap-tools"; builder = "${bootstrapFiles.tools}/bin/bash";

The error can also be replicated in darwin repl with freshBootstrapTools.bootstrapTools

Atemu requested a review from thefloweringash November 22, 2022 13:13

ofborg bot added the 6.topic: darwin Running or building packages on Darwin label Nov 22, 2022

stephank mentioned this pull request Nov 22, 2022

libcxx: link correct libcxxabi version #201268

Merged

13 tasks

ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels Nov 22, 2022

stephank force-pushed the feat/bootstrap-aarch64-darwin branch from bfac032 to ea1f4b6 Compare December 30, 2022 15:40

github-actions bot added the 6.topic: stdenv Standard environment label Dec 30, 2022

stephank force-pushed the feat/bootstrap-aarch64-darwin branch 4 times, most recently from e65de42 to 969c929 Compare January 3, 2023 10:03

stephank force-pushed the feat/bootstrap-aarch64-darwin branch from 969c929 to 283ca25 Compare January 3, 2023 15:08

stephank commented Jan 3, 2023

View reviewed changes

stephank marked this pull request as ready for review January 3, 2023 15:37

vcunat mentioned this pull request Jan 5, 2023

release.nix: don't block on aarch64-darwin boostrap tools #209193

Merged

SuperSandro2000 reviewed Feb 7, 2023

View reviewed changes

pkgs/stdenv/darwin/make-bootstrap-tools.nix Outdated Show resolved Hide resolved

SuperSandro2000 reviewed Feb 7, 2023

View reviewed changes

pkgs/stdenv/darwin/make-bootstrap-tools.nix Outdated Show resolved Hide resolved

SuperSandro2000 reviewed Feb 7, 2023

View reviewed changes

pkgs/stdenv/darwin/make-bootstrap-tools.nix Outdated Show resolved Hide resolved

SuperSandro2000 reviewed Feb 7, 2023

View reviewed changes

pkgs/stdenv/darwin/make-bootstrap-tools.nix Show resolved Hide resolved

SuperSandro2000 reviewed Feb 7, 2023

View reviewed changes

pkgs/tools/archivers/dumpnar/default.nix Outdated Show resolved Hide resolved

SuperSandro2000 reviewed Feb 7, 2023

View reviewed changes

stephank added 2 commits February 11, 2023 20:11

dumpnar: init at unstable-2023-01-01

487e295

stdenvBootstrapTools: native aarch64-darwin build

c3693fb

stephank force-pushed the feat/bootstrap-aarch64-darwin branch from 283ca25 to c3693fb Compare February 12, 2023 13:41

wegank approved these changes Mar 3, 2023

View reviewed changes

wegank merged commit 5028ec9 into NixOS:staging Mar 3, 2023

stephank deleted the feat/bootstrap-aarch64-darwin branch March 5, 2023 19:45

This was referenced Mar 23, 2023

Revert "release.nix: don't block on aarch64-darwin bootstrap tools" #221480

Merged

Tracking issue: Update Darwin bootstrap files #222717

Closed

Et7f3 reviewed Apr 23, 2023

View reviewed changes

Artturin reviewed Sep 7, 2023

View reviewed changes

wegank mentioned this pull request Mar 13, 2024

freshBootstrapTools: remove overlay, prune tools, use tar.xz archive #295557

Merged

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

stdenvBootstrapTools: native aarch64-darwin build #202347

stdenvBootstrapTools: native aarch64-darwin build #202347

stephank commented Nov 22, 2022 •

edited

Loading

stephank Jan 3, 2023 •

edited

Loading

stephank Jan 3, 2023

stephank Jan 3, 2023

stephank Jan 3, 2023

stephank Jan 3, 2023

stephank Jan 3, 2023

stephank Jan 3, 2023 •

edited

Loading

stephank Jan 3, 2023 •

edited

Loading

nixos-discourse commented Feb 7, 2023

SuperSandro2000 Feb 7, 2023

stephank Feb 11, 2023

stephank commented Feb 12, 2023

Et7f3 Apr 23, 2023

Et7f3 Apr 23, 2023

stephank Apr 23, 2023

Et7f3 Apr 23, 2023

Artturin Sep 7, 2023 •

edited

Loading

Artturin Sep 7, 2023

Artturin Sep 7, 2023

		sed -i -e 's\|/nix/store/.*/libobjc.A.dylib\|@executable_path/../libobjc.A.dylib\|g' \
		$out/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation.tbd

stdenvBootstrapTools: native aarch64-darwin build #202347

stdenvBootstrapTools: native aarch64-darwin build #202347

Conversation

stephank commented Nov 22, 2022 • edited Loading

Description of changes

Things done

stephank Jan 3, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stephank Jan 3, 2023 • edited Loading

Choose a reason for hiding this comment

stephank Jan 3, 2023 • edited Loading

Choose a reason for hiding this comment

nixos-discourse commented Feb 7, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stephank commented Feb 12, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Artturin Sep 7, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stephank commented Nov 22, 2022 •

edited

Loading

stephank Jan 3, 2023 •

edited

Loading

stephank Jan 3, 2023 •

edited

Loading

stephank Jan 3, 2023 •

edited

Loading

Artturin Sep 7, 2023 •

edited

Loading