Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nodejs: use ninja for build #327653

Merged
merged 5 commits into from
Aug 10, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 37 additions & 2 deletions pkgs/development/web/nodejs/bypass-darwin-xcrun-node16.patch
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please delete the old code instead of commenting it

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be inconsistent with the part of the patch that applies to the other copy of this code, which was already in‐tree. Is it important enough to re‐do the whole patch over?

Copy link
Member Author

@tie tie Jul 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #327653 (comment)
I don’t think it’s worth redoing this patch in the context of this PR.

Original file line number Diff line number Diff line change
@@ -1,7 +1,42 @@
Avoids needing xcrun or xcodebuild in PATH for native package builds

diff --git a/deps/npm/node_modules/node-gyp/gyp/pylib/gyp/xcode_emulation.py b/deps/npm/node_modules/node-gyp/gyp/pylib/gyp/xcode_emulation.py
index a75d8ee..476440d 100644
--- a/tools/gyp/pylib/gyp/xcode_emulation.py
+++ b/tools/gyp/pylib/gyp/xcode_emulation.py
@@ -522,7 +522,13 @@ class XcodeSettings:
# Since the CLT has no SDK paths anyway, returning None is the
# most sensible route and should still do the right thing.
try:
- return GetStdoutQuiet(["xcrun", "--sdk", sdk, infoitem])
+ #return GetStdoutQuiet(["xcrun", "--sdk", sdk, infoitem])
+ return {
+ "--show-sdk-platform-path": "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform",
+ "--show-sdk-path": "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk",
Comment on lines +12 to +13
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this work with the build sandbox?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Node.js has two copies of GYP and this change just extends the existing patch to the second GYP copy. This works fine with sandbox, but I'm not really sure how that's supposed to work without sandbox since technically nothing stops the build from accessing these paths.

Anyway, this allows us to drop xcbuild from Node.js native build inputs with minor copy-paste modification to an existing patch. I'm leaving refactoring or documenting this behavior to my future self (e.g. if we add split dev output, we can use fake xcbuild in propagatedNativeBuildInputs instead of patching GYP).

+ "--show-sdk-build-version": "19A547",
+ "--show-sdk-version": "10.15"
+ }[infoitem]
except GypError:
pass

@@ -1499,7 +1505,8 @@ def XcodeVersion():
version = ""
build = ""
try:
- version_list = GetStdoutQuiet(["xcodebuild", "-version"]).splitlines()
+ #version_list = GetStdoutQuiet(["xcodebuild", "-version"]).splitlines()
+ version_list = []
# In some circumstances xcodebuild exits 0 but doesn't return
# the right results; for example, a user on 10.7 or 10.8 with
# a bogus path set via xcode-select
@@ -1510,7 +1517,8 @@ def XcodeVersion():
version = version_list[0].split()[-1] # Last word on first line
build = version_list[-1].split()[-1] # Last word on last line
except GypError: # Xcode not installed so look for XCode Command Line Tools
- version = CLTVersion() # macOS Catalina returns 11.0.0.0.1.1567737322
+ #version = CLTVersion() # macOS Catalina returns 11.0.0.0.1.1567737322
+ version = "11.0.0.0.1.1567737322"
if not version:
raise GypError("No Xcode or CLT version detected!")
# Be careful to convert "4.2.3" to "0423" and "11.0.0" to "1100":
--- a/deps/npm/node_modules/node-gyp/gyp/pylib/gyp/xcode_emulation.py
+++ b/deps/npm/node_modules/node-gyp/gyp/pylib/gyp/xcode_emulation.py
@@ -522,7 +522,13 @@ class XcodeSettings:
15 changes: 15 additions & 0 deletions pkgs/development/web/nodejs/configure-armv6-vfpv2.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
Allows ARM FPU to be set to vfpv2, e.g. for Raspberry Pi.

See https://github.com/nodejs/node/issues/44357#issuecomment-1235821878

--- a/configure.py
+++ b/configure.py
@@ -50,7 +50,7 @@
valid_arch = ('arm', 'arm64', 'ia32', 'mips', 'mipsel', 'mips64el', 'ppc',
'ppc64', 'x64', 'x86', 'x86_64', 's390x', 'riscv64', 'loong64')
valid_arm_float_abi = ('soft', 'softfp', 'hard')
-valid_arm_fpu = ('vfp', 'vfpv3', 'vfpv3-d16', 'neon')
+valid_arm_fpu = ('vfp', 'vfpv2', 'vfpv3', 'vfpv3-d16', 'neon')
valid_mips_arch = ('loongson', 'r1', 'r2', 'r6', 'rx')
valid_mips_fpu = ('fp32', 'fp64', 'fpxx')
valid_mips_float_abi = ('soft', 'hard')
138 changes: 138 additions & 0 deletions pkgs/development/web/nodejs/configure-emulator-node18.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
From 4b83f714c821d6d4d2306673ee3a87907cfec80e Mon Sep 17 00:00:00 2001
From: Ivan Trubach <mr.trubach@icloud.com>
Date: Fri, 19 Jul 2024 10:45:13 +0300
Subject: [PATCH] build: support setting an emulator from configure script
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

V8’s JIT infrastructure requires binaries such as mksnapshot to be run
during the build. However, these binaries must have the same bit-width
as the host platform (e.g. a x86_64 build platform targeting ARMv6 needs
to produce a 32-bit binary).

To work around this issue, allow building the binaries for the host
platform and running them on the build platform with an emulator.

Based on Buildroot’s nodejs-src 0001-add-qemu-wrapper-support.patch.
https://gitlab.com/buildroot.org/buildroot/-/blob/c1d5eada4d4db9eeaa1c44dd1dea95a67c8a70ca/package/nodejs/nodejs-src/0001-add-qemu-wrapper-support.patch

Upstream: https://github.com/nodejs/node/pull/53899
---
common.gypi | 1 +
configure.py | 14 ++++++++++++++
node.gyp | 3 +++
tools/v8_gypfiles/v8.gyp | 4 ++++
4 files changed, 22 insertions(+)

diff --git a/common.gypi b/common.gypi
index ec92c9df4c..6474495ab6 100644
--- a/common.gypi
+++ b/common.gypi
@@ -13,6 +13,7 @@
'enable_pgo_generate%': '0',
'enable_pgo_use%': '0',
'python%': 'python',
+ 'emulator%': [],

'node_shared%': 'false',
'force_dynamic_crt%': 0,
diff --git a/configure.py b/configure.py
index 82916748fd..10dc0becbb 100755
--- a/configure.py
+++ b/configure.py
@@ -112,6 +112,12 @@ parser.add_argument('--dest-cpu',
choices=valid_arch,
help=f"CPU architecture to build for ({', '.join(valid_arch)})")

+parser.add_argument('--emulator',
+ action='store',
+ dest='emulator',
+ default=None,
+ help='emulator command that can run executables built for the target system')
+
parser.add_argument('--cross-compiling',
action='store_true',
dest='cross_compiling',
@@ -2160,6 +2166,14 @@ write('config.mk', do_not_edit + config_str)
gyp_args = ['--no-parallel', '-Dconfiguring_node=1']
gyp_args += ['-Dbuild_type=' + config['BUILDTYPE']]

+if options.emulator is not None:
+ if not options.cross_compiling:
+ # Note that emulator is a list so we have to quote the variable.
+ gyp_args += ['-Demulator=' + shlex.quote(options.emulator)]
+ else:
+ # TODO: perhaps use emulator for tests?
+ warn('The `--emulator` option has no effect when cross-compiling.')
+
if options.use_ninja:
gyp_args += ['-f', 'ninja-' + flavor]
elif flavor == 'win' and sys.platform != 'msys':
diff --git a/node.gyp b/node.gyp
index 08cb3f38e8..515b305933 100644
--- a/node.gyp
+++ b/node.gyp
@@ -332,6 +332,7 @@
'<(SHARED_INTERMEDIATE_DIR)/node_snapshot.cc',
],
'action': [
+ '<@(emulator)',
'<(node_mksnapshot_exec)',
'--build-snapshot',
'<(node_snapshot_main)',
@@ -351,6 +352,7 @@
'<(SHARED_INTERMEDIATE_DIR)/node_snapshot.cc',
],
'action': [
+ '<@(emulator)',
'<@(_inputs)',
'<@(_outputs)',
],
@@ -1520,6 +1522,7 @@
'<(PRODUCT_DIR)/<(node_core_target_name).def',
],
'action': [
+ '<@(emulator)',
'<(PRODUCT_DIR)/gen_node_def.exe',
'<@(_inputs)',
'<@(_outputs)',
diff --git a/tools/v8_gypfiles/v8.gyp b/tools/v8_gypfiles/v8.gyp
index ba8b161f0f..d5c90dad50 100644
--- a/tools/v8_gypfiles/v8.gyp
+++ b/tools/v8_gypfiles/v8.gyp
@@ -99,6 +99,7 @@
'<@(torque_outputs_inc)',
],
'action': [
+ '<@(emulator)',
'<(PRODUCT_DIR)/<(EXECUTABLE_PREFIX)torque<(EXECUTABLE_SUFFIX)',
'-o', '<(SHARED_INTERMEDIATE_DIR)/torque-generated',
'-v8-root', '<(V8_ROOT)',
@@ -219,6 +220,7 @@
'action': [
'<(python)',
'<(V8_ROOT)/tools/run.py',
+ '<@(emulator)',
'<@(_inputs)',
'<@(_outputs)',
],
@@ -442,6 +444,7 @@
}],
],
'action': [
+ '<@(emulator)',
'>@(_inputs)',
'>@(mksnapshot_flags)',
],
@@ -1577,6 +1580,7 @@
'action': [
'<(python)',
'<(V8_ROOT)/tools/run.py',
+ '<@(emulator)',
'<@(_inputs)',
'<@(_outputs)',
],
--
2.44.1

146 changes: 146 additions & 0 deletions pkgs/development/web/nodejs/configure-emulator.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
From 999d918bc8fefec1752243743a47c0ce5380bcec Mon Sep 17 00:00:00 2001
From: Ivan Trubach <mr.trubach@icloud.com>
Date: Wed, 17 Jul 2024 10:16:02 +0300
Subject: [PATCH] build: support setting an emulator from configure script
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

V8’s JIT infrastructure requires binaries such as mksnapshot to be run
during the build. However, these binaries must have the same bit-width
as the host platform (e.g. a x86_64 build platform targeting ARMv6 needs
to produce a 32-bit binary).

To work around this issue, allow building the binaries for the host
platform and running them on the build platform with an emulator.

Based on Buildroot’s nodejs-src 0001-add-qemu-wrapper-support.patch.
https://gitlab.com/buildroot.org/buildroot/-/blob/c1d5eada4d4db9eeaa1c44dd1dea95a67c8a70ca/package/nodejs/nodejs-src/0001-add-qemu-wrapper-support.patch

Upstream: https://github.com/nodejs/node/pull/53899
---
common.gypi | 1 +
configure.py | 14 ++++++++++++++
node.gyp | 4 ++++
tools/v8_gypfiles/v8.gyp | 4 ++++
4 files changed, 23 insertions(+)

diff --git a/common.gypi b/common.gypi
index 154bbf2a0d..54d2afe3b3 100644
--- a/common.gypi
+++ b/common.gypi
@@ -13,6 +13,7 @@
'enable_pgo_generate%': '0',
'enable_pgo_use%': '0',
'python%': 'python',
+ 'emulator%': [],

'node_shared%': 'false',
'force_dynamic_crt%': 0,
diff --git a/configure.py b/configure.py
index f7e3310723..f7c7acdf4f 100755
--- a/configure.py
+++ b/configure.py
@@ -112,6 +112,12 @@ parser.add_argument('--dest-cpu',
choices=valid_arch,
help=f"CPU architecture to build for ({', '.join(valid_arch)})")

+parser.add_argument('--emulator',
+ action='store',
+ dest='emulator',
+ default=None,
+ help='emulator command that can run executables built for the target system')
+
parser.add_argument('--cross-compiling',
action='store_true',
dest='cross_compiling',
@@ -2276,6 +2282,14 @@ if flavor == 'win' and python.lower().endswith('.exe'):
# will fail to run python scripts.
gyp_args += ['-Dpython=' + python]

+if options.emulator is not None:
+ if not options.cross_compiling:
+ # Note that emulator is a list so we have to quote the variable.
+ gyp_args += ['-Demulator=' + shlex.quote(options.emulator)]
+ else:
+ # TODO: perhaps use emulator for tests?
+ warn('The `--emulator` option has no effect when cross-compiling.')
+
if options.use_ninja:
gyp_args += ['-f', 'ninja-' + flavor]
elif flavor == 'win' and sys.platform != 'msys':
diff --git a/node.gyp b/node.gyp
index 9617596760..439c76aca6 100644
--- a/node.gyp
+++ b/node.gyp
@@ -703,6 +703,7 @@
'<(SHARED_INTERMEDIATE_DIR)/node_snapshot.cc',
],
'action': [
+ '<@(emulator)',
'<(node_mksnapshot_exec)',
'--build-snapshot',
'<(node_snapshot_main)',
@@ -722,6 +723,7 @@
'<(SHARED_INTERMEDIATE_DIR)/node_snapshot.cc',
],
'action': [
+ '<@(emulator)',
'<@(_inputs)',
'<@(_outputs)',
],
@@ -1010,6 +1012,7 @@
'<(SHARED_INTERMEDIATE_DIR)/node_javascript.cc',
],
'action': [
+ '<@(emulator)',
'<(node_js2c_exec)',
'<@(_outputs)',
'lib',
@@ -1477,6 +1480,7 @@
'<(PRODUCT_DIR)/<(node_core_target_name).def',
],
'action': [
+ '<@(emulator)',
'<(PRODUCT_DIR)/gen_node_def.exe',
'<@(_inputs)',
'<@(_outputs)',
diff --git a/tools/v8_gypfiles/v8.gyp b/tools/v8_gypfiles/v8.gyp
index d65a5c268e..5cd6c36b86 100644
--- a/tools/v8_gypfiles/v8.gyp
+++ b/tools/v8_gypfiles/v8.gyp
@@ -112,6 +112,7 @@
'<@(torque_outputs_inc)',
],
'action': [
+ '<@(emulator)',
'<(PRODUCT_DIR)/<(EXECUTABLE_PREFIX)torque<(EXECUTABLE_SUFFIX)',
'-o', '<(SHARED_INTERMEDIATE_DIR)/torque-generated',
'-v8-root', '<(V8_ROOT)',
@@ -232,6 +233,7 @@
'action': [
'<(python)',
'<(V8_ROOT)/tools/run.py',
+ '<@(emulator)',
'<@(_inputs)',
'<@(_outputs)',
],
@@ -453,6 +455,7 @@
}],
],
'action': [
+ '<@(emulator)',
'>@(_inputs)',
'>@(mksnapshot_flags)',
],
@@ -1842,6 +1845,7 @@
'action': [
'<(python)',
'<(V8_ROOT)/tools/run.py',
+ '<@(emulator)',
'<@(_inputs)',
'<@(_outputs)',
],
--
2.44.1

261 changes: 176 additions & 85 deletions pkgs/development/web/nodejs/nodejs.nix
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
{ lib, stdenv, fetchurl, openssl, python, zlib, libuv, util-linux, http-parser, bash
, pkg-config, which, buildPackages
{ lib, stdenv, fetchurl, openssl, python, zlib, libuv, http-parser, icu, bash
, ninja, pkgconf, unixtools, runCommand, buildPackages
, testers
# for `.pkgs` attribute
, callPackage
# Updater dependencies
, writeScript, coreutils, gnugrep, jq, curl, common-updater-scripts, nix, runtimeShell
, gnupg
, darwin, xcbuild
, procps, icu
, darwin
, installShellFiles
}:

@@ -16,35 +15,109 @@
let
inherit (darwin.apple_sdk.frameworks) CoreServices ApplicationServices;

isCross = stdenv.hostPlatform != stdenv.buildPlatform;

majorVersion = lib.versions.major version;
minorVersion = lib.versions.minor version;

pname = if enableNpm then "nodejs" else "nodejs-slim";

useSharedHttpParser = !stdenv.isDarwin && lib.versionOlder "${majorVersion}.${minorVersion}" "11.4";
canExecute = stdenv.buildPlatform.canExecute stdenv.hostPlatform;
emulator = stdenv.hostPlatform.emulator buildPackages;

# See valid_os and valid_arch in configure.py.
destOS =
let
platform = stdenv.hostPlatform;
in
if platform.isiOS then
"ios"
else if platform.isAndroid then
"android"
else if platform.isWindows then
"win"
else if platform.isDarwin then
"mac"
else if platform.isLinux then
"linux"
else if platform.isOpenBSD then
"openbsd"
else if platform.isFreeBSD then
"freebsd"
else
throw "unsupported os ${platform.uname.system}";
destCPU =
let
platform = stdenv.hostPlatform;
in
if platform.isAarch then
"arm" + lib.optionalString platform.is64bit "64"
else if platform.isMips32 then
"mips" + lib.optionalString platform.isLittleEndian "le"
else if platform.isMips64 && platform.isLittleEndian then
"mips64el"
else if platform.isPower then
"ppc" + lib.optionalString platform.is64bit "64"
else if platform.isx86_64 then
"x64"
else if platform.isx86_32 then
"ia32"
else if platform.isS390x then
"s390x"
else if platform.isRiscV64 then
"riscv64"
else if platform.isLoongArch64 then
"loong64"
else
throw "unsupported cpu ${platform.uname.processor}";
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not an idiomatic Nixpkgs pattern and should probably use some fallback for unknown platforms instead of throwing an error. I’m keeping it for now and will address this in subsequent code style PRs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's fine to throw if we don't know the platform.

destARMFPU =
let
platform = stdenv.hostPlatform;
in
if platform.isAarch32 && platform ? gcc.fpu then
lib.throwIfNot (builtins.elem platform.gcc.fpu [
"vfp"
"vfpv2"
"vfpv3"
"vfpv3-d16"
"neon"
]) "unsupported ARM FPU ${platform.gcc.fpu}" platform.gcc.fpu
else
null;
destARMFloatABI =
let
platform = stdenv.hostPlatform;
in
if platform.isAarch32 && platform ? gcc.float-abi then
lib.throwIfNot (builtins.elem platform.gcc.float-abi [
"soft"
"softfp"
"hard"
]) "unsupported ARM float ABI ${platform.gcc.float-abi}" platform.gcc.float-abi
else
null;
# TODO: also handle MIPS flags (mips_arch, mips_fpu, mips_float_abi).

useSharedHttpParser = !stdenv.hostPlatform.isDarwin && lib.versionOlder "${majorVersion}.${minorVersion}" "11.4";

sharedLibDeps = { inherit openssl zlib libuv; } // (lib.optionalAttrs useSharedHttpParser { inherit http-parser; });

sharedConfigureFlags = lib.concatMap (name: [
"--shared-${name}"
"--shared-${name}-libpath=${lib.getLib sharedLibDeps.${name}}/lib"
/** Closure notes: we explicitly avoid specifying --shared-*-includes,
* as that would put the paths into bin/nodejs.
* Including pkg-config in build inputs would also have the same effect!
*/
]) (builtins.attrNames sharedLibDeps) ++ [
"--with-intl=system-icu"
"--openssl-use-def-ca-store"
];

copyLibHeaders =
map
(name: "${lib.getDev sharedLibDeps.${name}}/include/*")
(builtins.attrNames sharedLibDeps);

extraConfigFlags = lib.optionals (!enableNpm) [ "--without-npm" ];
# Currently stdenv sets CC/LD/AR/etc environment variables to program names
# instead of absolute paths. If we add cctools to nativeBuildInputs, that
# would shadow stdenv’s bintools and potentially break other parts of the
# build. The correct behavior is to use absolute paths, and there is a PR for
# that, see https://github.com/NixOS/nixpkgs/pull/314920. As a temporary
# workaround, we use only a single program we need (and that is not part of
# the stdenv).
darwin-cctools-only-libtool =
# Would be nice to have onlyExe builder similar to onlyBin…
runCommand "darwin-cctools-only-libtool" { cctools = lib.getBin buildPackages.cctools; } ''
mkdir -p "$out/bin"
ln -s "$cctools/bin/libtool" "$out/bin/libtool"
'';

package = stdenv.mkDerivation (finalAttrs:
let
@@ -61,58 +134,90 @@ let

strictDeps = true;

env = lib.optionalAttrs (stdenv.isDarwin && stdenv.isx86_64) {
env = {
# Tell ninja to avoid ANSI sequences, otherwise we don’t see build
# progress in Nix logs.
#
# Note: do not set TERM=dumb environment variable globally, it is used in
# test-ci-js test suite to skip tests that otherwise run fine.
NINJA = "TERM=dumb ninja";
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not setting TERM=dumb globally because Node.js v18 needs nodejs/node@f76b28f backport.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should write that into a comment

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, this is unfortunate. I think Nix build environments have enough of a PTY to run these tests, but we’re using TERM=dumb to get Ninja to give us more detailed output. It’d be nice if we could still run those tests, but it’s probably not that important.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests run fine unless TERM=dumb is set 😅

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. We could override TERM for just those tests, but it’d be ugly. I’m guessing Ninja has probably already been approached about an alternate way to toggle non‐fancy build output and maybe rejected it. Oh well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

} // lib.optionalAttrs (stdenv.hostPlatform.isDarwin && stdenv.hostPlatform.isx86_64) {
# Make sure libc++ uses `posix_memalign` instead of `aligned_alloc` on x86_64-darwin.
# Otherwise, nodejs would require the 11.0 SDK and macOS 10.15+.
NIX_CFLAGS_COMPILE = "-D__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__=101300";
NIX_CFLAGS_COMPILE = "-D__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__=101300 -Wno-macro-redefined";
};

depsBuildBuild = [ buildPackages.stdenv.cc openssl libuv zlib icu ];

# NB: technically, we do not need bash in build inputs since all scripts are
# wrappers over the corresponding JS scripts. There are some packages though
# that use bash wrappers, e.g. polaris-web.
buildInputs = lib.optionals stdenv.isDarwin [ CoreServices ApplicationServices ]
buildInputs = lib.optionals stdenv.hostPlatform.isDarwin [ CoreServices ApplicationServices ]
++ [ zlib libuv openssl http-parser icu bash ];

nativeBuildInputs = [ installShellFiles pkg-config python which ]
++ lib.optionals stdenv.isDarwin [ xcbuild ];
nativeBuildInputs =
[
installShellFiles
ninja
pkgconf
python
]
++ lib.optionals stdenv.buildPlatform.isDarwin [
# gyp checks `sysctl -n hw.memsize` if `sys.platform == "darwin"`.
unixtools.sysctl
]
++ lib.optionals stdenv.hostPlatform.isDarwin [
# For gyp-mac-tool if `flavor == "mac"`.
darwin-cctools-only-libtool
];

# We currently rely on Makefile and stdenv for build phases, so do not let
# ninja’s setup hook to override default stdenv phases.
dontUseNinjaBuild = true;
dontUseNinjaCheck = true;
dontUseNinjaInstall = true;

outputs = [ "out" "libv8" ];
setOutputFlags = false;
moveToDev = false;

configureFlags = let
inherit (stdenv.hostPlatform) gcc isAarch32;
in sharedConfigureFlags ++ lib.optionals (lib.versionOlder version "19") [
"--without-dtrace"
] ++ (lib.optionals isCross [
"--cross-compiling"
"--dest-cpu=${let platform = stdenv.hostPlatform; in
if platform.isAarch32 then "arm"
else if platform.isAarch64 then "arm64"
else if platform.isMips32 && platform.isLittleEndian then "mipsel"
else if platform.isMips32 && !platform.isLittleEndian then "mips"
else if platform.isMips64 && platform.isLittleEndian then "mips64el"
else if platform.isPower && platform.is32bit then "ppc"
else if platform.isPower && platform.is64bit then "ppc64"
else if platform.isx86_64 then "x86_64"
else if platform.isx86_32 then "x86"
else if platform.isS390 && platform.is64bit then "s390x"
else if platform.isRiscV && platform.is64bit then "riscv64"
else throw "unsupported cpu ${stdenv.hostPlatform.uname.processor}"}"
]) ++ (lib.optionals (isCross && isAarch32 && lib.hasAttr "fpu" gcc) [
"--with-arm-fpu=${gcc.fpu}"
]) ++ (lib.optionals (isCross && isAarch32 && lib.hasAttr "float-abi" gcc) [
"--with-arm-float-abi=${gcc.float-abi}"
]) ++ extraConfigFlags;

configurePlatforms = [];
configureFlags =
[
"--ninja"
"--with-intl=system-icu"
"--openssl-use-def-ca-store"
"--no-cross-compiling"
"--dest-os=${destOS}"
"--dest-cpu=${destCPU}"
]
++ lib.optionals (destARMFPU != null) [ "--with-arm-fpu=${destARMFPU}" ]
++ lib.optionals (destARMFloatABI != null) [ "--with-arm-float-abi=${destARMFloatABI}" ]
++ lib.optionals (!canExecute) [
# Node.js requires matching bitness between build and host platforms, e.g.
# for V8 startup snapshot builder (see tools/snapshot) and some other
# tools. We apply a patch that runs these tools using a host platform
# emulator and avoid cross-compiling altogether (from the build system’s
# perspective).
"--emulator=${emulator}"
]
++ lib.optionals (lib.versionOlder version "19") [ "--without-dtrace" ]
++ lib.optionals (!enableNpm) [ "--without-npm" ]
++ lib.concatMap (name: [
"--shared-${name}"
"--shared-${name}-libpath=${lib.getLib sharedLibDeps.${name}}/lib"
/**
Closure notes: we explicitly avoid specifying --shared-*-includes,
as that would put the paths into bin/nodejs.
Including pkg-config in build inputs would also have the same effect!
FIXME: the statement above is outdated, we have to include pkg-config
in build inputs for system-icu.
*/
]) (builtins.attrNames sharedLibDeps);

configurePlatforms = [ ];

dontDisableStatic = true;

configureScript = writeScript "nodejs-configure" ''
export CC_host="$CC_FOR_BUILD" CXX_host="$CXX_FOR_BUILD"
exec ${python.executable} configure.py "$@"
'';

@@ -140,9 +245,10 @@ let

__darwinAllowLocalNetworking = true; # for tests

# TODO: what about tests when cross-compiling?
# Note that currently stdenv does not run check phase if build ≠ host.
doCheck = true;
doCheck = canExecute;

# See https://github.com/nodejs/node/issues/22006
enableParallelChecking = false;
Comment on lines +250 to +251
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like upstream hasn’t seen any issues here for years and closed this issue as a result. I’m wondering if it really is an upstream issue, but if so, it’d be good to ping the issue with example logs to hopefully get it reopened (and maybe even fixed).


# Some dependencies required for tools/doc/node_modules (and therefore
# test-addons, jstest and others) target are not included in the tarball.
@@ -158,8 +264,6 @@ let
# than a year (Node.js 18 will be EOL at 2025-04-30). Note that these
# failures are specific to Nix sandbox on macOS and should not affect
# actual functionality.
] ++ lib.optionals (!stdenv.isDarwin) [
# TODO: JS test suite is too flaky on Darwin; revisit at a later date.
"test-ci-js"
]);

@@ -182,7 +286,7 @@ let
"test-tls-cli-max-version-1.3"
"test-tls-client-auth"
"test-tls-sni-option"
] ++ lib.optionals stdenv.hostPlatform.isDarwin [
] ++ lib.optionals stdenv.buildPlatform.isDarwin [
# Disable tests that don’t work under macOS sandbox.
"test-macos-app-sandbox"
"test-os"
@@ -207,18 +311,23 @@ let
"test-runner-run"
"test-runner-watch-mode"
"test-watch-mode-files_watcher"
] ++ lib.optionals (stdenv.buildPlatform.isDarwin && stdenv.buildPlatform.isx86_64) [
# These tests fail on x86_64-darwin (even without sandbox).
# TODO: revisit at a later date.
"test-fs-readv"
"test-fs-readv-sync"
])}"
];

postInstall = ''
HOST_PATH=$out/bin patchShebangs --host $out
${lib.optionalString (stdenv.buildPlatform.canExecute stdenv.hostPlatform) ''
$out/bin/${self.meta.mainProgram} --completion-bash > ${self.meta.mainProgram}.bash
installShellCompletion ${self.meta.mainProgram}.bash
${lib.optionalString canExecute ''
$out/bin/node --completion-bash > node.bash
installShellCompletion node.bash
''}
${lib.optionalString (enableNpm) ''
${lib.optionalString enableNpm ''
mkdir -p $out/share/bash-completion/completions
ln -s $out/lib/node_modules/npm/lib/utils/completion.sh \
$out/share/bash-completion/completions/npm
@@ -231,24 +340,14 @@ let
''}
# install the missing headers for node-gyp
# TODO: add dev output and use propagatedBuildInputs instead of copying headers.
cp -r ${lib.concatStringsSep " " copyLibHeaders} $out/include/node
# assemble a static v8 library and put it in the 'libv8' output
mkdir -p $libv8/lib
pushd out/Release/obj.target
pushd out/Release/obj
find . -path "./torque_*/**/*.o" -or -path "./v8*/**/*.o" | sort -u >files
${if stdenv.buildPlatform.isGnu then ''
ar -cqs $libv8/lib/libv8.a @files
'' else ''
# llvm-ar supports response files, so take advantage of it if it’s available.
if [ "$(basename $(readlink -f $(command -v ar)))" = "llvm-ar" ]; then
ar -cqs $libv8/lib/libv8.a @files
else
cat files | while read -r file; do
ar -cqS $libv8/lib/libv8.a $file
done
fi
''}
$AR -cqs $libv8/lib/libv8.a @files
Copy link
Member Author

@tie tie Jul 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: before this change, we had fallback for ar that does not support response files. As far as I know, that is only the case for Darwin’s cctools, but stdenv already uses LLVM for most of the tools on Darwin.

isCCTools = true; # The fact ld64 is used instead of lld is why this isn’t `isLLVM`.

That is, I think we can assume that ar supports response files (unless there is some edge case I’m missing).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the future, ideally, we should update v8 package and avoid using nodejs as v8 library provider.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could set env.AR to "${lib.getBin llvm}/bin/llvm-ar" to play it really safe, but that’s not necessary.

Darwin bintools has been using llvm-ar since 23.11. Going forward, more tools will be switched to their LLVM versions as compatibility and stability improve. None are expected to switch back to their cctools versions.

popd
# copy v8 headers
@@ -290,14 +389,6 @@ let
platforms = platforms.linux ++ platforms.darwin;
mainProgram = "node";
knownVulnerabilities = optional (versionOlder version "18") "This NodeJS release has reached its end of life. See https://nodejs.org/en/about/releases/.";

# Node.js build system does not have separate host and target OS
# configurations (architectures are defined as host_arch and target_arch,
# but there is no such thing as host_os and target_os).
#
# We may be missing something here, but it doesn’t look like it is
# possible to cross-compile between different operating systems.
broken = stdenv.buildPlatform.parsed.kernel.name != stdenv.hostPlatform.parsed.kernel.name;
};

passthru.python = python; # to ensure nodeEnv uses the same version
2 changes: 2 additions & 0 deletions pkgs/development/web/nodejs/v18.nix
Original file line number Diff line number Diff line change
@@ -26,6 +26,8 @@ buildNodejs {
version = "18.20.4";
sha256 = "sha256-p2x+oblq62ljoViAYmDICUtiRNZKaWUp0CBUe5qVyio=";
patches = [
./configure-emulator-node18.patch
./configure-armv6-vfpv2.patch
./disable-darwin-v8-system-instrumentation.patch
./bypass-darwin-xcrun-node16.patch
./revert-arm64-pointer-auth.patch
2 changes: 2 additions & 0 deletions pkgs/development/web/nodejs/v20.nix
Original file line number Diff line number Diff line change
@@ -15,6 +15,8 @@ buildNodejs {
version = "20.15.1";
sha256 = "sha256-/dU6VynZNmkaKhFRBG+0iXchy4sPyir5V4I6m0D+DDQ=";
patches = [
./configure-emulator.patch
./configure-armv6-vfpv2.patch
./disable-darwin-v8-system-instrumentation-node19.patch
./bypass-darwin-xcrun-node16.patch
./node-npm-build-npm-package-logic.patch
2 changes: 2 additions & 0 deletions pkgs/development/web/nodejs/v22.nix
Original file line number Diff line number Diff line change
@@ -15,6 +15,8 @@ buildNodejs {
version = "22.4.1";
sha256 = "sha256-ZfyFf1qoJWqvyQCzRMARXJrq4loCVB/Vzg29Tf0cX7k=";
patches = [
./configure-emulator.patch
./configure-armv6-vfpv2.patch
./disable-darwin-v8-system-instrumentation-node19.patch
./bypass-darwin-xcrun-node16.patch
./node-npm-build-npm-package-logic.patch