[RRFC] Support --cpu and --os flag to specify platform specific install

[archfz]

Motivation ("The Why")

Given the os and cpu package.json attributes, and the rise on usage of optionalDependencies for native modules, and also given cross platform building of certain applications like electron, it would be nice to be able to install npm packages for a given platform, disregarding current platform.

Example

My case is having a thrid party native module using optionalDependencies to install platform specific node bindings for itself. One good example for this is napi-rs modules (see https://github.com/napi-rs/package-template). Now this module is bundled together with an electron app, and the build pipeline uses wine to build it for windows as well from debian. The issue is that the thrid party native module will always be installed with linux native bindings.

How

Current Behaviour

Currently optionalDependencies only matching current platform will be installed, and it's not configurable.

Desired Behaviour

The proposal is to have two extra possible arguments for npm install. --os would specify for what operating system to install packages, --cpu for what cpu. The defaults for these options would be from current platform.

Example: npm i --os win32 --cpu x64

References

• n/a

[ljharb]

A package that requires a specific platform likely needs to be compiled for that platform. Why do you have the expectation that a package would work on a different one?

[archfz]

@ljharb The main point here is cross platform build. One can basically build electron targeted for windows and linux, all from a linux platform (using wine). You can check how this is supported with electron-builder https://www.electron.build/multi-platform-build

[ljharb]

@archfz well sure, but in that case wouldn't it declare the OS and CPU combos it was compatible with?

I'm still unclear on why you'd want to pretend the platform is X, when it's Y - if the package works on Y, it'll declare it; if it doesn't, then it won't likely work even if you pretend it's X.

[archfz]

@ljharb Building electron works something like this:

1. bundle rendered code -> non platform specific / runs in browser
2. bundle main code -> non platform specific / runs in node
3. bundle rest -> platform specific / this is like the installer or executable that is generated for the app, that will run the electron backend and browser

Now on step 2 the code can basically require 3rd party node packages. These packages are basically included as they are into the installer / executable. These packages can be native modules, and they can use the optionalDependencies + os/cpu in package json way of handling the platform specific node bindings installation. Let's say this is package X. Then there is an issue with cross platform building, because I have built electron for win32 from linux, but package X has been installed with bindings for linux and not win32. Given I had an option to run npm i --os win32 before starting the cross platform build for win32, then the correct bindings would be installed for that package, and thus correct one included in build.

[archfz]

This feature request would achieve similar results as the --target.. options for node-pre-gyp (see https://github.com/mapbox/node-pre-gyp#options). The difference being that the modern native modules instead of using node-pre-gyp, they use optionalDependencies + os/cpu.

[yukukotani]

This options are useful for building Docker image.

In some cases, we want to run npm i and build the NodeJS artifacts on our local machine or CI environment, then copy them to the Docker image. The OS and CPU may be different between build time and runtime, so we want to specify them.

[timfish]

How would this work with dependencies and devDependecies?

If I'm building on arm64, targeting x64 and have development dependencies with native modules, I'll likely need a mix.

I think it would be better to have an --ignore-os-cpu option which means the filters are ignored and all optionalDependencies get downloaded. That way they can be selected at runtime.

[MRayermannMSFT]

A package that requires a specific platform likely needs to be compiled for that platform. Why do you have the expectation that a package would work on a different one?

An example of a native package where this does not apply can be found here: https://www.npmjs.com/package/@node-rs/crc32

This package, and others in the node-rs scope all have optional dependencies on a variety of packages for different os/cpu combinations. Each of these sub packages contain pre-compiled binaries.

I myself have come to this issue after running in a problem that these flags would solve. Our macOS ARM64 CI builds on built on macOS x64 machines. This means that our CI builds fail to contain the darwin-arm64 dependency for crc32 (the linked package) because they instead install the darwin-x64 dependency.

[lovell]

It looks like this has been closed due to npm/cli#6755 implementing what appears to be support for an installation tree that can work on a single alternative target platform/arch.

However the comments made here and examples provided seem to be looking for an installation tree that can work on multiple platform/arch combinations, which is significantly more useful/powerful but perhaps more complex.

Does this RFC cover multiarch? There is prior art for this in the supportedArchitectures feature of yarn.

Most importantly, thank you all for your work on this so far, when complete it's going to make things so much easier for those of use who maintain native modules.