-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] Platform-specific optional dependencies not being included in package-lock.json
when reinstalling with node_modules
present
#4828
Comments
node_modules
present
node_modules
presentpackage-lock.json
when reinstalling with node_modules
present
Sorry to ping you out of the blue, but this issue has been open for 11 days now without any movement. Is there anyone working on npm right now that might have the bandwidth to at least validate that this is indeed a problem as I've described it? Just so that when someone does become available to do some development work they know that this is in the queue? Please and thank you. |
Bump |
I'm also encountering this issue with a Next.js project:
Unfortunately developers often don't realise the Here is a reproduction:
|
I am also having this issue. I'm trying to run tests using jest with swc. The test runner is a linux image, but my dev machine is darwin. I can get it to work by either using --force to install the linux dependency, or I can install packages from inside the container... but github CI stands up the docker container in such a way that I can't easily install packages from in there, and that also prevents me from maintaining a cached node modules etc. |
bump |
bump - cannot get optional dependencies (namely |
bump |
Confirming that this issue is still present. It's particularly important for projects using NAPI modules, as tons of them use platform-specific packages. |
Ran into this issue when creating a CI process for a repo where I use a Windows machine and the CI process is using Linux. My quick "fix" for now is to start the CI process by deleting the package-lock.json and running npm install instead of npm ci. I know this is not good practice, so looking forward to a real fix to come through. |
bump |
I am having a similar issue. My project uses What has changed between v6 and v8 and is there an npm config option that will have v8 work similar to v6 when it comes to optional dependencies? |
I encountered a similar issue, and here’s how I resolved it. I’m working on a MacBook Pro with an M2 Pro chip, while my GitLab pipeline runs on a Linux (x64) environment. In my case, the problem stemmed from the "optionalDependencies": {
"@rollup/rollup-android-arm-eabi": "4.22.4",
"@rollup/rollup-android-arm64": "4.22.4",
"@rollup/rollup-darwin-arm64": "4.22.4",
"@rollup/rollup-darwin-x64": "4.22.4",
"@rollup/rollup-linux-arm-gnueabihf": "4.22.4",
"@rollup/rollup-linux-arm-musleabihf": "4.22.4",
"@rollup/rollup-linux-arm64-gnu": "4.22.4",
"@rollup/rollup-linux-arm64-musl": "4.22.4",
"@rollup/rollup-linux-powerpc64le-gnu": "4.22.4",
"@rollup/rollup-linux-riscv64-gnu": "4.22.4",
"@rollup/rollup-linux-s390x-gnu": "4.22.4",
"@rollup/rollup-linux-x64-gnu": "4.22.4",
"@rollup/rollup-linux-x64-musl": "4.22.4",
"@rollup/rollup-win32-arm64-msvc": "4.22.4",
"@rollup/rollup-win32-ia32-msvc": "4.22.4",
"@rollup/rollup-win32-x64-msvc": "4.22.4",
"fsevents": "~2.3.2"
} This package list is required by: In my To ensure consistency, I deleted the To address this, I deleted If you don’t mind deleting Best of luck! |
To clarify, now my ......
"node_modules/@rollup/rollup-darwin-arm64": {
"version": "4.22.4",
"resolved": "xxx",
"integrity": "xxx",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
]
},
"node_modules/@rollup/rollup-darwin-x64": {
"version": "4.22.4",
"resolved": "xxx",
"integrity": "xxx",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
]
},
"node_modules/@rollup/rollup-linux-arm-gnueabihf": {
"version": "4.22.4",
"resolved": "xxx",
"integrity": "xxx",,
"cpu": [
"arm"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
]
},
"node_modules/@rollup/rollup-linux-arm-musleabihf": {
"version": "4.22.4",
"resolved": "xxx",
"integrity": "xxx",
"cpu": [
"arm"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
]
},
"node_modules/@rollup/rollup-linux-arm64-gnu": {
"version": "4.22.4",
"resolved": "xxx",
"integrity": "xxx",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
]
},
etc .... Before, these packages were missing in the |
Thanks for this Infos. I will give it a try because I am experiencing similar issue. |
In the end it worked for me to add the following optional dependencies to get the standard angular 18 project running on my windows machine and the linux pipeline:
As I had a fresh project, I regenerated the complete |
* easylist experiment * update easylist * Tweak easylist hack * delay easylist detection * increase the easylist timeout * update easylist * Use fanboy-cookiemonster in the test extension * Initialize and apply cosmetic filters together with prehide rules * Fall back to filterist when no pop-up is found * Remove easylist rule * WIP * WIP: add perf metrics * add performance metrics * Allow disabling autoconsent per site (in the test extension) * Use uBO version of easylist cookie * remove the perf metrics code * Use constructed stylesheets for cosmetic styles * minify the extension content script * filterlist experiment * Add filterlist overrides * Update bundled filterlist * minor filterlist changes * Logging tweaks * Tweak logs * Add a rule for wise.com * add a rule for nike * Add a rule for dan.com * Remove generic-cosmetic rule (too many false positives) * add rule for medium.com * Add a rule for abc.net.au * Lint fix * removed unused file * support mobile aliexpress * Update rule for temu * Add rule for american airlines * Add rule for tesla * Wrap filterlist code in try catch * Add a rule for admiral GDPR popups * Tweak ensighten rule for britishairways.com * Address minor PR comments * comment overrides * update the adblocker library and use the provided style override * Bundle filterlist in JS * Update filterlist on every release * Tweak watch command * Remove outdated comments * Disable filterlist by default * Move log to a more appropriate place * remove unused ignore * Lint fix in auto-generated file * Update filterlist * Remove autogenerated filterlist from git index * Do not commit changes to filterlist * Update the build scripts to update filterlist when necessary * Regenerate package-lock (see npm/cli#4828) * remove performance marks * Produce separate builds with and without filterlist * Avoid generating filterlist twice in ci * Export json rules * Do not commit filterlist file during release * Build filterlist from easylist source * Update readme * add link to DDG download page * cliqz/adblocker is renamed to ghostery/adblocker * Address minor PR comments * Track the resulting filterlist in git * Add a ci job to update EasyList
I tried both optionalDependencies and dependencies. Doesn't work. Fails with the same error: ``` /private/var/tmp/_bazel_joe/1713d43747a35eb7e2009bc8fd9406b7/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/vite_/vite.runfiles/_main/node_modules/.aspect_rules_js/rollup@4.24.3/node_modules/rollup/dist/native.js:63 throw new Error( ^ Error: Cannot find module @rollup/rollup-darwin-arm64. npm has a bug related to optional dependencies (npm/cli#4828). Please try `npm i` again after removing both package-lock.json and node_modules directory. at requireWithFriendlyError (/private/var/tmp/_bazel_joe/1713d43747a35eb7e2009bc8fd9406b7/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/vite_/vite.runfiles/_main/node_modules/.aspect_rules_js/rollup@4.24.3/node_modules/rollup/dist/native.js:63:9) at Object.<anonymous> (/private/var/tmp/_bazel_joe/1713d43747a35eb7e2009bc8fd9406b7/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/vite_/vite.runfiles/_main/node_modules/.aspect_rules_js/rollup@4.24.3/node_modules/rollup/dist/native.js:72:76) ... 3 lines matching cause stack trace ... at Module._load (node:internal/modules/cjs/loader:1104:12) at cjsLoader (node:internal/modules/esm/translators:346:17) at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:286:7) at ModuleJob.run (node:internal/modules/esm/module_job:234:25) at async ModuleLoader.import (node:internal/modules/esm/loader:473:24) ```
Results in same error. Rollup declares an optional dependency on the arm64 variant. ``` pnpm install && bazelisk run //:vite -- --help private/var/tmp/_bazel_joe/1713d43747a35eb7e2009bc8fd9406b7/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/vite_/vite.runfiles/_main/node_modules/.aspect_rules_js/rollup@4.24.3/node_modules/rollup/dist/native.js:63 throw new Error( ^ Error: Cannot find module @rollup/rollup-darwin-arm64. npm has a bug related to optional dependencies (npm/cli#4828). Please try `npm i` again after removing both package-lock.json and node_modules directory. at requireWithFriendlyError (/private/var/tmp/_bazel_joe/1713d43747a35eb7e2009bc8fd9406b7/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/vite_/vite.runfiles/_main/node_modules/.aspect_rules_js/rollup@4.24.3/node_modules/rollup/dist/native.js:63:9) ``` It looks like rules_js doesn't link optional dependencies and also doesn't use the package.json `pnpm.packageExtensions` override. ``` ls -lh $BAZEL_OUT/darwin_arm64-fastbuild/bin/vite_/vite.runfiles/_main/node_modules/.aspect_rules_js/rollup@4.24.4/node_modules drwxr-xr-x@ 3 joe wheel 96B Nov 7 13:35 @types lrwxr-xr-x@ 1 joe wheel 176B Nov 7 13:35 rollup -> $BAZEL_OUT/darwin_arm64-fastbuild/bin/node_modules/.aspect_rules_js/rollup@4.24.4/node_modules/rollup ``` https://bazelbuild.slack.com/archives/CEZUUKQ6P/p1731014606302199?thread_ts=1730916056.708769&cid=CEZUUKQ6P
Error: Cannot find module @rollup/rollup-linux-x64-gnu. npm has a bug related to optional dependencies (npm/cli#4828). Please try `npm i` again after removing both package-lock.json and node_modules directory.
Cannot find module @rollup/rollup-linux-x64-gnu.(npm/cli#4828).
…4828 I deleted package-lock.json and node_modules, then ran `npm install --platform=linux --arch=x64 --include=optional`
…4828 I deleted package-lock.json and node_modules, then ran `npm install --platform=linux --arch=x64 --include=optional`
… related to optional dependencies (npm/cli#4828). Please try `npm i` again after removing both package-lock.json and node_modules directory.
* Upgrade all dependencies * Clean up package-lock.json * Regenerate package-lock.json due to npm/cli#4828 * Downgrade chai
bump |
Is there an existing issue for this?
This issue exists in the latest npm version
Current Behavior
I'm working on a team that utilizes a mix of x64-based and m1-based macs, and has CI build processes that uses musl. We're seeing that
npm
is skipping platform-specific optional dependencies for packages such as@swc/core
as a result of thepackage-lock.json
file being generated without all of them included. In our case, this then causes linting to throw an exception, because one of our eslint plugins depends on @swc, which depends on having the platform specific @swc package also installed.There seems to be at least two stages of cause to this. Firstly, when installing
@swc/core
from a clean slate working directorynpm
generates apackage-lock.json
with all of the optional dependencies for@swc/core
listed:And it only installs the platform specific package:
If I then remove my
package-lock.json
, leave mynode_modules
directory as-is, and then reinstall, I get:That is, it then generates a package-lock.json with only the platform-specific dependency that was installed on this machine, and not with the other optional dependencies that should also be listed.
If you delete both
node_modules
ANDpackage-lock.json
, and then re-runnpm install
, it generates the correct lockfile with all of those optional dependencies listed.The problem is that then, If the
package-lock.json
with the missing optional platform-specific dependencies gets checked into git and an x64 user pulls it down, or vice-versa,npm
fails to detect that your platform's optional dependencies are missing in the lockfile and just silently skips installing the platform-specific dependency. For example, when I've got a package-lock.json that only contains the x64 @swc package because of the above problem (generated by my coworker on his x64 machine):And I then install:
You can see that it fails to install the arm64 dependency or warn me in any way that the
package-lock.json
is missing my platform's dependency.So yeah, two problems:
Expected Behavior
npm
should preserve the full set of platform-specific optional deps for a package like @swc when rebuildingpackage-lock.json
from an existingnode_modules
treenpm install
should warn if thepackage-lock.json
becomes inconsistent because of the first caseSteps To Reproduce
See above.
Environment
The text was updated successfully, but these errors were encountered: