Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] failed optional dependency builds should not prevent global install of packages #7355

Open
2 tasks done
rigor789 opened this issue Apr 9, 2024 · 5 comments
Open
2 tasks done
Labels
Bug thing that needs fixing Needs Triage needs review for next steps

Comments

@rigor789
Copy link

rigor789 commented Apr 9, 2024

Is there an existing issue for this?

  • I have searched the existing issues

This issue exists in the latest npm version

  • I am using the latest npm

Current Behavior

With the inclusion of #7126 in v10.4 global installs of packages that have transient dependencies on packages like fsevents started failing.

Example Logs

npm ERR! code 1
npm ERR! path /Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/fsevents
npm ERR! command failed
npm ERR! command sh -c node-gyp rebuild
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@10.1.0
npm ERR! gyp info using node@21.7.2 | darwin | arm64
npm ERR! gyp info find Python using Python version 3.9.6 found at "/Applications/Xcode-15.2.0.app/Contents/Developer/usr/bin/python3"
npm ERR! gyp info spawn /Applications/Xcode-15.2.0.app/Contents/Developer/usr/bin/python3
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args '/Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args 'binding.gyp',
npm ERR! gyp info spawn args '-f',
npm ERR! gyp info spawn args 'make',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/fsevents/build/config.gypi',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/node-gyp/addon.gypi',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/Users/rigor789/Library/Caches/node-gyp/21.7.2/include/node/common.gypi',
npm ERR! gyp info spawn args '-Dlibrary=shared_library',
npm ERR! gyp info spawn args '-Dvisibility=default',
npm ERR! gyp info spawn args '-Dnode_root_dir=/Users/rigor789/Library/Caches/node-gyp/21.7.2',
npm ERR! gyp info spawn args '-Dnode_gyp_dir=/Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/node-gyp',
npm ERR! gyp info spawn args '-Dnode_lib_file=/Users/rigor789/Library/Caches/node-gyp/21.7.2/<(target_arch)/node.lib',
npm ERR! gyp info spawn args '-Dmodule_root_dir=/Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/fsevents',
npm ERR! gyp info spawn args '-Dnode_engine=v8',
npm ERR! gyp info spawn args '--depth=.',
npm ERR! gyp info spawn args '--no-parallel',
npm ERR! gyp info spawn args '--generator-output',
npm ERR! gyp info spawn args 'build',
npm ERR! gyp info spawn args '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp: binding.gyp not found (cwd: /Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/fsevents) while trying to load binding.gyp
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: `gyp` failed with exit code: 1
npm ERR! gyp ERR! stack at ChildProcess.<anonymous> (/Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/node-gyp/lib/configure.js:297:18)
npm ERR! gyp ERR! stack at ChildProcess.emit (node:events:519:28)
npm ERR! gyp ERR! stack at ChildProcess._handle.onexit (node:internal/child_process:294:12)
npm ERR! gyp ERR! System Darwin 23.4.0
npm ERR! gyp ERR! command "/Users/rigor789/.nvm/versions/node/v21.7.2/bin/node" "/Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/.bin/node-gyp" "rebuild"
npm ERR! gyp ERR! cwd /Users/rigor789/.nvm/versions/node/v21.7.2/lib/node_modules/nativescript/node_modules/fsevents
npm ERR! gyp ERR! node -v v21.7.2
npm ERR! gyp ERR! node-gyp -v v10.1.0
npm ERR! gyp ERR! not ok

npm ERR! A complete log of this run can be found in: /Users/rigor789/.npm/_logs/2024-04-09T09_05_41_833Z-debug-0.log

Expected Behavior

Global installs should not fail. fsevents is an optional peerDependency, and installing it should not trigger a rebuild, nor fail on unsupported systems (like Linux/Windows).

Steps To Reproduce

  1. install npm 10.4 or newer
  2. run npm i -g nativescript
  3. the install will fail

To confirm the linked change breaks it:

  1. open node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js of your global packages
  2. edit line 1235 and comment out // fullMetadata: true,
  3. re-run: npm i -g nativescript
  4. the install works as expected.

Environment

  • npm: 10.4.0 (or newer)
  • Node.js: v21.7.2
  • OS Name: macOS Sonoma 14.4.1
  • System Model Name: MacBook Pro
  • npm config:
; "user" config from /Users/rigor789/.npmrc

//registry.npmjs.org/:_authToken = (protected)

; node bin location = /Users/rigor789/.nvm/versions/node/v21.7.2/bin/node
; node version = v21.7.2
; npm local prefix = /Users/rigor789/Code/
; npm version = 10.4.0
; cwd = /Users/rigor789/Code/
; HOME = /Users/rigor789
; Run `npm config ls -l` to show all defaults.
@rigor789
Copy link
Author

rigor789 commented Apr 9, 2024

If it's helpful, here are the manifests before/after including fullMetadata: true

Without `fullMetadata: true`

{
  name: 'fsevents',
  version: '2.3.3',
  devDependencies: { 'node-gyp': '^9.4.0' },
  dist: {
    integrity: 'sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==',
    shasum: 'cac6407785d03675a2a5e1a5305c697b347d90d6',
    tarball: 'https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz',
    fileCount: 6,
    unpackedSize: 173224,
    signatures: [ [Object] ]
  },
  engines: { node: '^8.16.0 || ^10.6.0 || >=11.0.0' },
  os: [ 'darwin' ],
  hasInstallScript: true,
  _id: 'fsevents@2.3.3',
  _resolved: 'https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz',
  _from: 'fsevents@~2.3.2',
  _integrity: 'sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==',
  _signatures: [
    {
      keyid: 'SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA',
      sig: 'MEUCIBqKmzRJwFJcSLXc/SdNTdFsUQvsS/tX+pFHr2JkniDbAiEAiTjoxV3W2IK2g8Udbes/ZP46+TtL40PX3IX/bxosfWs='
    }
  ]
}

With `fullMetadata: true`

{
  name: 'fsevents',
  version: '2.3.3',
  description: 'Native Access to MacOS FSEvents',
  main: 'fsevents.js',
  types: 'fsevents.d.ts',
  os: [ 'darwin' ],
  engines: { node: '^8.16.0 || ^10.6.0 || >=11.0.0' },
  scripts: {
    clean: 'node-gyp clean && rm -f fsevents.node',
    build: 'node-gyp clean && rm -f fsevents.node && node-gyp rebuild && node-gyp clean',
    test: '/bin/bash ./test.sh 2>/dev/null',
    prepublishOnly: 'npm run build',
    install: 'node-gyp rebuild'
  },
  repository: { type: 'git', url: 'git+https://github.com/fsevents/fsevents.git' },
  keywords: [ 'fsevents', 'mac' ],
  contributors: [
    { name: 'Philipp Dunkel', email: 'pip@pipobscure.com' },
    { name: 'Ben Noordhuis', email: 'info@bnoordhuis.nl' },
    { name: 'Elan Shankar', email: 'elan.shanker@gmail.com' },
    { name: 'Miroslav Bajtoš', email: 'mbajtoss@gmail.com' },
    { name: 'Paul Miller', url: 'https://paulmillr.com' }
  ],
  bugs: { url: 'https://github.com/fsevents/fsevents/issues' },
  homepage: 'https://github.com/fsevents/fsevents',
  devDependencies: { 'node-gyp': '^9.4.0' },
  gypfile: true,
  gitHead: '2db891e51aa0f2975c5eaaf6aa30f13d720a830a',
  _id: 'fsevents@2.3.3',
  _nodeVersion: '18.17.1',
  _npmVersion: '9.6.7',
  dist: {
    integrity: 'sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==',
    shasum: 'cac6407785d03675a2a5e1a5305c697b347d90d6',
    tarball: 'https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz',
    fileCount: 6,
    unpackedSize: 173224,
    signatures: [ [Object] ]
  },
  _npmUser: { name: 'pipobscure', email: 'pip@pipobscure.com' },
  directories: {},
  maintainers: [
    { name: 'pipobscure', email: 'pip@pipobscure.com' },
    { name: 'paulmillr', email: 'paul@paulmillr.com' }
  ],
  _npmOperationalInternal: {
    host: 's3://npm-registry-packages',
    tmp: 'tmp/fsevents_2.3.3_1692635062642_0.01197687980904627'
  },
  _hasShrinkwrap: false,
  _time: '2023-08-21T16:24:22.854Z',
  _resolved: 'https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz',
  _from: 'fsevents@~2.3.2',
  _integrity: 'sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==',
  _signatures: [
    {
      keyid: 'SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA',
      sig: 'MEUCIBqKmzRJwFJcSLXc/SdNTdFsUQvsS/tX+pFHr2JkniDbAiEAiTjoxV3W2IK2g8Udbes/ZP46+TtL40PX3IX/bxosfWs='
    }
  ]
}

@wraithgar
Copy link
Member

This appears to have been working by accident before. The minified packument does not have scripts which likely means that any package that required a build step would not work globally if we reverted that line.

This is still a bug, but the bug is that this should not be failing in global mode. Local mode gracefully catches this:

npm info run fsevents@2.3.3 install node_modules/fsevents node-gyp rebuild
npm info run fsevents@2.3.3 install { code: 1, signal: null }
npm verb reify failed optional dependency /Users/wraithgar/Development/npm/scratch/optional/node_modules/fsevents
npm info run nativescript@8.7.0 postinstall node_modules/nativescript node postinstall.js
npm info run nativescript@8.7.0 postinstall { code: 0, signal: null }

@wraithgar wraithgar changed the title [BUG] global installs unexpectedly rebuilding via node-gyp [BUG] failed optional dependency builds should not prevent global install of packages Apr 9, 2024
@rigor789
Copy link
Author

rigor789 commented Apr 9, 2024

Thanks for confirming, indeed that sounds more reasonable as an explanation - I wouldn't expect optional dependencies to fail a global install.

I do wonder when that install: 'node-gyp rebuild' is being added however, because it's not in the published package.json of fsevents:

"scripts": {
  "clean": "node-gyp clean && rm -f fsevents.node",
  "build": "node-gyp clean && rm -f fsevents.node && node-gyp rebuild && node-gyp clean",
  "test": "/bin/bash ./test.sh 2>/dev/null",
  "prepublishOnly": "npm run build"
},

(from https://www.npmjs.com/package/fsevents?activeTab=code)

@wraithgar
Copy link
Member

It's added automatically by npm during publish because it detects no existing install script but it finds a binding.gyp file. You can see an old issue where folks had similar problems here: #5234

@koiker
Copy link

koiker commented Aug 7, 2024

Got the same issue.
The fix was to downgrade npm:
npm i -g npm@~10.3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug thing that needs fixing Needs Triage needs review for next steps
Projects
None yet
Development

No branches or pull requests

4 participants