Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gyp: binding.gyp not found (...) while trying to load binding.gyp #156

Closed
PowerSupply opened this issue Sep 12, 2023 · 34 comments
Closed

gyp: binding.gyp not found (...) while trying to load binding.gyp #156

PowerSupply opened this issue Sep 12, 2023 · 34 comments

Comments

@PowerSupply
Copy link

PowerSupply commented Sep 12, 2023

npm i @parcel/watcher fails with the error in the title when trying to install right after npm init. I'm sure this has something to do with my environment but I do not understand what is wrong.

node: 18.15.0 (installed via nvm)
npm: 9.5.0
Python: 3.10.4
OS: macOS Ventura 13.4
Computer: Apple M1 Macbook Pro, 16-inch 2021

Full error:

npm ERR! code 1
npm ERR! path /Users/myuser/Documents/Dev/testgyp/node_modules/@parcel/watcher
npm ERR! command failed
npm ERR! command sh -c node-gyp rebuild
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@9.3.1
npm ERR! gyp info using node@18.15.0 | darwin | arm64
npm ERR! gyp info find Python using Python version 3.10.4 found at "/Users/myuser/.pyenv/versions/3.10.4/bin/python3"
npm ERR! gyp info spawn /Users/myuser/.pyenv/versions/3.10.4/bin/python3
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args   '/Users/myuser/.nvm/versions/node/v18.15.0/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args   'binding.gyp',
npm ERR! gyp info spawn args   '-f',
npm ERR! gyp info spawn args   'make',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/Users/myuser/Documents/Dev/testgyp/node_modules/@parcel/watcher/build/config.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/Users/myuser/.nvm/versions/node/v18.15.0/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/Users/myuser/Library/Caches/node-gyp/18.15.0/include/node/common.gypi',
npm ERR! gyp info spawn args   '-Dlibrary=shared_library',
npm ERR! gyp info spawn args   '-Dvisibility=default',
npm ERR! gyp info spawn args   '-Dnode_root_dir=/Users/myuser/Library/Caches/node-gyp/18.15.0',
npm ERR! gyp info spawn args   '-Dnode_gyp_dir=/Users/myuser/.nvm/versions/node/v18.15.0/lib/node_modules/npm/node_modules/node-gyp',
npm ERR! gyp info spawn args   '-Dnode_lib_file=/Users/myuser/Library/Caches/node-gyp/18.15.0/<(target_arch)/node.lib',
npm ERR! gyp info spawn args   '-Dmodule_root_dir=/Users/myuser/Documents/Dev/testgyp/node_modules/@parcel/watcher',
npm ERR! gyp info spawn args   '-Dnode_engine=v8',
npm ERR! gyp info spawn args   '--depth=.',
npm ERR! gyp info spawn args   '--no-parallel',
npm ERR! gyp info spawn args   '--generator-output',
npm ERR! gyp info spawn args   'build',
npm ERR! gyp info spawn args   '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp: binding.gyp not found (cwd: /Users/myuser/Documents/Dev/testgyp/node_modules/@parcel/watcher) while trying to load binding.gyp
npm ERR! gyp ERR! configure error 
npm ERR! gyp ERR! stack Error: `gyp` failed with exit code: 1
npm ERR! gyp ERR! stack     at ChildProcess.onCpExit (/Users/myuser/.nvm/versions/node/v18.15.0/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:325:16)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:513:28)
npm ERR! gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:291:12)
npm ERR! gyp ERR! System Darwin 22.5.0
npm ERR! gyp ERR! command "/Users/myuser/.nvm/versions/node/v18.15.0/bin/node" "/Users/myuser/.nvm/versions/node/v18.15.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd /Users/myuser/Documents/Dev/testgyp/node_modules/@parcel/watcher
npm ERR! gyp ERR! node -v v18.15.0
npm ERR! gyp ERR! node-gyp -v v9.3.1
npm ERR! gyp ERR! not ok
@r-r-lima
Copy link

Having the same issue since at least Aug 16:
node: v18.17.1
npm: v9.6.7
Tried downgrading to node 16.15 (npm 8.11), same error.
Also tried forcing @parcel/watcher version to 2.1 and 2.3.

@r-r-lima
Copy link

r-r-lima commented Sep 15, 2023

In my case @parcel/watcher is a direct dependency of another package. Solved by fixing version on that dependency:

"overrides": {
"package-that-uses-parcel": {
"@parcel/watcher": "2.1.0"
}
}

@fouteox
Copy link

fouteox commented Sep 21, 2023

@devongovett

Hi, I use nuxt which uses this package.

I'm using nuxt behind a proxy and in a node18-alpine3.17 docker container.

Since version 2.2.0, the "npm install" command, when it arrives at the @parcel/watcher module, tries to compile using python and gyp which itself tries for an unknown reason to download node despite the fact that it is already available locally since I'm using a node container.

My proxy refuses the download.

The problem is also present in all other docker images, whether node20-alpine or node20-debian.

The temporary solution is r-r-lima's answer, to do an ovverides to version 2.1.0

Is it possible to resolve this problem?

@edmondweiss
Copy link

I have the same exact error when installing Nuxt.

@PowerSupply
Copy link
Author

So, is anything happening with this?

@mischnic
Copy link
Member

mischnic commented Oct 2, 2023

Someone needs to investigate why the prebuilds (e.g. @parcel/watcher-darwin-arm64) are not working sometimes. I can't reproduce it on my machine

In https://unpkg.com/browse/@parcel/watcher@2.3.0/index.js, print out the err thrown by binding = require(name);.

@javiercno
Copy link

I have the same exact error when installing Nuxt.

@lovell
Copy link

lovell commented Oct 6, 2023

This looks suspiciously like the infamous npm lockfile bug - see npm/cli#2606 - possible workaround in npm/cli#2606 (comment)

@kmashint
Copy link

kmashint commented Oct 7, 2023

I've had similar problems. I had to use "@parcel/watcher": "~2.1.0" in my project devDependencies to avoid problems with 'gyp: binding.gyp not found' on both Windows 10 with npm 18 or 16 and Python 3.11, and MacOS 12.6.5 with npm 18 or 16 and Python 3.10. The problems seem to start in "@parcel/watcher": "~2.2.0" with its "node-addon-api": "^7.0.0" dependency, whereas "@parcel/watcher": "~2.1.0" with node-addon-api: ^3.2.1 and node-gyp-build: ^4.3.0 dependencies works.

@javiercno
Copy link

Solution temp

package.json

"overrides": {
    "@parcel/watcher": "2.1.0"
}

@lukecbt
Copy link

lukecbt commented Dec 1, 2023

Experiencing the same while attempting an npm install on a Nuxt 3 project. A few others in my team are also experiencing this but some are not.

@lancecaraccioli
Copy link

+1 npx nuxi init (Nuxt 3 app generator) fails due this error. Package override works as a temp fix.

@qaqland
Copy link

qaqland commented Dec 24, 2023

+1 npm install -D parcel

node: 21.4.0 (installed via apk)
npm: 10.2.5
Python: 3.11.6
OS: Alpine Linux 3.19 x86-64

@wavedeck
Copy link

+1 Updated package.lock in nuxt 3 and thought it’s the infamous npm cache permissions bug. But even after chowning and reinstalling node, the error stayed. Turns out it was due to this package.

Tested overrides with all sorts of in-between versions. The first version to work is 2.1.0

Env:
Node 21.6
OS macOS Sonoma
Arch arm64 (M1 Pro)

CallumWatkins added a commit to CallumWatkins/website that referenced this issue Jan 27, 2024
Override @parcel/watcher to 2.1.0 to work around parcel-bundler/watcher#156
@andreasjhagen
Copy link

Can confirm that this causes fresh nuxt installations to fail when using npx nuxi init.
Quite annoying for people that are new to the framework since they won't know they have to override the package.json to fix it

@MikPisula
Copy link

Can reproduce this issue as well when running npx nuxi@latest init.

$ node --version
v20.10.0
$ node-gyp --version
v10.0.1
$ npm --version
10.4.0
$ uname -a
Linux 6.7.1-arch1-1 #1 SMP PREEMPT_DYNAMIC Sun, 21 Jan 2024 22:14:10 +0000 x86_64 GNU/Linux

Running with the override worked as expected.

@mcfarljw
Copy link

I ran npm ls @parcel/watcher and it looks like nitropack is using a dependency which is causing the issue.

  ├── @parcel/watcher@2.1.0 overridden
  └─┬ nitropack@2.8.1
    └─┬ listhen@1.6.0
      └── @parcel/watcher@2.1.0 deduped

@danielweck
Copy link

Same here. My workaround:

  "overrides": {
    "@parcel/watcher": "~2.1.0"
  }

@TheDirigible
Copy link

"@parcel/watcher": "2.1.0"

First time trying Nuxt today, and I get this :(

@vincesp
Copy link

vincesp commented Feb 5, 2024

Someone needs to investigate why the prebuilds (e.g. @parcel/watcher-darwin-arm64) are not working sometimes. I can't reproduce it on my machine

In https://unpkg.com/browse/@parcel/watcher@2.3.0/index.js, print out the err thrown by binding = require(name);.

@mischnic Here are 2 Dockerfiles for reproducing the error.

Failing Dockerfile

FROM ubuntu:latest

RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
  apt-get update \
  && apt-get install -y --no-install-recommends \
    curl \
    gnupg \
    ca-certificates \
    wget \
    g++ \
  && curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \
  && apt-get update \
  && apt-get install -y --no-install-recommends nodejs \
  && npm install -g npm@latest \
  && rm -rf /var/lib/apt/lists/*

WORKDIR /home

Build and run:

docker build -t npm-20.
docker run -it --rm npm-20

Then, inside the running container:

npm i -g @parcel/watcher

❌ Error:

npm ERR! gyp: binding.gyp not found (cwd: /usr/lib/node_modules/@parcel/watcher) while trying to load binding.gyp

Working Dockerfile

FROM ubuntu:latest

RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
  apt-get update \
  && apt-get install -y --no-install-recommends \
    curl \
    gnupg \
    ca-certificates \
    wget \
    g++ \
  && curl -fsSL https://deb.nodesource.com/setup_18.x | bash - \
  && apt-get update \
  && apt-get install -y --no-install-recommends nodejs \
  && rm -rf /var/lib/apt/lists/*

WORKDIR /home

Build and run:

docker build -t npm-18 .
docker run -it --rm npm-18

Then, inside the running container:

npm i -g @parcel/watcher

✔️ Result:

added 13 packages in 3s

@vincesp
Copy link

vincesp commented Feb 5, 2024

I tried some more combinations of Node and npm:

Node 18.19.0 Node 20.11.0
npm v10.2.3 ✔️ ✔️
npm v10.2.4 ✔️ ✔️
npm v10.2.5 ✔️ ✔️
npm v10.3.0 ✔️ ✔️
npm v10.4.0

@componentscience
Copy link

componentscience commented Feb 5, 2024

Can reproduce this with 'npm install' in a SolidJS project.

$ node –version
v20.11.0
$ node-gyp –version
v10.0.1
$ npm –version
10.4.0
$ uname -a
Linux 6.7.3-arch1-1 #1 SMP PREEMPT_DYNAMIC Thu, 01 Feb 2024 10:30:35 +0000 x86_64 GNU/Linux

[host@myhost site]$ npm install
npm ERR! code 1
npm ERR! path /home/host/site/node_modules/@parcel/watcher
npm ERR! command failed
npm ERR! command sh -c node-gyp rebuild
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@10.0.1
npm ERR! gyp info using node@20.11.0 | linux | x64
npm ERR! gyp info find Python using Python version 3.11.6 found at "/sbin/python3"
npm ERR! gyp info spawn /sbin/python3
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args '/home/host/.nvm/versions/node/v20.11.0/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args 'binding.gyp',
npm ERR! gyp info spawn args '-f',
npm ERR! gyp info spawn args 'make',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/home/host/site/node_modules/@parcel/watcher/build/config.gypi',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/home/host/.nvm/versions/node/v20.11.0/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args '/home/host/.cache/node-gyp/20.11.0/include/node/common.gypi',
npm ERR! gyp info spawn args '-Dlibrary=shared_library',
npm ERR! gyp info spawn args '-Dvisibility=default',
npm ERR! gyp info spawn args '-Dnode_root_dir=/home/host/.cache/node-gyp/20.11.0',
npm ERR! gyp info spawn args '-Dnode_gyp_dir=/home/host/.nvm/versions/node/v20.11.0/lib/node_modules/npm/node_modules/node-gyp',
npm ERR! gyp info spawn args '-Dnode_lib_file=/home/host/.cache/node-gyp/20.11.0/<(target_arch)/node.lib',
npm ERR! gyp info spawn args '-Dmodule_root_dir=/home/host/site/node_modules/@parcel/watcher',
npm ERR! gyp info spawn args '-Dnode_engine=v8',
npm ERR! gyp info spawn args '--depth=.',
npm ERR! gyp info spawn args '--no-parallel',
npm ERR! gyp info spawn args '--generator-output',
npm ERR! gyp info spawn args 'build',
npm ERR! gyp info spawn args '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp: binding.gyp not found (cwd: /home/host/site/node_modules/@parcel/watcher) while trying to load binding.gyp
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: gyp failed with exit code: 1
npm ERR! gyp ERR! stack at ChildProcess. (/home/host/.nvm/versions/node/v20.11.0/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:271:18)
npm ERR! gyp ERR! stack at ChildProcess.emit (node:events:518:28)
npm ERR! gyp ERR! stack at ChildProcess._handle.onexit (node:internal/child_process:294:12)
npm ERR! gyp ERR! System Linux 6.7.3-arch1-1
npm ERR! gyp ERR! command "/home/host/.nvm/versions/node/v20.11.0/bin/node" "/home/host/.nvm/versions/node/v20.11.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd /home/host/site/node_modules/@parcel/watcher
npm ERR! gyp ERR! node -v v20.11.0
npm ERR! gyp ERR! node-gyp -v v10.0.1
npm ERR! gyp ERR! not ok

npm ERR! A complete log of this run can be found in: /home/host/.npm/_logs/2024-02-05T14_27_16_860Z-debug-0.log

@vincesp
Copy link

vincesp commented Feb 7, 2024

There is another combination that does not work: I did not find any npm version that can install @parcel/watcher successfully when using Artifactory 6 as an intermediate proxy repository. So if you have an .npmrc file like the following:

❌ Fails (even with npm v10.3.0):

registry=https://myartifactory6server.mycompany.com/artifactory/api/npm/NPM-REMOTE/
always-auth=true

We have to switch off the proxying explicitly for namespace @parcel but that means we lose the record in our own repo.

✔️ Works (with npm v10.3.0):

registry=https://myartifactory6server.mycompany.com/artifactory/api/npm/NPM-REMOTE/
@parcel:registry=https://registry.npmjs.org/
always-auth=true

johnbcodes added a commit to johnbcodes/rust-quote-editor that referenced this issue Feb 8, 2024
@meyerderek
Copy link

meyerderek commented Feb 11, 2024

I tried some more combinations of Node and npm:

Node 18.19.0 Node 20.11.0
npm v10.2.3 ✔️ ✔️
npm v10.2.4 ✔️ ✔️
npm v10.2.5 ✔️ ✔️
npm v10.3.0 ✔️ ✔️
npm v10.4.0 ❌ ❌

Confirming the downgrade to npm v10.3.0 fixed the issue for me while using Node v20.11.0

Was on npm v10.4.0 and trying to set up Analog.js using npm install @analogjs/platform --save-dev and ran into this error:

npm ERR! code 1
npm ERR! path /Users/<redacted>/Projects/<redacted>/website/node_modules/@parcel/watcher
npm ERR! command failed
npm ERR! command sh -c node-gyp rebuild
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@10.0.1
npm ERR! gyp info using node@20.11.0 | darwin | x64
npm ERR! gyp info find Python using Python version 3.7.3 found at "/Library/Developer/CommandLineTools/usr/bin/python3"

After downgrading, the install completed successfully.

@Saraswathi-Lemon
Copy link

I tried some more combinations of Node and npm:

Node 18.19.0 Node 20.11.0
npm v10.2.3 ✔️ ✔️
npm v10.2.4 ✔️ ✔️
npm v10.2.5 ✔️ ✔️
npm v10.3.0 ✔️ ✔️
npm v10.4.0 ❌ ❌

It works thanks!

@YuriFontella
Copy link

I tried some more combinations of Node and npm:

Node 18.19.0 Node 20.11.0
npm v10.2.3 ✔️ ✔️
npm v10.2.4 ✔️ ✔️
npm v10.2.5 ✔️ ✔️
npm v10.3.0 ✔️ ✔️
npm v10.4.0 ❌ ❌

node 21.6.1
npm 10.3.0

It worked for me

@AlexanderYW
Copy link

I tried some more combinations of Node and npm:

Node 18.19.0 Node 20.11.0
npm v10.2.3 ✔️ ✔️
npm v10.2.4 ✔️ ✔️
npm v10.2.5 ✔️ ✔️
npm v10.3.0 ✔️ ✔️
npm v10.4.0 ❌ ❌

node: 20.11.0
npm: 10.2.4, 10.4.0
Mac m1 Max: Sonoma 14.3.1

Didn't work for me.

What worked for me is specifying 2.1.0 as a version. 2.2.0, 2.3.0 and 2.4.0 didn't work at all.

@devongovett
Copy link
Member

I don't understand what we are supposed to do about this issue. Appears to be common but I don't think it's our fault? Seems like a bug in npm? Is that the consensus here? Has anyone reported this to npm yet? Does it work with other package managers (e.g. yarn/pnpm)?

@vincesp
Copy link

vincesp commented Feb 23, 2024

I don't understand what we are supposed to do about this issue. Appears to be common but I don't think it's our fault? Seems like a bug in npm? Is that the consensus here? Has anyone reported this to npm yet? Does it work with other package managers (e.g. yarn/pnpm)?

Unfortunately, our solution was to switch to watcher (just watcher without "@parcel/"). But that only worked because @parcel/watcher is not the dependency of a dependency.

@lovell
Copy link

lovell commented Feb 23, 2024

@devongovett PR #144 removed the install script from package.json as part of v2.2.0 however the npm registry still reports an install script:

$ npm show @parcel/watcher@2.1.0 --json | grep install
    "install": "node-gyp-build",
$ npm show @parcel/watcher@2.2.0 --json | grep install
    "install": "node-gyp rebuild"
$ npm show @parcel/watcher@latest --json | grep install
    "install": "node-gyp rebuild"

I think this erroneous install script is being injected at npm publish time because npm sees a binding.gyp file (thanks to the logic in @npmcli/node-gyp), however this file isn't published.

A few possible solutions, from hardest to easiest:

  • Fix @npmcli/node-gyp so it checks to see if binding.gyp will actually be published (e.g. look in files or .npmignore).
  • Move all the src and related build files, including binding.gyp, out of the root directory of this repo so it no longer looks like a native module (I did this for sharp, it works a treat, removes a whole class of error in most Electron tooling as well).
  • Remove the binding.gyp file from the filesystem just before running npm publish.

npm v10.4.0 introduced npm/run-script#192 which completely changes the way that package.json files are parsed at npm install time and is likely to have exposed this issue.

@devongovett
Copy link
Member

devongovett commented Feb 23, 2024

Great info @lovell! Thanks so much for sharing. It's interesting because this seems to have been happening even before npm 10.4 Perhaps that's actually a different issue (with lockfiles?) though.

I updated our publishing workflow to copy the files needed into a separate directory and run npm publish from there. That way binding.gyp doesn't get added by npm (even though it already wasn't included in package.json files).

In a related change, I also added better error handling to the package so that if an error occurs while requiring the native module, we only continue trying fallback locations if the error was a module not found error. Other errors (e.g. incompatibilities with OS versions) are re-thrown so they are reported to users.

These changes are published in v2.4.1. I tested locally with npm 10.4.0 and was able to install the package successfully. Please let me know if you continue to run into issues.

@bellaj
Copy link

bellaj commented Mar 11, 2024

only V.10.40 worked for me

m5r added a commit to medic/cht-conf that referenced this issue Jul 22, 2024
… have incompatibilities with npm 10.4 (see parcel-bundler/watcher#156) or the watcher hangs ~5% of the time when unsubscribing (see #623 (comment))
m5r added a commit to medic/cht-conf that referenced this issue Jul 22, 2024
* upgrade non-breaking dev dependencies

* why rewire...

* forgot .only

* upgrade prod dependencies

* silence grandpa eslint who cannot resolve a `require`

* not sure why test is passing locally but failing in CI

* .only

* debug CI

* debug CI - try force color in diff

* debug CI colors

* debug CI colors

* okay that fixed it, revert test back and remove logs

* .only

* remove unnecessary `{ color: true }` parameter

* comment eslint ignore

* sonar

* will this space make sonar happy?

* will this?

* this should.

* clamp `@parcel/watcher` to v2.1.0 because more recent versions either have incompatibilities with npm 10.4 (see parcel-bundler/watcher#156) or the watcher hangs ~5% of the time when unsubscribing (see #623 (comment))

* remove todos
@aznashwan
Copy link

For anyone experiencing this issue on Ubuntu 22.04 ARM64 and Node 16.20.1 installed as a snap (I need to re-package legacy stuff, don't ask), 2.2.0 is the first version I found builds fine:

// Add this to package.json
"overrides": {
    "@parcel/watcher": "2.2.0"
}

jjnaude added a commit to WildEyeConservation/Detweb that referenced this issue Oct 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.