installation error on windows for node 15.3.0 (falls through to gyp, gyp errors out)

[Pomax]

I'm trying to install node-sqlite3 on win10 pro x64 with node 15.3.0 and python 3.7.7, and things are not well...

npm ERR! code 1
npm ERR! path c:\Users\Mike\Documents\git\projects\lightroom-catalog-helper\node_modules\sqlite3
npm ERR! command failed
npm ERR! command C:\WINDOWS\system32\cmd.exe /d /s /c node-gyp rebuild
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@7.1.2
npm ERR! gyp info using node@15.3.0 | win32 | x64
npm ERR! gyp info find Python using Python version 3.7.7 found at "C:\Program Files\Python37\python.exe"
npm ERR! gyp http GET https://nodejs.org/download/release/v15.3.0/node-v15.3.0-headers.tar.gz
npm ERR! gyp http 200 https://nodejs.org/download/release/v15.3.0/node-v15.3.0-headers.tar.gz
npm ERR! gyp http GET https://nodejs.org/download/release/v15.3.0/SHASUMS256.txt
npm ERR! gyp http GET https://nodejs.org/download/release/v15.3.0/win-arm64/node.lib
npm ERR! gyp http GET https://nodejs.org/download/release/v15.3.0/win-x64/node.lib
npm ERR! gyp http GET https://nodejs.org/download/release/v15.3.0/win-x86/node.lib
npm ERR! gyp http 200 https://nodejs.org/download/release/v15.3.0/win-x86/node.lib
npm ERR! gyp http 200 https://nodejs.org/download/release/v15.3.0/SHASUMS256.txt
npm ERR! gyp http 404 https://nodejs.org/download/release/v15.3.0/win-arm64/node.lib
npm ERR! gyp http 200 https://nodejs.org/download/release/v15.3.0/win-x64/node.lib
npm ERR! gyp info find VS using VS2019 (16.5.29911.84) found at:
npm ERR! gyp info find VS "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools"
npm ERR! gyp info find VS run with --verbose for detailed information
npm ERR! gyp info spawn C:\Program Files\Python37\python.exe
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args   'C:\\Users\\Mike\\AppData\\Roaming\\nvm\\v15.3.0\\node_modules\\npm\\node_modules\\node-gyp\\gyp\\gyp_main.py',
npm ERR! gyp info spawn args   'binding.gyp',
npm ERR! gyp info spawn args   '-f',
npm ERR! gyp info spawn args   'msvs',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   'c:\\Users\\Mike\\Documents\\git\\projects\\lightroom-catalog-helper\\node_modules\\sqlite3\\build\\config.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   'C:\\Users\\Mike\\AppData\\Roaming\\nvm\\v15.3.0\\node_modules\\npm\\node_modules\\node-gyp\\addon.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   'C:\\Users\\Mike\\AppData\\Local\\node-gyp\\Cache\\15.3.0\\include\\node\\common.gypi',
npm ERR! gyp info spawn args   '-Dlibrary=shared_library',
npm ERR! gyp info spawn args   '-Dvisibility=default',
npm ERR! gyp info spawn args   '-Dnode_root_dir=C:\\Users\\Mike\\AppData\\Local\\node-gyp\\Cache\\15.3.0',
npm ERR! gyp info spawn args   '-Dnode_gyp_dir=C:\\Users\\Mike\\AppData\\Roaming\\nvm\\v15.3.0\\node_modules\\npm\\node_modules\\node-gyp',
npm ERR! gyp info spawn args   '-Dnode_lib_file=C:\\\\Users\\\\Mike\\\\AppData\\\\Local\\\\node-gyp\\\\Cache\\\\15.3.0\\\\<(target_arch)\\\\node.lib',
npm ERR! gyp info spawn args   '-Dmodule_root_dir=c:\\Users\\Mike\\Documents\\git\\projects\\lightroom-catalog-helper\\node_modules\\sqlite3',
npm ERR! gyp info spawn args   '-Dnode_engine=v8',
npm ERR! gyp info spawn args   '--depth=.',
npm ERR! gyp info spawn args   '--no-parallel',
npm ERR! gyp info spawn args   '--generator-output',
npm ERR! gyp info spawn args   'c:\\Users\\Mike\\Documents\\git\\projects\\lightroom-catalog-helper\\node_modules\\sqlite3\\build',
npm ERR! gyp info spawn args   '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp: Undefined variable module_name in binding.gyp while trying to load binding.gyp
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: `gyp` failed with exit code: 1
npm ERR! gyp ERR! stack     at ChildProcess.onCpExit (C:\Users\Mike\AppData\Roaming\nvm\v15.3.0\node_modules\npm\node_modules\node-gyp\lib\configure.js:351:16)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:376:20)
npm ERR! gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:277:12)
npm ERR! gyp ERR! System Windows_NT 10.0.19041
npm ERR! gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\Mike\\AppData\\Roaming\\nvm\\v15.3.0\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd c:\Users\Mike\Documents\git\projects\lightroom-catalog-helper\node_modules\sqlite3
npm ERR! gyp ERR! node -v v15.3.0
npm ERR! gyp ERR! node-gyp -v v7.1.2
npm ERR! gyp ERR! not ok

[springmeyer]

I've tested node v15.3.0 and it works just fine with the latest node-sqlite3@5.0.0 on OS X (and I presume also linux). I don't have access to windows to debug, but I think what is happening is that some regression happened recently such that the node-sqlite3 appveyor builds are not longer publishing windows binaries successfully. So, when you go to install on windows the build falls back to a source compile. But it appears that you are calling node-gyp directly - that won't work: node-pre-gyp needs to pass the module_name to node-gyp when node-pre-gyp is called by npm install. So, can you try just doing:

npm install sqlite3

And let us know if that works and if not what the error is?

[Pomax]

At the time of filing, node-pre-gryp was failing its builds, leading to the error log from the issue comment, but this seems to have gotten fixed a few hours after your comment, and npm install now works again. Although, I should point out that the build for node-sqlite3 is still failing according to the status buttons linked in the readme (even if it installs correctly now using the prebuilt binary via node-pre-gyp):

[Pomax]

Hm, I spoke too soon:

c:\Users\Mike\Documents\git\projects\lightroom-catalog-helper>rm -rf node_modules

c:\Users\Mike\Documents\git\projects\lightroom-catalog-helper>npm install
npm ERR! code 1
npm ERR! path c:\Users\Mike\Documents\git\projects\lightroom-catalog-helper\node_modules\sqlite3
npm ERR! command failed
npm ERR! command C:\WINDOWS\system32\cmd.exe /d /s /c node-gyp rebuild
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@7.1.2
npm ERR! gyp info using node@15.3.0 | win32 | x64
npm ERR! gyp info find Python using Python version 3.7.7 found at "C:\Program Files\Python37\python.exe"
npm ERR! gyp info find VS using VS2019 (16.5.29911.84) found at:
npm ERR! gyp info find VS "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools"
npm ERR! gyp info find VS run with --verbose for detailed information
npm ERR! gyp info spawn C:\Program Files\Python37\python.exe
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args   'C:\\Users\\Mike\\AppData\\Roaming\\nvm\\v15.3.0\\node_modules\\npm\\node_modules\\node-gyp\\gyp\\gyp_main.py',
npm ERR! gyp info spawn args   'binding.gyp',
npm ERR! gyp info spawn args   '-f',
npm ERR! gyp info spawn args   'msvs',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   'c:\\Users\\Mike\\Documents\\git\\projects\\lightroom-catalog-helper\\node_modules\\sqlite3\\build\\config.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   'C:\\Users\\Mike\\AppData\\Roaming\\nvm\\v15.3.0\\node_modules\\npm\\node_modules\\node-gyp\\addon.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   'C:\\Users\\Mike\\AppData\\Local\\node-gyp\\Cache\\15.3.0\\include\\node\\common.gypi',
npm ERR! gyp info spawn args   '-Dlibrary=shared_library',
npm ERR! gyp info spawn args   '-Dvisibility=default',
npm ERR! gyp info spawn args   '-Dnode_root_dir=C:\\Users\\Mike\\AppData\\Local\\node-gyp\\Cache\\15.3.0',
npm ERR! gyp info spawn args   '-Dnode_gyp_dir=C:\\Users\\Mike\\AppData\\Roaming\\nvm\\v15.3.0\\node_modules\\npm\\node_modules\\node-gyp',
npm ERR! gyp info spawn args   '-Dnode_lib_file=C:\\\\Users\\\\Mike\\\\AppData\\\\Local\\\\node-gyp\\\\Cache\\\\15.3.0\\\\<(target_arch)\\\\node.lib',
npm ERR! gyp info spawn args   '-Dmodule_root_dir=c:\\Users\\Mike\\Documents\\git\\projects\\lightroom-catalog-helper\\node_modules\\sqlite3',
npm ERR! gyp info spawn args   '-Dnode_engine=v8',
npm ERR! gyp info spawn args   '--depth=.',
npm ERR! gyp info spawn args   '--no-parallel',
npm ERR! gyp info spawn args   '--generator-output',
npm ERR! gyp info spawn args   'c:\\Users\\Mike\\Documents\\git\\projects\\lightroom-catalog-helper\\node_modules\\sqlite3\\build',
npm ERR! gyp info spawn args   '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp: Undefined variable module_name in binding.gyp while trying to load binding.gyp
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: `gyp` failed with exit code: 1
npm ERR! gyp ERR! stack     at ChildProcess.onCpExit (C:\Users\Mike\AppData\Roaming\nvm\v15.3.0\node_modules\npm\node_modules\node-gyp\lib\configure.js:351:16)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:376:20)
npm ERR! gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:277:12)
npm ERR! gyp ERR! System Windows_NT 10.0.19041
npm ERR! gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\Mike\\AppData\\Roaming\\nvm\\v15.3.0\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd c:\Users\Mike\Documents\git\projects\lightroom-catalog-helper\node_modules\sqlite3
npm ERR! gyp ERR! node -v v15.3.0
npm ERR! gyp ERR! node-gyp -v v7.1.2
npm ERR! gyp ERR! not ok

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\Mike\AppData\Local\npm-cache\_logs\2020-12-07T06_52_20_262Z-debug.log

So that's still the same. Then a manual install for sqlite3:

c:\Users\Mike\Documents\git\projects\lightroom-catalog-helper>npm install sqlite3

up to date, audited 298 packages in 888ms

found 0 vulnerabilities

which seems to work, but actually doesn't:

c:\Users\Mike\Documents\git\projects\lightroom-catalog-helper>npm start

> lightroom-catalog-helper@1.0.0 start
> node help.js

node:internal/modules/cjs/loader:922
  throw err;
  ^

Error: Cannot find module 'c:\Users\Mike\Documents\git\projects\lightroom-catalog-helper\node_modules\sqlite3\lib\binding\node-v88-win32-x64\node_sqlite3.node'
Require stack:
- c:\Users\Mike\Documents\git\projects\lightroom-catalog-helper\node_modules\sqlite3\lib\sqlite3.js
- c:\Users\Mike\Documents\git\projects\lightroom-catalog-helper\lib\db.js
- c:\Users\Mike\Documents\git\projects\lightroom-catalog-helper\help.js
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:919:15)
    at Function.Module._load (node:internal/modules/cjs/loader:763:27)
    at Module.require (node:internal/modules/cjs/loader:991:19)
    at require (node:internal/modules/cjs/helpers:92:18)
    at Object.<anonymous> (c:\Users\Mike\Documents\git\projects\lightroom-catalog-helper\node_modules\sqlite3\lib\sqlite3.js:4:15)
    at Module._compile (node:internal/modules/cjs/loader:1102:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1131:10)
    at Module.load (node:internal/modules/cjs/loader:967:32)
    at Function.Module._load (node:internal/modules/cjs/loader:807:14)
    at Module.require (node:internal/modules/cjs/loader:991:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    'c:\\Users\\Mike\\Documents\\git\\projects\\lightroom-catalog-helper\\node_modules\\sqlite3\\lib\\sqlite3.js',
    'c:\\Users\\Mike\\Documents\\git\\projects\\lightroom-catalog-helper\\lib\\db.js',
    'c:\\Users\\Mike\\Documents\\git\\projects\\lightroom-catalog-helper\\help.js'
  ]
}
npm ERR! code 1
npm ERR! path c:\Users\Mike\Documents\git\projects\lightroom-catalog-helper
npm ERR! command failed
npm ERR! command C:\WINDOWS\system32\cmd.exe /d /s /c node help.js

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\Mike\AppData\Local\npm-cache\_logs\2020-12-07T06_53_06_390Z-debug.log

Checking the tree in node_modules/node-sqlite3/lib dir, this makes sense: there is no binding dir:

>dir /b/o/s

...
.\node_modules\sqlite3\deps\common-sqlite.gypi
.\node_modules\sqlite3\deps\extract.py
.\node_modules\sqlite3\deps\sqlite3.gyp
.\node_modules\sqlite3\deps\sqlite-autoconf-3300100.tar.gz
.\node_modules\sqlite3\lib\index.js
.\node_modules\sqlite3\lib\sqlite3.js
.\node_modules\sqlite3\lib\trace.js
.\node_modules\sqlite3\src\async.h
.\node_modules\sqlite3\src\backup.cc
.\node_modules\sqlite3\src\backup.h
.\node_modules\sqlite3\src\database.cc
...

[springmeyer]

Quite bizarre, not sure what is happening.

What happens if you do a manual install like npm install sqlite3 --build-from-source ?

Also, do you have another windows machine available to try the install on, to see if you get the same result?

[Pomax]

I don't, although I should point out that for debugging purposes win 10, that OS is technically free to install. You can run it without activating, and while it means you don't get to customize it, it's 100% suitable for installing VS2019 and node on.

Having said that, running npm install sqlite3 --build-from-source does not seem to build anything from source at all =x

[Uko]

I get the similar problem on Mac OS. Here is my trace:

error code 1
error path <project-dir>/server/node_modules/sqlite3
error command failed
error command sh -c node-gyp rebuild
error gyp: Undefined variable module_name in binding.gyp while trying to load binding.gyp
error gyp ERR! configure error
error gyp ERR! stack Error: `gyp` failed with exit code: 1
error gyp ERR! stack     at ChildProcess.onCpExit (<project-dir>/server/node_modules/sqlite3/node_modules/node-gyp/lib/configure.js:345:16)
error gyp ERR! stack     at ChildProcess.emit (node:events:376:20)
error gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:284:12)
error gyp ERR! System Darwin 18.7.0
error gyp ERR! command "~/.nvm/versions/node/v15.4.0/bin/node" "<project-dir>/server/node_modules/sqlite3/node_modules/.bin/node-gyp" "rebuild"
error gyp ERR! cwd <project-dir>/server/node_modules/sqlite3
error gyp ERR! node -v v15.4.0
error gyp ERR! node-gyp -v v3.8.0
error gyp ERR! not ok
verbose exit 1

[springmeyer]

@Uko can you provide detailed steps to replicate?

One odd thing I see is that a node-gyp version inside of node-sqlite3 is being used instead of the node-gyp that should come with the node install. So I wonder if this #1313 is the source of the breakage (in that it started installing node-gyp@3x which could be incompatible perhaps with latest node). Just a hunch, but if one of you could try reverting #1313 + do a fresh install to ensure the node-gyp from node is used, that would be helpful.

[Pomax]

If there are some instructions on how to do that, I'll be more than happy to test that for you.

[Uko]

@springmeyer I second what @Pomax said. What should we do? Is there a specific version we need to load, or would we patch something in the already existing code that we have?

[springmeyer]

The steps I have in mind are:

• create a reproducible testcase (writing docs or code and putting it in repo that, when followed reproduces the problem every time)
• then fork node-sqlite and revert fix: [BUG] yarn 2 pnp issues #1312 #1313 by doing git revert 1752f42 and committing your change. Or manually undoing the change.
• integrate your fork into the reproducible testcase either by using npm link ../path-to-node-sqlite3 or by publishing your fork and integrating it into your testcase by running npm install <name of your fork>
• trying your testcase again? is the problem gone?

[Pomax]

the first item is literally just "run npm install sqlite3" though. There isn't any code or docs to write: the very act of trying to install fails.

[Pomax]

Just tried installing @5.0.0 and that worked without a hitch, so this issue has been obviated for me, but if others are still locked into using 4.x they may still want to see if they can figure out what's going wrong.

From me, though: many thanks for the hard work on node-sqlite3, and a Merry Christmas to all!

[kibertoad]

Can confirm that it is also resolved for Node 15.x on Linux as well, so I think this issue can be closed.

[Pomax]

nice - I'll close this one, and if anyone still has this problem, they can file a more targeted issue