N-api builds not cross runtime with Electron on windows

[jpkolbush]

• Node Version: 8.9.4
• Platform: Windows
• Compiler: MSVC 15

I was led to believe that the new n-api was not only going to be compatible across all node runtimes, but also going to be compatible when building a native module for node vs building for electron.
If you build the basic hello-world example for n-api found here, the addon.node file cannot be required when running from the electron runtime. I get the following error:

A dynamic link library (DLL) initialization routine failed.
\\?\C:\Users\kelannen\source\napi-test\build\Release\addon.node
Error: A dynamic link library (DLL) initialization routine failed.
\\?\C:\Users\kelannen\source\napi-test\build\Release\addon.node
    at process.module.(anonymous function) [as dlopen] (ELECTRON_ASAR.js:172:20)
    at Object.Module._extensions..node (module.js:671:18)
    at Object.module.(anonymous function) [as .node] (ELECTRON_ASAR.js:172:20)
    at Module.load (module.js:561:32)
    at tryModuleLoad (module.js:504:12)
    at Function.Module._load (module.js:496:3)
    at Module.require (module.js:586:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (C:\Users\kelannen\source\napi-test\test.js:2:15)
    at Object.<anonymous> (C:\Users\kelannen\source\napi-test\test.js:6:3)
    at Module._compile (module.js:642:30)
    at Object.Module._extensions..js (module.js:653:10)
    at Module.load (module.js:561:32)
    at tryModuleLoad (module.js:504:12)
    at Function.Module._load (module.js:496:3)
    at Module.require (module.js:586:17)
    at require (internal/module.js:11:18)
    at C:\Users\kelannen\source\napi-test\node_modules\mocha\lib\mocha.js:253:27
    at Array.forEach (<anonymous>)
    at Mocha.loadFiles (C:\Users\kelannen\source\napi-test\node_modules\mocha\lib\mocha.js:250:14)
    at Mocha.run (C:\Users\kelannen\source\napi-test\node_modules\mocha\lib\mocha.js:577:10)
    at Object.run (C:\Users\kelannen\source\napi-test\node_modules\electron-mocha\mocha.js:54:30)
    at App.app.on (C:\Users\kelannen\source\napi-test\node_modules\electron-mocha\index.js:54:13)
    at emitTwo (events.js:131:20)
    at App.emit (events.js:214:7)

I set up a basic test here:

https://github.com/jpkolbush/napi-test

Steps to reproduce:

npm install

#runs node-gyp configure-build
npm run build

#confirms that the module can be included with node runtime
node test.js

#requires the node module while running electron-mocha, this command will fail
npm run electron-test

This is only an issue on Windows, does not appear on Unix.

[mhdawson]

There have been some discussions with the electron team but there are additional factors that mean this likely require the electron team to do work as well.

The main issue I believe is that electron uses different versions of V8 than what may have been used with Node.js (if I remember correctly to match the version of chrome used), and therefore the implementation of the N-API methods that are exported might have to be tweaked for a version of V8 that was never shipped with Node.

Not quite sure why it would be Windows only though.

Probably best to loop in @groundwater into the discussion to get the full picture.

[jpkolbush]

It seems like to me that its related to the node.lib file that gets linked into the build on windows. If you build a module for electron 2.0.0 runtime the build gets linked to this library: https://atom.io/download/atom-shell/v2.0.0/x64/node.lib

However the version of node.lib that gets linked in for node builds is https://nodejs.org/dist/v8.9.3/win-x64/node.lib. Even though electron 2.0.0 is backed by node version 8.9.3, these are clearly different as they are different sizes. This would explain why it only happens on windows and why the error mentions DLL initialization. The electron runtime is looking for symbols that aren't there in the node runtime.

With the n-api environment, it would seem to me that even if electron used slightly different V8 headers it would still be compatible (as we see on Unix). It' once these node.lib libraries are aligned the runtimes will probably be compatible on windows as well.

Update: This also fails if you build for electron and try running in the node enviroment.

module.js:681

  return process.dlopen(module, path._makeLong(filename));
                 ^

Error: The specified module could not be found.
\\?\C:\path\to\bin\N-API-Greeter.node
    at Object.Module._extensions..node (module.js:681:18)
    at Module.load (module.js:565:32)

Once again, only on windows

[anaisbetts]

Error: The specified module could not be found.

https://blogs.msdn.microsoft.com/junfeng/2006/11/20/debugging-loadlibrary-failures will help you figure out the root cause here - the short version is:

1. Enable Loader snaps for electron.exe
2. Run electron.exe under WinDbg
3. Try to load that Napi module
4. See the debug output

[jpkolbush]

@paulcbetts
I assume you mean running node.exe under WinDbg since thats the process I'm trying to load the module with. It was just compiled against the headers (and node.lib) provided from electron.

[mhdawson]

Any update on the discussion here?

[mafintosh]

Running into the same problems with n-api builds done against Node.js not working with Electron on windows.

[BotellaA]

Still no update?

[mhdawson]

@codebytere this is the issue we mentioned at the summit.

[NEVADV]

@here Hi
i face the same problem, my addon build with node doesn't work with the version of node in Electron, i tested with many version of node and electron, and he never worked, i tested with electron-rebuild and he never work even electron find the good header when i done :
this "node-gyp rebuild --target=3.0.6 --arch=x64 --dist-url=https://atom.io/download/atom-shell"
with many different target to map my version of electron, but i got always the same messsage
"bindings.js:93 Uncaught Error: Could not locate the bindings file. Tried:
→
C:\Users\Ignazio\Desktop\4M\BRIDGE\4mantis\bridge\bridgeElectron\BridgeReact\build\addon.node

→ C:\Users\Ignazio\Desktop\4M\BRIDGE\4mantis\bridge\bridgeElectron\BridgeReact\build\Debug\addon.node
→ C:\Users\Ignazio\Desktop\4M\BRIDGE\4mantis\bridge\bridgeElectron\BridgeReact\build\Release\addon.node
"

the addon is never find

-------- EDITED
finally i tested with a new simple electron app ( the sample quick-start from electron site) and it work with the use of electron-rebuild and this command
"node-gyp rebuild --target=3.0.4 --arch=x64 --dist-url=https://atom.io/download/atom-shell"
make sure you target the right target ( here i target the version 3.0.4 of electron )
hope this help !

[addaleax]

@nodejs/platform-windows Any ideas here?

[refack]

I have a hunch it's related to nodejs/node-gyp#1566.
I'm verifying now.

[mhdawson]

@codebytere any chance you've been able to look at this?

[mhdawson]

Although I guess worth waiting until @refack confirms if it is nodejs/node-gyp#1566 or not.

[refack]

@refack confirms if it is nodejs/node-gyp#1566 or not.

No, nodejs/node-gyp#1566 did not solve this.

[stevebaxter]

Is there a solution or workaround for this? We are building an Electron app that runs on Mac and Windows and needs to call a C++ lib which is packaged as a napi module. Everything works perfectly on the Mac, but we get this problem on Windows.

This is a bit of a disaster for us if this doesn't work!

[mhdawson]

@codebytere would you be able to help us better understand the problem and what options there are for improving this when using modules with Electron?

[stevebaxter]

I have managed to work around this problem. I'm building my native module using cmake-js, here is what I did:

1. Add set_target_properties(${PROJECT_NAME} PROPERTIES LINK_FLAGS "/DELAYLOAD:node.exe" ) to the Windows version of our native module in CMakeLists.txt
2. Add a delay load hook based on the code here: https://github.com/billti/create-vs-napi/blob/master/template/win_delay_load_hook.cc.txt

The problem is that cmake-js links the native module directly to node.exe (you will see this if you open it in Dependency Viewer). When we try and run this in Electron, node.exe isn't available so LoadModule() fails. What the hook does is look for a request to load node.exe and intercept it. If node.exe is available then fine, it lets it through. If not it looks for node.dll (which will work for Electron 3.x) or failing that returns the hosting .exe (which I think will be needed for Electron 4.x).

Hope this helps someone else looking to solve this problem!

[Karthick0790]

Thanks @stevebaxter , The above solution worked for us.

[mhdawson]

@digitalinfinity could you possibly take a look at this since it affects Windows and help the N-API understand the problem and any way forward?

[vweevers]

Does this only apply to Windows (and not any other platform)?

[gabrielschulhof]

@vweevers I'm pretty sure it's Windows-specific, because on Windows the source of the symbols required by the native addon matters. If the native addon specifies that the symbols have to come from "node.exe", then the loader will only use "node.exe" to resolve the symbols. "node.dll" will not satisfy the requirement. However, the workaround above presented by @stevebaxter should do the trick.

[stevebaxter]

I think that's correct, AFAIK it's only on Windows that the symbols are exported from different DSOs depending on how everything is linked (or at least this hasn't been a problem on macOS).

[stevebaxter]

The right way to do this on Windows is probably to repackage node.exe into node.exe (a small executable that just acts as a command-line handler for node.dll), and node.dll (the main DLL that exports all the symbols). This way, NAPI modules would always link to node.dll, executables that use node (e.g. Electron) would also link to node.dll.

This might be too big a change for the node team though, so my workaround above could also be incorporated somehow.

[vweevers]

or at least this hasn't been a problem on macOS

Yes, I've confirmed that with tests.

I could not reproduce the issue though on Windows with Electron 4, only Electron 3.

[mhdawson]

@codebytere was there something changed in electron4 that would have helped on this front?

[refack]

If the native addon specifies that the symbols have to come from "node.exe", then the loader will only use "node.exe" to resolve the symbols. "node.dll" will not satisfy the requirement. However, the workaround above presented by @stevebaxter should do the trick.

BTW that's what node-gyp does by adding win_delay_load_hook.cc to every addon in
https://github.com/nodejs/node-gyp/blob/bb8b294ae76ed16d6633344ff06f6dc5d5ddbe02/addon.gypi#L67-L89

/CC @unbornchikken @cmake-js

[vweevers]

It may be a separate issue. I get the A dynamic link library (DLL) initialization routine failed error with node-gyp in fact. To reproduce (pairing Electron 3 with leveldown@5):

git clone git@github.com:Level/electron-demo.git
cd electron-demo

npm install --build-from-source
npm install electron@3
npm start

[mafintosh]

Does anyone know if this has been resolved?

[vweevers]

On Electron >=4 I've seen no issues (and I gave up on Electron 3).

[mhdawson]

@codebytere do you think this is resolved and can be closed?

[codebytere]

Closing as non-reproducible on currently supported versions of Electron (v6 and above)

[bsrdjan]

I am getting A dynamic link library (DLL) initialization routine failed with node-rfc NAPI module in Electron, on Windows 10. The module works just fine without Electron. Here the configuration:

• Windows 10
• electron 8.2.5
• electron-rebuild 1.10.1
• node 12.13.0
• npm 6.12.0

Reproducible with require('node-rfc') step in preload.js of electron-quick-start example.

[yerkeyzet]

@s1hofmann

I see that you were able to resolve this issue in your project here 126

Please can you share with us, what was the exact solution?

[s1hofmann]

@yerkeyzet

I added a gist of my CMakeLists.txt file I use for N-API projects (including win_delay_load_hook.cc).
I'm only considering Electron v4 or higher, so no guarantee for v3 or earlier.

The weird thing which caused problems in my project was the following:

The project contains a rather old version of Electron which is used to run UI tests for my project (no time to update yet).
On Windows, my module was not usable with Electron (Uncaught Error: A dynamic link library (DLL) initialization routine failed...) after running npm install && npm run build:release:win. No problems on Linux and macOS.
After I dissected the whole project I noticed that as soon as I included the outdated Electron version required for tests in my devDependencies, the module would no longer work with Electron on Windows. Builds which do not have Electron installed worked fine on Windows as well.

My workaround for this problem was to move dependencies required for tests to a separate sub-package so they wont be installed on builds.

Still sounds strange to me and I'm still not 100% sure about the root cause of this behaviour.
Anyways, since N-API builds in combination with Electron seem to not only confuse myself sometimes, I added my findings here.

Best regards

Simon

[lilong7676]

Can anyone help to read this issue electron/electron#27205
@mhdawson Please help...