Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows: Broken node alias #751

Closed
kkoopa opened this issue Feb 7, 2015 · 34 comments
Closed

Windows: Broken node alias #751

kkoopa opened this issue Feb 7, 2015 · 34 comments
Labels
windows Issues and PRs related to the Windows platform.

Comments

@kkoopa
Copy link

kkoopa commented Feb 7, 2015

The node compatibility link is unusable on windows. Addons depend on doing a callback to the specified executable module via node_module_register, which is not possible when the name is wrong.

If an addon depends on iojs.exe, it cannot be used through node.exe and vice versa, giving the 'Module did not self-register.' error.

node -e "require('./test/build/Release/addon')"
Error: Module did not self-register.

iojs -e "require('./test/build/Release/addon')"
@rvagg
Copy link
Member

rvagg commented Feb 7, 2015

/cc @piscisaureus

do we need to do that stub executable after all?

@kkoopa
Copy link
Author

kkoopa commented Feb 7, 2015

Perhaps node.exe can spawn iojs.exe with the same command line arguments and redirected stdin, stdout and stderr?

kkoopa added a commit to nodejs/nan that referenced this issue Feb 7, 2015
@Fishrock123 Fishrock123 added the windows Issues and PRs related to the Windows platform. label Feb 7, 2015
@piscisaureus
Copy link
Contributor

This issue is that the name of the library that compiled addons link against is fixed. For this reason it also has never been possible to rename node.exe on windows and have compiled addons still work.

In theory this issue could be solved by delay-loading the symbols that come from iojs.exe and adding a delay-load notification hook to the compiled addon.

Unfortunately I've never gotten around to implementing this.

do we need to do that stub executable after all?

The original plan I had where node.exe would "load" iojs.exe as a dynamic library turned out not to work.

Perhaps node.exe can spawn iojs.exe with the same command line arguments and redirected stdin, stdout and stderr?

That's possible but it I'm afraid it'll break other scripts that assume that they can obtain the PID for their child node process.
At this point it also breaks clustering.

@kkoopa
Copy link
Author

kkoopa commented Feb 7, 2015

delayload might not work. It is not allowed to delay-load imported data, which happens when someone uses v8::String::ExternalStringResource which is a class exported from v8 with virtual members. Cannot link without the vtable.

LINK : fatal error LNK1194: cannot delay-load 'iojs.exe' due to import of data symbol '"__declspec(dllimport) const v8::String::ExternalStringResource::`vftable'" (__imp_??_7ExternalStringResource@String@v8@@6B@)';
link without /DELAYLOAD:iojs.exe

@piscisaureus
Copy link
Contributor

delayload might not work. It is not allowed to delay-load imported data, which happens when someone uses v8::String::ExternalStringResource which is a class exported from v8 with virtual members. Cannot link without the vtable.

Right, I didn't think of that.
I don't have any other ideas at this point.

@kkoopa
Copy link
Author

kkoopa commented Feb 7, 2015

Is it possible to break out all shared functionality into a real dynamic library, make addons link against that and have two thin frontends, "node.exe" and "iojs.exe"? Addons would then not necessarily be tied to executable names. If there still is a need for a registration callback inside the frontends then that can be found by dynamically loading the right library based on GetModuleFileName.

In short: turn iojs.exe into iojs.dll, make new iojs.exe and node.exe as frontends for iojs.dll.

node_main.cc could become the new iojs executable target by itself and the rest of iojs would become libiojs, now dynamically link iojs to libiojs and have addons link to libiojs. Now addons would no longer be dependent on the name of the main executable.

@kobalicek
Copy link

kkoopa: +1. I think this should have happened long time ago and I think I have filled an issue related to this already. I'm interested if there is anything I can help with.

@kobalicek
Copy link

I have filled this one, which is related: nodejs/roadmap#9

hex7c0 added a commit to kesla/node-snappy that referenced this issue Feb 8, 2015
@lidlanca
Copy link

lidlanca commented Feb 8, 2015

Suggestion:

Adding a symbolic link in the iojs_install_path
node.exe <====> iojs.exe

> cd iojs_install_path
> mklink node.exe iojs.exe

For systems that node and iojs need to co-exists, additional work around is needed.
Prioritizing the iojs_install_path in the environmental PATH variable, when running iojs.exe. so it does not pick the original node binary on the system.

Not sure if the suggestion will solve the issue @kkoopa reported.

Edit:
I noticed a case where a module build reference the node.lib in an external path ~/node-gyp
and not iojs_install_path where I assume the installation will place it.

I experimented on node-sqlite3 which did not build for me out of the box.
The main issue was that it tried to reference ~.node-gyp\1.1.0\x64\node.lib, which does not exists.

To resolve I added a symbolic link to the node.lib <== ==>iojs.lib in the proper ~.node-gyp path and patched node-sqlite3/package.json to support the 1.1.0 version.

The result is a successful build with iojs on windows.

@Mithgol
Copy link

Mithgol commented Feb 17, 2015

For future references, nodejs/node-gyp#564 is an attempt to fix this problem on build tools' level.

springmeyer pushed a commit to mapbox/node-pre-gyp that referenced this issue Feb 17, 2015
@kkoopa
Copy link
Author

kkoopa commented Feb 17, 2015

This problem cannot be fixed from there. Addons have to register with the executable. If they depend on exports from iojs.exe, they cannot resolve them against node.exe and vice versa.

@am11
Copy link

am11 commented Mar 1, 2015

Redirecting from #871.

If we use nvmw, this problem does not happen because nvmw only downloads iojs.exe, downloads the corresponding npm version separately and then injects its own alias-node.cmd as node.cmd in iojs-vx.y.z directory, which acts exactly like symlink on *nix (for instance node.cmd -p process.execPath returns path/to/iojs.exe).

Unfortunately, AppVeyor does not use nvmw, but their own version manager, which downloads and extracts io.js' .msi installer which comes packaged with node.exe.

As you can see in #871 comments by @piscisaureus, node.cmd is not a viable solution for io.js OOTB due to the limitations of CreatProcess().

Nonetheless, @kkoopa's solution is interesting: make iojs.vcxproj emit a DLL and then keep exes separate interacting with DLL. Yet again, going by nodejs/roadmap#9 (comment), there seems to be other issues with this approach as well.

@kkoopa, on that note, can this be worked out using Adjuster Thunk in assembly: http://stackoverflow.com/a/1090914/863980? The linked blog by Raymond Chen provides some insights on v-table thunking. On a related note, this project SharpNeedle by @ChadSki injects C# assembly in any running process using DllMain injection as prescribe in another blog. Perhaps we can compose a workaround for "fixed library name" problem and add it to uv_dlopen (libuv\win\dl.c)? This way we will be able to rename iojs.exe to node.exe or even my-favorite-name.exe, without breaking dependencies.

@Fishrock123
Copy link
Contributor

Related to #895

othiym23 pushed a commit to npm/node that referenced this issue Apr 10, 2015
On Windows, when node or io.js attempts to dynamically load a compiled
addon, the compiled addon tries to load node.exe or iojs.exe again -
depending on which import library the module used when it was linked.
This causes many compiled addons to break when node.exe or iojs.exe are
renamed, because when the binary has been renamed the addon DLL can't
find the (right) .exe file to load its imports from.

This patch gives compiled addon developers an option to overcome this
restriction by compiling a delay-load hook into their binary. The
delay-load hook ensures that whenever a module tries to load imports
from node.exe/iojs.exe, it'll just look at the process image, thereby
making the addon work regardless of what name the node/iojs binary has.

To enable this feature, the addon developer must set the
'win_delay_load_hook' option to 'true' in their binding.gyp file, like
this:

```
{
  'targets': [
    {
      'target_name': 'ernie',
      'win_delay_load_hook': 'true',
      ...
```

Bug: nodejs#751
Bug: nodejs#965
Upstream PR: nodejs/node-gyp#599

PR-URL: nodejs#1251
Reviewed-By: Rod Vagg <rod@vagg.org>

PR-URL: nodejs#1266
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
othiym23 pushed a commit to npm/node that referenced this issue Apr 10, 2015
On Windows, when node or io.js attempts to dynamically load a compiled
addon, the compiled addon tries to load node.exe or iojs.exe again -
depending on which import library the module used when it was linked.
This causes many compiled addons to break when node.exe or iojs.exe are
renamed, because when the binary has been renamed the addon DLL can't
find the (right) .exe file to load its imports from.

This patch gives compiled addon developers an option to overcome this
restriction by compiling a delay-load hook into their binary. The
delay-load hook ensures that whenever a module tries to load imports
from node.exe/iojs.exe, it'll just look at the process image, thereby
making the addon work regardless of what name the node/iojs binary has.

To enable this feature, the addon developer must set the
'win_delay_load_hook' option to 'true' in their binding.gyp file, like
this:

```
{
  'targets': [
    {
      'target_name': 'ernie',
      'win_delay_load_hook': 'true',
      ...
```

Bug: nodejs#751
Bug: nodejs#965
Upstream PR: nodejs/node-gyp#599

PR-URL: nodejs#1251
Reviewed-By: Rod Vagg <rod@vagg.org>

PR-URL: nodejs#1266
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
othiym23 pushed a commit to npm/node that referenced this issue Apr 10, 2015
On Windows, when node or io.js attempts to dynamically load a compiled
addon, the compiled addon tries to load node.exe or iojs.exe again -
depending on which import library the module used when it was linked.
This causes many compiled addons to break when node.exe or iojs.exe are
renamed, because when the binary has been renamed the addon DLL can't
find the (right) .exe file to load its imports from.

This patch gives compiled addon developers an option to overcome this
restriction by compiling a delay-load hook into their binary. The
delay-load hook ensures that whenever a module tries to load imports
from node.exe/iojs.exe, it'll just look at the process image, thereby
making the addon work regardless of what name the node/iojs binary has.

To enable this feature, the addon developer must set the
'win_delay_load_hook' option to 'true' in their binding.gyp file, like
this:

```
{
  'targets': [
    {
      'target_name': 'ernie',
      'win_delay_load_hook': 'true',
      ...
```

Bug: nodejs#751
Bug: nodejs#965
Upstream PR: nodejs/node-gyp#599

PR-URL: nodejs#1251
Reviewed-By: Rod Vagg <rod@vagg.org>

PR-URL: nodejs#1266
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
othiym23 pushed a commit to npm/node that referenced this issue Apr 17, 2015
On Windows, when node or io.js attempts to dynamically load a compiled
addon, the compiled addon tries to load node.exe or iojs.exe again -
depending on which import library the module used when it was linked.
This causes many compiled addons to break when node.exe or iojs.exe are
renamed, because when the binary has been renamed the addon DLL can't
find the (right) .exe file to load its imports from.

This patch gives compiled addon developers an option to overcome this
restriction by compiling a delay-load hook into their binary. The
delay-load hook ensures that whenever a module tries to load imports
from node.exe/iojs.exe, it'll just look at the process image, thereby
making the addon work regardless of what name the node/iojs binary has.

To enable this feature, the addon developer must set the
'win_delay_load_hook' option to 'true' in their binding.gyp file, like
this:

```
{
  'targets': [
    {
      'target_name': 'ernie',
      'win_delay_load_hook': 'true',
      ...
```

Bug: nodejs#751
Bug: nodejs#965
Upstream PR: nodejs/node-gyp#599

PR-URL: nodejs#1251
Reviewed-By: Rod Vagg <rod@vagg.org>

PR-URL: nodejs#1266
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
piscisaureus added a commit that referenced this issue Apr 17, 2015
On Windows, when node or io.js attempts to dynamically load a compiled
addon, the compiled addon tries to load node.exe or iojs.exe again -
depending on which import library the module used when it was linked.
This causes many compiled addons to break when node.exe or iojs.exe are
renamed, because when the binary has been renamed the addon DLL can't
find the (right) .exe file to load its imports from.

This patch gives compiled addon developers an option to overcome this
restriction by compiling a delay-load hook into their binary. The
delay-load hook ensures that whenever a module tries to load imports
from node.exe/iojs.exe, it'll just look at the process image, thereby
making the addon work regardless of what name the node/iojs binary has.

To enable this feature, the addon developer must set the
'win_delay_load_hook' option to 'true' in their binding.gyp file, like
this:

```
{
  'targets': [
    {
      'target_name': 'ernie',
      'win_delay_load_hook': 'true',
      ...
```

Bug: #751
Bug: #965
Upstream PR: nodejs/node-gyp#599

PR-URL: #1251
Reviewed-By: Rod Vagg <rod@vagg.org>

PR-URL: #1266
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
piscisaureus added a commit that referenced this issue Apr 17, 2015
On Windows, when node or io.js attempts to dynamically load a compiled
addon, the compiled addon tries to load node.exe or iojs.exe again -
depending on which import library the module used when it was linked.
This causes many compiled addons to break when node.exe or iojs.exe are
renamed, because when the binary has been renamed the addon DLL can't
find the (right) .exe file to load its imports from.

This patch gives compiled addon developers an option to overcome this
restriction by compiling a delay-load hook into their binary. The
delay-load hook ensures that whenever a module tries to load imports
from node.exe/iojs.exe, it'll just look at the process image, thereby
making the addon work regardless of what name the node/iojs binary has.

To enable this feature, the addon developer must set the
'win_delay_load_hook' option to 'true' in their binding.gyp file, like
this:

```
{
  'targets': [
    {
      'target_name': 'ernie',
      'win_delay_load_hook': 'true',
      ...
```

Bug: #751
Bug: #965
Upstream PR: nodejs/node-gyp#599

PR-URL: #1251
Reviewed-By: Rod Vagg <rod@vagg.org>

PR-URL: #1266
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
piscisaureus added a commit that referenced this issue Apr 17, 2015
On Windows, when node or io.js attempts to dynamically load a compiled
addon, the compiled addon tries to load node.exe or iojs.exe again -
depending on which import library the module used when it was linked.
This causes many compiled addons to break when node.exe or iojs.exe are
renamed, because when the binary has been renamed the addon DLL can't
find the (right) .exe file to load its imports from.

This patch gives compiled addon developers an option to overcome this
restriction by compiling a delay-load hook into their binary. The
delay-load hook ensures that whenever a module tries to load imports
from node.exe/iojs.exe, it'll just look at the process image, thereby
making the addon work regardless of what name the node/iojs binary has.

To enable this feature, the addon developer must set the
'win_delay_load_hook' option to 'true' in their binding.gyp file, like
this:

```
{
  'targets': [
    {
      'target_name': 'ernie',
      'win_delay_load_hook': 'true',
      ...
```

Bug: #751
Bug: #965
Upstream PR: nodejs/node-gyp#599

PR-URL: #1251
Reviewed-By: Rod Vagg <rod@vagg.org>

PR-URL: #1266
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
othiym23 pushed a commit to npm/node that referenced this issue Apr 24, 2015
On Windows, when node or io.js attempts to dynamically load a compiled
addon, the compiled addon tries to load node.exe or iojs.exe again -
depending on which import library the module used when it was linked.
This causes many compiled addons to break when node.exe or iojs.exe are
renamed, because when the binary has been renamed the addon DLL can't
find the (right) .exe file to load its imports from.

This patch gives compiled addon developers an option to overcome this
restriction by compiling a delay-load hook into their binary. The
delay-load hook ensures that whenever a module tries to load imports
from node.exe/iojs.exe, it'll just look at the process image, thereby
making the addon work regardless of what name the node/iojs binary has.

To enable this feature, the addon developer must set the
'win_delay_load_hook' option to 'true' in their binding.gyp file, like
this:

```
{
  'targets': [
    {
      'target_name': 'ernie',
      'win_delay_load_hook': 'true',
      ...
```

Bug: nodejs#751
Bug: nodejs#965
Upstream PR: nodejs/node-gyp#599

PR-URL: nodejs#1251
Reviewed-By: Rod Vagg <rod@vagg.org>

PR-URL: nodejs#1266
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
othiym23 pushed a commit to npm/node that referenced this issue May 1, 2015
On Windows, when node or io.js attempts to dynamically load a compiled
addon, the compiled addon tries to load node.exe or iojs.exe again -
depending on which import library the module used when it was linked.
This causes many compiled addons to break when node.exe or iojs.exe are
renamed, because when the binary has been renamed the addon DLL can't
find the (right) .exe file to load its imports from.

This patch gives compiled addon developers an option to overcome this
restriction by compiling a delay-load hook into their binary. The
delay-load hook ensures that whenever a module tries to load imports
from node.exe/iojs.exe, it'll just look at the process image, thereby
making the addon work regardless of what name the node/iojs binary has.

To enable this feature, the addon developer must set the
'win_delay_load_hook' option to 'true' in their binding.gyp file, like
this:

```
{
  'targets': [
    {
      'target_name': 'ernie',
      'win_delay_load_hook': 'true',
      ...
```

Bug: nodejs#751
Bug: nodejs#965
Upstream PR: nodejs/node-gyp#599

PR-URL: nodejs#1251
Reviewed-By: Rod Vagg <rod@vagg.org>

PR-URL: nodejs#1266
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
othiym23 pushed a commit to npm/node that referenced this issue May 1, 2015
On Windows, when node or io.js attempts to dynamically load a compiled
addon, the compiled addon tries to load node.exe or iojs.exe again -
depending on which import library the module used when it was linked.
This causes many compiled addons to break when node.exe or iojs.exe are
renamed, because when the binary has been renamed the addon DLL can't
find the (right) .exe file to load its imports from.

This patch gives compiled addon developers an option to overcome this
restriction by compiling a delay-load hook into their binary. The
delay-load hook ensures that whenever a module tries to load imports
from node.exe/iojs.exe, it'll just look at the process image, thereby
making the addon work regardless of what name the node/iojs binary has.

To enable this feature, the addon developer must set the
'win_delay_load_hook' option to 'true' in their binding.gyp file, like
this:

```
{
  'targets': [
    {
      'target_name': 'ernie',
      'win_delay_load_hook': 'true',
      ...
```

Bug: nodejs#751
Bug: nodejs#965
Upstream PR: nodejs/node-gyp#599

PR-URL: nodejs#1251
Reviewed-By: Rod Vagg <rod@vagg.org>

PR-URL: nodejs#1266
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
othiym23 pushed a commit to npm/node that referenced this issue May 1, 2015
On Windows, when node or io.js attempts to dynamically load a compiled
addon, the compiled addon tries to load node.exe or iojs.exe again -
depending on which import library the module used when it was linked.
This causes many compiled addons to break when node.exe or iojs.exe are
renamed, because when the binary has been renamed the addon DLL can't
find the (right) .exe file to load its imports from.

This patch gives compiled addon developers an option to overcome this
restriction by compiling a delay-load hook into their binary. The
delay-load hook ensures that whenever a module tries to load imports
from node.exe/iojs.exe, it'll just look at the process image, thereby
making the addon work regardless of what name the node/iojs binary has.

To enable this feature, the addon developer must set the
'win_delay_load_hook' option to 'true' in their binding.gyp file, like
this:

```
{
  'targets': [
    {
      'target_name': 'ernie',
      'win_delay_load_hook': 'true',
      ...
```

Bug: nodejs#751
Bug: nodejs#965
Upstream PR: nodejs/node-gyp#599

PR-URL: nodejs#1251
Reviewed-By: Rod Vagg <rod@vagg.org>

PR-URL: nodejs#1266
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
othiym23 pushed a commit to npm/node that referenced this issue May 2, 2015
On Windows, when node or io.js attempts to dynamically load a compiled
addon, the compiled addon tries to load node.exe or iojs.exe again -
depending on which import library the module used when it was linked.
This causes many compiled addons to break when node.exe or iojs.exe are
renamed, because when the binary has been renamed the addon DLL can't
find the (right) .exe file to load its imports from.

This patch gives compiled addon developers an option to overcome this
restriction by compiling a delay-load hook into their binary. The
delay-load hook ensures that whenever a module tries to load imports
from node.exe/iojs.exe, it'll just look at the process image, thereby
making the addon work regardless of what name the node/iojs binary has.

To enable this feature, the addon developer must set the
'win_delay_load_hook' option to 'true' in their binding.gyp file, like
this:

```
{
  'targets': [
    {
      'target_name': 'ernie',
      'win_delay_load_hook': 'true',
      ...
```

Bug: nodejs#751
Bug: nodejs#965
Upstream PR: nodejs/node-gyp#599

PR-URL: nodejs#1251
Reviewed-By: Rod Vagg <rod@vagg.org>

PR-URL: nodejs#1266
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
othiym23 pushed a commit to npm/node that referenced this issue May 8, 2015
On Windows, when node or io.js attempts to dynamically load a compiled
addon, the compiled addon tries to load node.exe or iojs.exe again -
depending on which import library the module used when it was linked.
This causes many compiled addons to break when node.exe or iojs.exe are
renamed, because when the binary has been renamed the addon DLL can't
find the (right) .exe file to load its imports from.

This patch gives compiled addon developers an option to overcome this
restriction by compiling a delay-load hook into their binary. The
delay-load hook ensures that whenever a module tries to load imports
from node.exe/iojs.exe, it'll just look at the process image, thereby
making the addon work regardless of what name the node/iojs binary has.

To enable this feature, the addon developer must set the
'win_delay_load_hook' option to 'true' in their binding.gyp file, like
this:

```
{
  'targets': [
    {
      'target_name': 'ernie',
      'win_delay_load_hook': 'true',
      ...
```

Bug: nodejs#751
Bug: nodejs#965
Upstream PR: nodejs/node-gyp#599

PR-URL: nodejs#1251
Reviewed-By: Rod Vagg <rod@vagg.org>

PR-URL: nodejs#1266
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
iarna pushed a commit to npm/node that referenced this issue May 22, 2015
On Windows, when node or io.js attempts to dynamically load a compiled
addon, the compiled addon tries to load node.exe or iojs.exe again -
depending on which import library the module used when it was linked.
This causes many compiled addons to break when node.exe or iojs.exe are
renamed, because when the binary has been renamed the addon DLL can't
find the (right) .exe file to load its imports from.

This patch gives compiled addon developers an option to overcome this
restriction by compiling a delay-load hook into their binary. The
delay-load hook ensures that whenever a module tries to load imports
from node.exe/iojs.exe, it'll just look at the process image, thereby
making the addon work regardless of what name the node/iojs binary has.

To enable this feature, the addon developer must set the
'win_delay_load_hook' option to 'true' in their binding.gyp file, like
this:

```
{
  'targets': [
    {
      'target_name': 'ernie',
      'win_delay_load_hook': 'true',
      ...
```

Bug: nodejs#751
Bug: nodejs#965
Upstream PR: nodejs/node-gyp#599

PR-URL: nodejs#1251
Reviewed-By: Rod Vagg <rod@vagg.org>

PR-URL: nodejs#1266
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
@JCMais
Copy link
Contributor

JCMais commented Aug 14, 2015

Is this still a issue?

@justinmchase
Copy link

It does still exist unfortunately. It still exists specifically for apps that statically link to node but have a different executable name, e.g. electron.exe, nw.exe, etc.

The delay load hook hack is a way to work around it. I have a PR out for node-gyp which would fix it automatically for everyone, I believe:
nodejs/node-gyp#653

This would allow embedders to ship their own build of node.dll and automatically get compatibility with native modules compiled against node. Node would not have to ship a node.dll, embedders could optionally do it to get compatibility.

@Fishrock123
Copy link
Contributor

This won't be an issue in the converged 4.0.0 release. Since it's impractical to address in io.js (1.0.0> <4.0.0), I'm going to close this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
windows Issues and PRs related to the Windows platform.
Projects
None yet
Development

No branches or pull requests