win,msi: install tools for native modules #22645

joaocgreis · 2018-09-01T23:19:46Z

Add a dialog during installation with information about native modules that can optionally run a Boxstarter script at the end of the installation. This script can also be run from Start menu.

Adding semver-minor since this is adding a new feature. Not semver-major since this should not break anything (I tested upgrading).

Fixes: #22311

cc @nodejs/platform-windows

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
commit message follows commit guidelines

Add a dialog during installation with information about native modules that can optionally run a Boxstarter script at the end of the installation. This script can also be run from Start menu. Fixes: nodejs#22311

nodejs-github-bot · 2018-09-01T23:19:48Z

@joaocgreis build started: https://ci.nodejs.org/blue/organizations/jenkins/node-test-pull-request-lite-pipeline/detail/node-test-pull-request-lite-pipeline/762/pipeline

tools/msvs/install_tools/install_tools.bat

joaocgreis · 2018-09-02T00:11:46Z

CI: https://ci.nodejs.org/job/node-test-pull-request/16930/
Test build: https://nodejs.org/download/test/v11.0.0-test201809029f195144ce/

tools/msvs/msi/product.wxs

joaocgreis · 2018-09-02T00:24:49Z

@jdalton updated for https. This follows the instructions at https://boxstarter.org/Learn/WebLauncher. The script comes from https://github.com/chocolatey/boxstarter/blob/master/BuildScripts/bootstrapper.ps1, but since this is an external tool, shouldn't we be using their published and stable version?

jdalton · 2018-09-02T00:45:21Z

@joaocgreis

Ah nice! They have tagged stable versions too. Are those tags exposed via the web address?

joaocgreis · 2018-09-02T01:04:27Z

@jdalton not that I know of... There is https://github.com/chocolatey/boxstarter/releases/latest, but I don't know of an easy way to access the files from there.

I'm not sure I understand your issue with using the Boxstarter site directly. We're trusting an external tool, Boxstarter (and Chocolatey), and it'll run a lot more scripts when installing the tools. It's a well established tool, so I believe this makes sense. If the issue is about their website getting hacked, isn't that also a concern with GitHub or the Chocolatey packages themselves?

jdalton · 2018-09-02T01:35:59Z

@joaocgreis

You're right that it's going to be pulling in plenty of other dependencies. I was just curious mostly. The switch to https is win though, so thanks!

\cc @ferventcoder as the creator/lead of Chocolatey and the publisher of Node releases on Chocolatey. This PR is introducing the use of chocolatey/boxstarter so also pinging @mwrock for visibility.

BridgeAR · 2018-09-05T13:34:27Z

Resumed CI https://ci.nodejs.org/job/node-test-commit/21240/

richardlau · 2018-09-05T14:34:44Z

tools/msvs/msi/i18n/en-us.wxl

+ <String Id="NativeToolsDlgTitle">{\WixUI_Font_Title}Tools for Native Modules</String>
+ <String Id="NativeToolsDlgDescription">Optionally install the tools necessary to compile native modules.</String>
+ <String Id="NativeToolsDlgBannerBitmap">WixUI_Bmp_Banner</String>
+ <String Id="NativeToolsDlgIntro">Some npm modules need to be compiled from C/C++ when installing. If you want to be able to install such modules, some tools (Python 2 and VS Build Tools) need to be installed.</String>


nit: VS Build Tools -> Visual Studio Build Tools

richardlau · 2018-09-05T15:09:59Z

For reference, this is what the additional dialog looks like (with the installer from the test build link in #22645 (comment)):

targos · 2018-09-05T15:17:21Z

What happens if Visual Studio is already installed and one checks the box?

joaocgreis · 2018-09-05T16:50:44Z

@richardlau updated, thanks!

@targos VS2017 supports installations of different VS editions side by side. So this should just install the VS Build Tools or update with the required components if already installed.

joaocgreis · 2018-09-05T16:51:56Z

Updated screenshots:

richardlau · 2018-09-05T17:35:24Z

FWIW the installer isn't built by the regular PR CI's but João did a test build in #22645 (comment) and I have separately built the installer locally with the current changes for this PR with Visual Studio 2017 and WiX Toolset 3.11.1.

Tested the dialog pops up as expected, that the link to https://github.com/nodejs/node-gyp#on-windows works and the batch file starts at the end of the installation (if the checkbox is ticked) and from the Start menu link (both times I closed the Window to cancel as I didn't want to make changes to my development machine).

Add a dialog during installation with information about native modules that can optionally run a Boxstarter script at the end of the installation. This script can also be run from Start menu. Fixes: #22311 PR-URL: #22645 Reviewed-By: Richard Lau <riclau@uk.ibm.com>

joaocgreis · 2018-09-12T02:23:19Z

Landed in 3993793

Thanks @richardlau!

oliversalzburg · 2018-10-11T09:40:04Z

Why not npm install windows-build-tools? Seems like that would be a much more obvious choice than the choco approach taken here.

joaocgreis · 2018-10-11T16:06:26Z

@oliversalzburg essentially for reliability. It would be great if installing things would be just start the installer and wait for it to complete, but there are many details that can make it fail. Boxstarter and Chocolatey are solutions dedicated to make automatic installations work well, and handle many points for us (installing Windows upgrades, reboots, and so on). windows-build-tools is a nice tool for anyone familiar with npm, but not as reliable as Boxstarter and Chocolatey. In fact, I believe that windows-build-tools would be better if it used something like the script here in the background. I'm not claiming the solution here will work perfectly in every situation, but it already works quite well and there's a community invested in improving it.

There are also other points to consider, that we could probably work around but don't have to with this. When new versions of Python or VS come out the current script should be able to upgrade without issues, while windows-build-tools would have to be updated and tested by us. Also, Node currently does not come with any module installed. We would have to consider where to install and how well it would work for other users in the machine.

So, the current solution should be as close as possible to "one click and it works". For anyone wanting more control, this includes links to instructions that should be quite simple to follow and one of the options there is windows-build-tools.

gep13 · 2018-10-12T18:54:51Z

I wanted to make sure that @yodurr, @crutkas and @bitcrazed had seen this as well. The usage of BoxStarter and Chocolatey here is similar to what is being created here: https://github.com/Microsoft/windows-dev-box-setup-scripts for different developer setups.

crutkas · 2018-10-12T19:19:28Z

would love to better understand the exact needs and what can be done. @yodurr without a doubt would too.

Notable changes: * assert * The diff output is now a tiny bit improved by sorting object properties when inspecting the values that are compared with each other. #22788 * cli * The options parser now normalizes `_` to `-` in all multi-word command-line flags, e.g. `--no_warnings` has the same effect as `--no-warnings`. #23020 * Added bash completion for the `node` binary. To generate a bash completion script, run `node --completion-bash`. The output can be saved to a file which can be sourced to enable completion. #20713 * crypto * Added support for PEM-level encryption. #23151 * Added an API asymmetric key pair generation. The new methods `crypto.generateKeyPair` and `crypto.generateKeyPairSync` can be used to generate public and private key pairs. The API supports RSA, DSA and EC and a variety of key encodings (both PEM and DER). #22660 * fs * Added a `recursive` option to `fs.mkdir` and `fs.mkdirSync`. If this option is set to true, non-existing parent folders will be automatically created. #21875 * http2 * Added a `'ping'` event to `Http2Session` that is emitted whenever a non-ack `PING` is received. #23009 * Added support for the `ORIGIN` frame. #22956 * Updated nghttp2 to 1.34.0. This adds RFC 8441 extended connect protocol support to allow use of WebSockets over HTTP/2. #23284 * module * Added `module.createRequireFromPath(filename)`. This new method can be used to create a custom require function that will resolve modules relative to the filename path. #19360 * process * Added a `'multipleResolves'` process event that is emitted whenever a `Promise` is attempted to be resolved multiple times, e.g. if the `resolve` and `reject` functions are both called in a `Promise` executor. #22218 * url * Added `url.fileURLToPath(url)` and `url.pathToFileURL(path)`. These methods can be used to correctly convert between file: URLs and absolute paths. #22506 * util * Added the `sorted` option to `util.inspect()`. If set to `true`, all properties of an object and Set and Map entries will be sorted in the returned string. If set to a function, it is used as a compare function. #22788 * The `util.instpect.custom` symbol is now defined in the global symbol registry as `Symbol.for('nodejs.util.inspect.custom')`. #20857 * Added support for `BigInt` numbers in `util.format()`. #22097 * V8 API * A number of V8 C++ APIs have been marked as deprecated since they have been removed in the upstream repository. Replacement APIs are added where necessary. #23159 * Windows * The Windows msi installer now provides an option to automatically install the tools required to build native modules. #22645 * Workers * Debugging support for Workers using the DevTools protocol has been implemented. #21364 * The public `inspector` module is now enabled in Workers. #22769 * Added new collaborators: * digitalinfinity - Hitesh Kanwathirtha PR-URL: #23313

XieJiSS · 2018-10-20T11:12:49Z

nodejs/Release#369

Could it be possible to change the prompt (of the Boxstarter Script) into a Yes/No question? Some users may proceed without reading the text before.

i.e. Have you saved all your files and closed all open programs? (y/N)

jasnell · 2018-10-30T22:26:10Z

@joaocgreis, et al, this appears to be causing some issues for some users... #23838

tniessen · 2018-11-05T22:18:23Z

This also appears to cause nodejs/security-wg#439.

MPagel · 2018-11-13T19:55:59Z

What happens if Visual Studio is already installed and one checks the box?

At least for python, it installs a new copy, ignoring the prior presence of it on the system. It also alters the path such that the new Py path is before or instead of the old install, requiring either manual adjustments to path or re-install of pip libraries needed for non-node.js python scripts.

This reverts: Revision: 257a5e9 win: add prompt to tools installation script Revision: e9a2915 win: clarify Boxstarter behavior on install tools Revision: 3b895d1 win,msi: display license notes before installing tools Revision: cf284c8 win,msi: install Boxstarter from elevated shell Revision: 2b7e18d win,msi: highlight installation of 3rd-party tools Revision: ebf36cd win,msi: install tools for native modules PR-URL: nodejs#24344 Refs: nodejs#22645 Refs: nodejs#23987 Refs: nodejs/Release#369 Refs: nodejs#23838 Refs: nodejs/security-wg#439 Reviewed-By: João Reis <reis@janeasystems.com> Reviewed-By: Richard Lau <riclau@uk.ibm.com>

runewake2 · 2018-11-25T06:54:08Z

Can this be rolled back? It caused my PC to be added to a group policy, Windows Updates to be disabled, my PC to be hard restarted multiple times and littered my PC with a number of other permanent changes.

This reverts: Revision: 257a5e9 win: add prompt to tools installation script Revision: e9a2915 win: clarify Boxstarter behavior on install tools Revision: 3b895d1 win,msi: display license notes before installing tools Revision: cf284c8 win,msi: install Boxstarter from elevated shell Revision: 2b7e18d win,msi: highlight installation of 3rd-party tools Revision: ebf36cd win,msi: install tools for native modules PR-URL: #24344 Refs: #22645 Refs: #23987 Refs: nodejs/Release#369 Refs: #23838 Refs: nodejs/security-wg#439 Reviewed-By: João Reis <reis@janeasystems.com> Reviewed-By: Richard Lau <riclau@uk.ibm.com>

win,msi: install tools for native modules

9f19514

Add a dialog during installation with information about native modules that can optionally run a Boxstarter script at the end of the installation. This script can also be run from Start menu. Fixes: nodejs#22311

joaocgreis added windows Issues and PRs related to the Windows platform. install Issues and PRs related to the installers. semver-minor PRs that contain new features and should be released in the next minor version. dont-land-on-v6.x labels Sep 1, 2018

nodejs-github-bot added install Issues and PRs related to the installers. tools Issues and PRs related to the tools directory. windows Issues and PRs related to the Windows platform. labels Sep 1, 2018

joaocgreis mentioned this pull request Sep 1, 2018

RFC: make node on Windows install, or prompt users to get, windows-build-tools #22311

Closed

jdalton reviewed Sep 1, 2018

View reviewed changes

tools/msvs/install_tools/install_tools.bat Outdated Show resolved Hide resolved

fixup: use https

2b632a0

refack reviewed Sep 2, 2018

View reviewed changes

tools/msvs/msi/product.wxs Show resolved Hide resolved

joaocgreis mentioned this pull request Sep 2, 2018

(GH-339) Exit from elevated shell after completion chocolatey/boxstarter#340

Closed

richardlau approved these changes Sep 5, 2018

View reviewed changes

fixup: Visual Studio

1d0872a

This was referenced Oct 11, 2018

2018-10-10 Version 10.12.0 (Current) @targos beevelop/docker-nodejs#228

Closed

2018-10-10 Version 10.12.0 (Current) @targos beevelop/docker-android-nodejs#243

Closed

2018-10-10 Version 10.12.0 (Current) @targos beevelop/docker-nodejs#229

Closed

vsemozhetbyt mentioned this pull request Oct 13, 2018

windows 10, nodejs 10.12.0 boxstarter dependency force reboot nodejs/Release#369

Closed

jasnell mentioned this pull request Oct 30, 2018

Node v11.0 windows installation opting into chocolatey+boxstarter results in infinite reboots #23838

Closed

joaocgreis mentioned this pull request Oct 31, 2018

win: add prompt to tools installation script #23987

Closed

2 tasks

refack mentioned this pull request Nov 5, 2018

Boxstarter asking for my Windows password in a shady PowerShell prompt nodejs/security-wg#439

Closed

refack mentioned this pull request Nov 5, 2018

tools,win: refactor install_tools.bat #24113

Closed

4 tasks

This was referenced Nov 7, 2018

Does not complete installation felixrieseberg/windows-build-tools#150

Open

Consider directing >=10.13.0 users to built-in script felixrieseberg/windows-build-tools#159

Open

This comment has been minimized.

Sign in to view

This was referenced Nov 8, 2018

build,win: open shortcut folder instead of running script #24237

Closed

Revert "win,msi: install tools for native modules" #24344

Merged

joaocgreis mentioned this pull request Nov 13, 2018

win: add customization warning to tools script #24348

Closed

2 tasks

This was referenced Nov 25, 2018

Installation via Node.JS Permanently Disables Windows Update chocolatey/boxstarter#371

Closed

Boxstarter Not Cleaned Up After Installation #24637

Closed

coagmano mentioned this pull request Jan 22, 2019

Install should include windows-build-tools on Windows meteor/meteor#10113

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

win,msi: install tools for native modules #22645

win,msi: install tools for native modules #22645

joaocgreis commented Sep 1, 2018

nodejs-github-bot commented Sep 1, 2018

joaocgreis commented Sep 2, 2018

joaocgreis commented Sep 2, 2018

jdalton commented Sep 2, 2018 •

edited

Loading

joaocgreis commented Sep 2, 2018

jdalton commented Sep 2, 2018

BridgeAR commented Sep 5, 2018

richardlau Sep 5, 2018

richardlau commented Sep 5, 2018

targos commented Sep 5, 2018

joaocgreis commented Sep 5, 2018

joaocgreis commented Sep 5, 2018

richardlau commented Sep 5, 2018

joaocgreis commented Sep 12, 2018

oliversalzburg commented Oct 11, 2018

joaocgreis commented Oct 11, 2018

gep13 commented Oct 12, 2018

crutkas commented Oct 12, 2018

XieJiSS commented Oct 20, 2018

jasnell commented Oct 30, 2018

tniessen commented Nov 5, 2018 •

edited

Loading

This comment has been minimized.

MPagel commented Nov 13, 2018 •

edited

Loading

runewake2 commented Nov 25, 2018

win,msi: install tools for native modules #22645

win,msi: install tools for native modules #22645

Conversation

joaocgreis commented Sep 1, 2018

Checklist

nodejs-github-bot commented Sep 1, 2018

joaocgreis commented Sep 2, 2018

joaocgreis commented Sep 2, 2018

jdalton commented Sep 2, 2018 • edited Loading

joaocgreis commented Sep 2, 2018

jdalton commented Sep 2, 2018

BridgeAR commented Sep 5, 2018

richardlau Sep 5, 2018

Choose a reason for hiding this comment

richardlau commented Sep 5, 2018

targos commented Sep 5, 2018

joaocgreis commented Sep 5, 2018

joaocgreis commented Sep 5, 2018

richardlau commented Sep 5, 2018

joaocgreis commented Sep 12, 2018

oliversalzburg commented Oct 11, 2018

joaocgreis commented Oct 11, 2018

gep13 commented Oct 12, 2018

crutkas commented Oct 12, 2018

XieJiSS commented Oct 20, 2018

jasnell commented Oct 30, 2018

tniessen commented Nov 5, 2018 • edited Loading

This comment has been minimized.

MPagel commented Nov 13, 2018 • edited Loading

runewake2 commented Nov 25, 2018

jdalton commented Sep 2, 2018 •

edited

Loading

tniessen commented Nov 5, 2018 •

edited

Loading

MPagel commented Nov 13, 2018 •

edited

Loading