Verify build / test / package / release scripts/tasks work on Windows

[cpcallen]

There are lots of parts of the build / test / package / release process which may depend on UNIX features (e.g. ,bash, mkdir -p, etc.) that may not exist on Windows.

Describe the solution you'd like

We should ascertain which of the npm scripts work properly on Windows, fix any which are easy to fix, and document any which work and which do not as well as whatever preconditions are required to make them work (e.g., do you need WSL?)

Describe alternatives you've considered

Let would-be Blockly contributors who use Windows fend for themselves.

Additional context

See also #4983 re: npm script documentation.

[yamadayutaka]

I suggest one of the solutions to run tests on Windows (using WSL).

Preparation

On Windows.

1. Install WSL.
2. Install a Linux distribution (I used Ubuntu 20.04).
3. Install Google Chrome (if not installed).
4. Install Node.js (I used 14.x).

In WSL.

1. Install Node.js (I used 14.x).

Problem

Simply run "npm run test" in WSL then following error occurred.

== mocha
Starting webdriverio...
Error: connect ECONNREFUSED 127.0.0.1:46825
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1157:16) {
  errno: -111,
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: '127.0.0.1',
  port: 46825
}
FAILED: mocha

While Chrome is running on Windows, Node.js is running on WSL, so webdriverio can't connect to Chrome.

Solution

I wrote the following wrapper script.
When you call a Windows binary in WSL, you will use the mechanism that the program is executed on Windows world.

#!/bin/bash

script=$1
win_script=${script//\//\\\\}

if [ -f /proc/sys/fs/binfmt_misc/WSLInterop ]; then
  # Windows environment (WSL).
  node.exe $win_script
else
  # Other environment.
  node $script
fi

And I changed the part that calls the test of webdriverio as follows.

• before

# Run Mocha tests inside a browser.
run_test_command "mocha" "node tests/mocha/run_mocha_tests_in_browser.js"

• after

# Run Mocha tests inside a browser.
run_test_command "mocha" "tests/scripts/run_browser_test.sh tests/mocha/run_mocha_tests_in_browser.js"

This will cause both Node.js and Chrome to launch on Windows, so it works fine.

All the corresponding content is in my branch.
https://github.com/yamadayutaka/blockly/tree/feature/run_test_on_windows

If there is no problem with this solution, I would like to PR it.

[BeksOmega]

Heya @yamadayutaka Thank you for looking into this =) I think this is going to be something @cpcallen needs to look at, and he's currently on vacation. So we'll probably have more info for you sometime next week! Thank you again :D

[yamadayutaka]

I suggest another solution to run build/tests on Windows (without WSL).

Preparation

npm config is not required if you just want to run build/build-debug without run test.

1. Install Python(3.x).
2. Set npm shell to bash.exe. (e.g. npm config set script-shell 'C:\Program Files\Git\bin\bash.exe')

Problems

1. grep command not found.
2. bc command not found.
3. closure-make-deps makes unexpected path (Probably there is a bug that doesn't work properly on Windows).
4. closure-calculate-chunks makes unexpected path (Path separator is "\").
5. deps.js/deps.mocha.js is created with a Windows line feed (CRLF).

Solution

1. Use JavaScript instead of grep.
2. Use awk instead of bc.
3. Use depswriter.py from google-closure-library. However it is workaround because depswriter.py is deprecated.
4. Replace "\" with "/" in the path generated by closure-calculate-chunks.
5. When a file is generated by redirection, it is created with an OS-dependent line feed, so generate the file with JavaScript.

All the corresponding content is in my branch.
https://github.com/yamadayutaka/blockly/tree/feature/run_build_on_windows

If there is no problem with this solution, I would like to PR it.

[yamadayutaka]

As I also commented on another issue, the closure-make-deps command is not working properly on Windows.
#5967 (comment)

I created a PR in the "closure-library" to solve this problem.
google/closure-library#1160

[BeksOmega]

Hi @yamadayutaka It turns out @cpcallen is on leave for a bit, so I'll try to pick this up. I should be able to test with a windows machine tomorrow, and then I'll update you.

Thank you again for working on this!

[cpcallen]

Just popping my head in to say thanks, @yamadayutaka, for your investigation. @BeksOmega: ideally we would have a Windows user on the core team who could work on this. Unfortunately I'm currently a macOS user but if we don't have any native Windows devs then I guess one of us (including myself) could get a virtual Windows machine set up for this purpose.

[BeksOmega]

Just popping my head in to say thanks, @yamadayutaka, for your investigation. @BeksOmega: ideally we would have a Windows user on the core team who could work on this. Unfortunately I'm currently a macOS user but if we don't have any native Windows devs then I guess one of us (including myself) could get a virtual Windows machine set up for this purpose.

I have a windows laptop, it's just not with me today. I used to dev on windows, I just haven't done it in several years. I'm hoping I still remember how it works a bit hehe. At least enough to test this anyway 🤞