VS Code extension tests using vscode-test do not load

[carlocardella]

I an trying to use Mocha Test Explorer to run the integration tests for my VSCode extension, I am aware of limitations running tests from the command line but I am also trying to follow other docs and suggestions here:

• Running VS Code extension tests using vscode-test
• Cannot be used to debug VSCode extensions
• hello-world-testsample

I have these settings in my workspace (I am using VSCode Stable, so CODE_VERSION here is Insiders):

"mochaExplorer.logpanel": true,
  "mochaExplorer.files": "out/test/**/*.test.js",
  "mochaExplorer.launcherScript": "node_modules/mocha-explorer-launcher-scripts/vscode-test",
  "mochaExplorer.autoload": false,
  "mochaExplorer.ipcRole": "server",
  "mochaExplorer.env": {
    "VSCODE_VERSION": "insiders",
    "ELECTRON_RUN_AS_NODE": null,
    // "VSCODE_LAUNCH_ARGS": "--disable-extensions"
  },
  "mochaExplorer.esmLoader": false,
  "mochaExplorer.mochaPath": "node_modules/mocha",
  "mochaExplorer.ui": "tdd",
  "mochaExplorer.timeout": 999999,
  "testExplorer.showOnRun": true

I tried a number of different settings, but unfortunately Mocha Test Explorer never properly shows the tests.
Running tests, according to Mocha Explorer log, the test file is found (this is just a sample extension):

[2021-05-24 00:05:54.396] [INFO] Worker finished with code 0 and signal null
[2021-05-24 00:15:43.568] [INFO] Loading test files of c:\Users\carloc\Git\hbenl\vscode-extension-samples\helloworld-test-sample
[2021-05-24 00:15:43.573] [DEBUG] Using working directory: C:\Users\carloc\Git\hbenl\vscode-extension-samples\helloworld-test-sample
[2021-05-24 00:15:43.986] [DEBUG] Using nodePath: C:\Program Files\nodejs\node.exe
[2021-05-24 00:15:44.636] [DEBUG] Using Mocha options: {"ui":"tdd","timeout":999999,"retries":0,"requires":[],"delay":false,"fullTrace":false,"exit":false,"asyncOnly":false,"parallel":false}
[2021-05-24 00:15:44.636] [DEBUG] Looking for test files ["out/test/**/*.test.js"] in c:\Users\carloc\Git\hbenl\vscode-extension-samples\helloworld-test-sample
[2021-05-24 00:15:44.639] [DEBUG] Found test files ["C:\\Users\\carloc\\Git\\hbenl\\vscode-extension-samples\\helloworld-test-sample\\out\\test\\suite\\extension.test.js"] <== THIS IS THE PROPER TEST FILE
[2021-05-24 00:15:44.640] [DEBUG] Using environment variables from config: {"VSCODE_VERSION":"insiders","ELECTRON_RUN_AS_NODE":null}
[2021-05-24 00:15:44.643] [DEBUG] Spawning c:\Users\carloc\Git\hbenl\vscode-extension-samples\helloworld-test-sample\node_modules\mocha-explorer-launcher-scripts\vscode-test with IPC options {"role":"client","port":9449,"host":"localhost"}
[2021-05-24 00:15:45.203] [INFO] Worker (stdout): Found .vscode-test/vscode-insiders matching latest Insiders release. Skipping download.

[2021-05-24 00:15:45.358] [INFO] Worker (stdout): 

Anyway Test Explorer always shows this:

Couldn't establish IPC:
Error: IPC server timed out before receiving a client connection
    at Timeout.onTimeout [as _onTimeout] (c:\Users\carloc\.vscode\extensions\hbenl.vscode-mocha-test-adapter-2.11.1\node_modules\vscode-test-adapter-remoting-util\out\ipc.js:94:20)
    at listOnTimeout (internal/timers.js:554:17)
    at processTimers (internal/timers.js:497:7)

What am I missing? I can run tests launching the proper Task in VSCode, I was hoping to also have a visual representation in Test Explorer as for other languages (Go, .NET, Python etc...): is it possible?

Thanks

[hbenl]

Does the hello-world-testsample work for you? I.e. do you see the test from that repo in Mocha Test Explorer?

[carlocardella]

No, the logs and screenshot above are using the hello-world-testsample: since I was not able make it work with my extension I wanted to check with your sample (hoping it would have worked) to then hunt for differences between the two, but I cannot make it work with your sample either....

[carlocardella]

Worth mentioning: my VSCode takes a minute to load (I have way too many extensions installed... 🤔), since I could not have it start without extensions ("VSCODE_LAUNCH_ARGS": "--disable-extensions" did not work but that's a separate problem), I extended the timeout with "mochaExplorer.timeout": 999999, still with no luck

[hbenl]

my VSCode takes a minute to load

Ah, that could explain it: the mochaExplorer.timeout only sets the timeout for how long a test may run. You need to increase mochaExplorer.ipcTimeout instead.

[carlocardella]

That helped a bit, thanks, at least I can go past that error now. It is still failing though, with some error coming from other extensions.

On a different topic (I can open a separate issue if you prefer), I do not want other extensions loaded anyway when I'm testing mine, I am trying with:

  "mochaExplorer.env": {
    "VSCODE_VERSION": "insiders",
    "ELECTRON_RUN_AS_NODE": null,
    "VSCODE_LAUNCH_ARGS": "--disable-extensions"
  },

But I am getting:

[2021-05-24 18:31:07.965] [INFO] Test Explorer found
[2021-05-24 18:31:07.965] [INFO] Creating adapter for c:\Users\carloc\Git\hbenl\vscode-extension-samples\helloworld-test-sample
[2021-05-24 18:31:07.966] [INFO] Registering adapter for c:\Users\carloc\Git\hbenl\vscode-extension-samples\helloworld-test-sample
[2021-05-24 18:31:07.967] [INFO] Skipping the initial load request for c:\Users\carloc\Git\hbenl\vscode-extension-samples\helloworld-test-sample
[2021-05-24 18:31:07.967] [INFO] Initialization finished
[2021-05-24 18:31:27.021] [INFO] Loading test files of c:\Users\carloc\Git\hbenl\vscode-extension-samples\helloworld-test-sample
[2021-05-24 18:31:27.023] [DEBUG] Using working directory: C:\Users\carloc\Git\hbenl\vscode-extension-samples\helloworld-test-sample
[2021-05-24 18:31:27.239] [DEBUG] Using nodePath: C:\Program Files\nodejs\node.exe
[2021-05-24 18:31:27.610] [DEBUG] Using Mocha options: {"ui":"tdd","timeout":2000,"retries":0,"requires":[],"delay":false,"fullTrace":false,"exit":false,"asyncOnly":false,"parallel":false}
[2021-05-24 18:31:27.611] [DEBUG] Looking for test files ["out/test/**/*.test.js"] in c:\Users\carloc\Git\hbenl\vscode-extension-samples\helloworld-test-sample
[2021-05-24 18:31:27.616] [DEBUG] Found test files ["C:\\Users\\carloc\\Git\\hbenl\\vscode-extension-samples\\helloworld-test-sample\\out\\test\\suite\\extension.test.js"]
[2021-05-24 18:31:27.617] [DEBUG] Using environment variables from config: {"VSCODE_VERSION":"insiders","ELECTRON_RUN_AS_NODE":null,"VSCODE_LAUNCH_ARGS":"--disable-extensions"}
[2021-05-24 18:31:27.618] [DEBUG] Spawning c:\Users\carloc\Git\hbenl\vscode-extension-samples\helloworld-test-sample\node_modules\mocha-explorer-launcher-scripts\vscode-test with IPC options {"role":"client","port":9449,"host":"localhost"}
[2021-05-24 18:31:27.788] [ERROR] Worker (stderr): Failed to run tests: SyntaxError: Unexpected number in JSON at position 1
    at JSON.parse (<anonymous>)
    at c:\Users\carloc\Git\hbenl\vscode-extension-samples\helloworld-test-sample\node_modules\mocha-explorer-launcher-scripts\vscode-test\index.js:23:35
    at Generator.next (<anonymous>)
    at c:\Users\carloc\Git\hbenl\vscode-extension-samples\helloworld-test-sample\node_modules\mocha-explorer-launcher-scripts\vscode-test\index.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (c:\Users\carloc\Git\hbenl\vscode-extension-samples\helloworld-test-sample\node_modules\mocha-explorer-launcher-scripts\vscode-test\index.js:4:12)
    at c:\Users\carloc\Git\hbenl\vscode-extension-samples\helloworld-test-sample\node_modules\mocha-explorer-launcher-scripts\vscode-test\index.js:15:12
    at Object.<anonymous> (c:\Users\carloc\Git\hbenl\vscode-extension-samples\helloworld-test-sample\node_modules\mocha-explorer-launcher-scripts\vscode-test\index.js:42:3)
    at Module._compile (node:internal/modules/cjs/loader:1108:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1137:10)

This is still with your hello-world sample.

[carlocardella]

Interesting: when i refresh the test view, I see the spawned Insiders window (because I'm starting from Stable). Then Insiders hangs (because of an unrelated problem with one of the other extensions), when I kill it, the Test Explorer view shows the test as expected (but not till I kill Insiders):

So I am left with the error using --disable-extensions, I'm going to open a separate issue for that one, thanks for your help!

[lonix1]

I also had this problem, and only managed to solve it by adding the vscode-test package. I read that this was deprecated in lieu of @vscode/test-electron.

I'm new to this. Is there any harm in having both in my project?