[Windows] vitest exits silently without watching or printing coverage when using canvas

[kleinfreund]

Describe the bug

(It seems the issue described here applies to Windows only. All failing cases listed below pass on Ubuntu via WSL2 in Windows but fail in Windows PowerShell.)

For some runs of vitest,

• it doesn't watch. It does print the watch output (i.e. "Waiting for file changes...", etc.).
• it doesn't print a coverage report. It does print the coverage output (i.e. "% Coverage report from c8").

Reproduction

1. Clone https://github.com/kleinfreund/vue-accessible-color-picker/tree/a650e7fd451e1ea23eeb2f95ce4d29ec8183b3c2 (sorry, can't reproduce this in a more minimal case but the repo should be small enough), run npm install.
2. Set test.singleThread to true in vite.config.js.
3. Run the following commands:
   • npx vitest
     ❌ doesn't watch
   • npx vitest .\src\utilities\
     ❌ doesn't watch
   • npx vitest --coverage
     ❌ doesn't watch, doesn't print coverage report
   • npx vitest --coverage .\src\utilities\
     ❌ doesn't watch, doesn't print coverage report
   • npx vitest run --coverage
     ❌ doesn't print coverage report
   • npx vitest run --coverage .\src\utilities\
     ❌ doesn't print coverage report
   • npx vitest .\src\utilities\color-conversions\
     ✅ watches
   • npx vitest --coverage .\src\utilities\color-conversions\
     ✅ watches, prints coverage report
   • npx vitest run --coverage .\src\utilities\color-conversions\
     ✅ prints coverage report

More context

• Running tests using wildcards in path segments doesn't seem to work whatsoever. All the following runs produce the "No test files found, exiting with code 1" error. Path separator doesn't seem to matter; filter, include, exclude, and watch exclude values seem to all be correct and correctly normalized. I suspect this is some sort of lack of path resolving against the current working directory in Windows environments.
  • npx vitest run --coverage .\src
    ✅
  • npx vitest run --coverage .\src\**\*.test.js
    ❌ prints "No test files found, exiting with code 1"
  • npx vitest run --coverage .\src\utilities\*.test.js
    ❌ prints "No test files found, exiting with code 1"
  • npx vitest run --coverage .\src\utilities\**\*.test.js
    ❌ prints "No test files found, exiting with code 1"
• This is not related to Vue. I've ported the package to a separate web component and the exact same issue occurs in that project as well.

System Info

System:
    OS: Windows 10 10.0.22621
    CPU: (16) x64 Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
    Memory: 53.80 GB / 63.92 GB
  Binaries:
    Node: 20.5.0 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.19 - C:\Program Files\nodejs\yarn.CMD
    npm: 9.8.1 - ~\dev\vue-accessible-color-picker\node_modules\.bin\npm.CMD
  npmPackages:
    vite: ^4.4.8 => 4.4.8
    vitest: ^0.34.1 => 0.34.1

Used Package Manager

npm

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[sheremet-va]

(Vitest won't clear the terminal in the next version)

[kleinfreund]

I've narrowed this down to likely be an issue on Windows specifically. I cannot reproduce any of the failing cases provided above on Ubuntu (via WSL2). I've updated the original report accordingly.

[sheremet-va]

Is it still an issue?

[kleinfreund]

@sheremet-va I’m afraid so. I’ve checked this again a few days ago and updated the original report with (newer) affected versions.

[sheremet-va]

Running tests using wildcards in path segments doesn't seem to work whatsoever. All the following runs produce the "No test files found, exiting with code 1" error. Path separator doesn't seem to matter; filter, include, exclude, and watch exclude values seem to all be correct and correctly normalized

This depends on the shell you are using. Linux automatically wraps this and passed down an array of files. Vitest only does regexp with these files.

[mendrik]

I have same problem but maybe this stack trace helps (works on linux)

> vitest run --silent --coverage

Error: Cannot find module './lib/coverage-map'
Require stack:
- C:\desk\*****\***-*****\node_modules\istanbul-lib-coverage\index.js
- C:\desk\*****\***-*****\node_modules\c8\lib\report.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1039:15)
    at Module._load (node:internal/modules/cjs/loader:885:27)
    at Module.require (node:internal/modules/cjs/loader:1105:19)
    at require (node:internal/modules/cjs/helpers:103:18)
    at Object.<anonymous> (C:\desk\*****\***-*****\node_modules\istanbul-lib-coverage\index.js:16:25)
    at Module._compile (node:internal/modules/cjs/loader:1218:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1272:10)
    at Module.load (node:internal/modules/cjs/loader:1081:32)
    at Module._load (node:internal/modules/cjs/loader:922:12)
    at Module.require (node:internal/modules/cjs/loader:1105:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    'C:\\desk\\*****\\***-*****\\node_modules\\istanbul-lib-coverage\\index.js',
    'C:\\desk\\*****\\***-*****\\node_modules\\c8\\lib\\report.js'
  ]
}

[kleinfreund]

@mendrik Not sure this is caused by the same underlying issue. I'm not getting any errors when running the commands above. For me, it prints the regular test output and then:

 % Coverage report from c8

After which it simply exits.

[kleinfreund]

After updating several dependencies in my project, I can no longer reproduce this issue. All commands reported as failing above now behave as expected. The following dependencies related to this issue were updated (most likely a change in https://github.com/vitest-dev/vitest/releases/tag/v0.29.0 fixes the reported issue):

• @vitest/coverage-c8 ^0.28.3 -> ^0.29.1
• c8 ^7.12.0 -> ^7.13.0
• vite ^4.0.4 -> ^4.1.4
• vitest ^0.28.3 -> ^0.29.1

[kleinfreund]

I'm reopening this because this issue is only addressed as long as I don't enable --single-thread.

As part of https://github.com/vitest-dev/vitest/releases/tag/v0.29.0, #740 was fixed by offering the alternative to set --single-thread instead of --no-threads:

If you were using --no-threads before, you might consider enabling --single-thread instead [...]

Now, while keeping --no-threads in my project, the issues reported here cannot be reproduced. However, switching to --single-thread re-introduces them.

[kleinfreund]

I've updated the original post to reflect current package versions and using singleThread: true instead of threads: false.

[kleinfreund]

Trying to find where this breaks in the vitest code, stepping through code in the vendor-node bundle.

I'm running some of the commands from my original post above in both a Windows and an Ubuntu environment.

I noticed that in createThreadsPool, the call await pool.run(data, ...) doesn't always resolve before the test run is over (i.e. the process exits).

vitest/packages/vitest/src/node/pools/threads.ts

Line 93 in a08e599

await pool.run(data, { transferList: [workerPort], name })

• Ubuntu, without --coverage: does always resolve
• Ubuntu, with --coverage: does never resolve (process exits before, port.close() in finally branch is never called)
• Windows, without --coverage: does not always resolve (process exits before, port.close() in finally branch is never called)
• Windows, with --coverage: does never resolve (process exits before, port.close() in finally branch is never called)

When pool.run doesn't resolve, the process exits meaning the rest of the function won't be executed (neither will the post.close call in the finally branch). That explains why the coverage report isn't always printed.

This all sounds like a race condition somewhere. Maybe something in Tinypool or the interactions with it goes wrong?

More notes:

• When the test run exits early (i.e. without coverage report), it doesn't seem to be a response to one of the many process.exit calls in the code.

[kleinfreund]

I think I figured out why this happens.

One of the files in my project (src/utilities/parse-props-color.js) uses canvas (document.createElement('canvas').getContext('2d')). Removing the tests that import this file fixes the issue. I have the canvas package installed in the project, too.

It's surprising that the tests don't crash or throw an error of a sort. How come it just silently exits?

[BretHudson]

I am seeing similar behavior when using jsdom and canvas! Vitest runs a subset of my tests and then crashes, outputting Exit status 3221225477. I am on Windows, and am just calling code I wrote in the test, no third-party libraries/frameworks/packages.

Commenting out the following in my imported code resolves the crash:

const pixelCanvas =
	typeof OffscreenCanvas !== 'undefined'
		? new OffscreenCanvas(1, 1)
		: document.createElement('canvas');
const _pixelCtx = pixelCanvas.getContext('2d');
if (!_pixelCtx) {
	throw Error('pixelCtx failed to create');
}
const pixelCtx = _pixelCtx;

Ping if you think having access to the code would help & I can gladly send it over :)

[AriPerkkio]

The canvas package is unstable when run in node:worker_threads. And also the old --no-threads options seems to have fixed this so I'll close this issue with following:

Fixed by #3925 and #4172.

Vitest 1.0.0 has now --pool=forks which uses multiple node:child_process in parallel. The previous --no-threads was using a single node:child_process.

If the code you are testing is incompatible with node:worker_threads, switch to --pool=forks. If you run into same error with that pool, feel free to open new issue with minimal reproduction.

[BretHudson]

Vitest 1.0.0 has now --pool=forks which uses multiple node:child_process in parallel. The previous --no-threads was using a single node:child_process.

--pool=forks worked perfectly for me, thank you!