feat!: coverage-c8 to use V8 profiler directly instead of NODE_V8_COVERAGE

[AriPerkkio]

Breaking changes:

• vitest and @vitest/coverage-c8 versions must match as the CoverageProviderModule introduces new APIs and c8-provider starts to use these.
• Removes CoverageProvider.onBeforeFilesRun. Custom coverage providers should use CoverageProviderModule.startCoverage instead.

Currently Vitest is using NODE_V8_COVERAGE environment variable to generate V8 coverage reports for @vitest/coverage-c8. We've used this environment variable to trigger Node to start writing JSON reports to <reports-dir>/tmp.

These reports contain coverage results of all run code, including Node's internal node:<module> and unnecessary third party **/node_modules/** packages. In addition the results contain source-map-cache information which is completely irrelevant for Vitest. There is so much data on these JSON reports. In #2108 user reported 435 MB coverage/tmp directory on CI.

Also using NODE_V8_COVERAGE with worker_threads has some weird features: nodejs/node#46378. I think it's meant more for running a single script instead of to be utilized by test runners.

This PR replaces process.env.NODE_V8_COVERAGE usage by controlling V8 Profiler directly using node:inspect. We don't enable unncessary source-map-cache and filter the ScriptCoverage results early so the amount of stored data is reduced a lot. Results are stored in memory instead of file system - exactly as @vitest/coverage-istanbul does. This reduces amount of file system handles. Since we now know what files are in the report, we can reduce the amount of Vite source map handling in provider.ts.

This should help with flaky coverage test on CI since workers no more write to <reports-dir>/tmp when they exit.

[AriPerkkio]

Minimal example for testing Profiler's coverage commands: https://gist.github.com/AriPerkkio/c2df034cd71b0dfcc934c477758c20ee

[AriPerkkio]

I did some manual regression testing with vuejs/core repository as these changes are quite critical. This seems to make report generation a lot faster now that amount of data is reduced, 3600ms -> 400ms:

Main branch:
provider.ts :: reportCoverage: 3.667s
provider.ts :: reportCoverage: 3.526s
provider.ts :: reportCoverage: 3.699s

PR branch:
provider.ts :: reportCoverage: 405.988ms
provider.ts :: reportCoverage: 399.793ms
provider.ts :: reportCoverage: 449.856ms

Coverage reports are almost identical. Branch count is off by one but after checking that one manually it doesn't make sense in either case.

Main:
=============================== Coverage summary ===============================
Statements   : 94.27% ( 39675/42086 )
Branches     : 92.07% ( 8987/9760 )
Functions    : 91.73% ( 2053/2238 )
Lines        : 94.27% ( 39675/42086 )
================================================================================

PR branch:
=============================== Coverage summary ===============================
Statements   : 94.27% ( 39675/42086 )
Branches     : 92.07% ( 8986/9759 )
Functions    : 91.73% ( 2053/2238 )
Lines        : 94.27% ( 39675/42086 )
================================================================================

[sheremet-va]

This seems to make report generation a lot faster now that amount of data is reduced, 3600ms -> 400ms:

These are great improvements 😱 👍🏻

[AriPerkkio]

❗ Some manual resolving will be required if #2817 is merged first. Custom provider test should be updated to use the new CoverageProviderModule's APIs as well.

[sheremet-va]

This PR has a conflict now since #2817 was merged first, as you said. 😄

[AriPerkkio]

I'll start to work on rebasing main here now.

[AriPerkkio]

Rebase done, most conflicts were in integrations/coverage.ts. Ubuntu+Node18 CI is failing on unrelated test.
The test/coverage-test/ tests pass and manual regression with vuejs/core gives good results:

# PR branch

 % Coverage report from c8

=============================== Coverage summary ===============================
Statements   : 94.27% ( 39675/42086 )
Branches     : 92.07% ( 8986/9759 )
Functions    : 91.73% ( 2053/2238 )
Lines        : 94.27% ( 39675/42086 )
================================================================================
reportCoverage: 418.102ms


# Main branch
 % Coverage report from c8

=============================== Coverage summary ===============================
Statements   : 94.27% ( 39675/42086 )
Branches     : 92.07% ( 8987/9760 )
Functions    : 91.73% ( 2053/2238 )
Lines        : 94.27% ( 39675/42086 )
================================================================================
reportCoverage: 3.828s

Now that #2836 is merged, should we bump the version here in this PR? Done. ✅

vitest/packages/coverage-c8/package.json

Lines 44 to 46 in 2672c58

	"peerDependencies": {
	"vitest": ">=0.28.0 <1"
	},