Skip to content

Commit 4822d04

Browse files
feat!: rewrite pools without tinypool (#8705)
Co-authored-by: Vladimir Sheremet <sleuths.slews0s@icloud.com>
1 parent 9dc1867 commit 4822d04

File tree

117 files changed

+2432
-3088
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

117 files changed

+2432
-3088
lines changed

CONTRIBUTING.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -149,7 +149,7 @@ Avoid deps that has large transitive dependencies that results in bloated size c
149149

150150
If there are libraries that are needed and don't comply with our size
151151
requirements, a fork can be tried to reduce its size while we work with them to
152-
upstream our changes (see [tinypool](https://github.com/tinylibs/tinypool) for example)
152+
upstream our changes.
153153

154154
### Think before adding yet another option
155155

README.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,6 @@ Next generation testing framework powered by Vite.
4141
- [JSDOM](https://github.com/jsdom/jsdom) and [happy-dom](https://github.com/capricorn86/happy-dom) for DOM and browser API mocking
4242
- [Browser Mode](https://vitest.dev/guide/browser/) for running component tests in the browser
4343
- Components testing ([Vue](https://github.com/vitest-tests/browser-examples/tree/main/examples/vue), [React](https://github.com/vitest-tests/browser-examples/tree/main/examples/react), [Svelte](https://github.com/vitest-tests/browser-examples/tree/main/examples/svelte), [Lit](./examples/lit), [Marko](https://github.com/marko-js/examples/tree/master/examples/library-ts))
44-
- Workers multi-threading via [Tinypool](https://github.com/tinylibs/tinypool) (a lightweight fork of [Piscina](https://github.com/piscinajs/piscina))
4544
- Benchmarking support with [Tinybench](https://github.com/tinylibs/tinybench)
4645
- [Projects](https://vitest.dev/guide/projects) support
4746
- [expect-type](https://github.com/mmkal/expect-type) for type-level testing

docs/.vitepress/components/FeaturesList.vue

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,6 @@
1010
<ListItem>Component testing for Vue, React, Svelte, Lit, Marko and more</ListItem>
1111
<ListItem>Out-of-the-box TypeScript / JSX support</ListItem>
1212
<ListItem>ESM first, top level await</ListItem>
13-
<ListItem>Workers multi-threading via <a target="_blank" href="https://github.com/tinylibs/tinypool" rel="noopener noreferrer">Tinypool</a></ListItem>
1413
<ListItem>Benchmarking support with <a target="_blank" href="https://github.com/tinylibs/tinybench" rel="noopener noreferrer">Tinybench</a></ListItem>
1514
<ListItem>Filtering, timeouts, concurrent for suite and tests</ListItem>
1615
<ListItem><a href="/guide/projects">Projects</a> support</ListItem>

docs/.vitepress/config.ts

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ export default ({ mode }: { mode: string }) => {
5252
['link', { rel: 'icon', href: '/favicon.ico', sizes: '48x48' }],
5353
['link', { rel: 'icon', href: '/logo.svg', sizes: 'any', type: 'image/svg+xml' }],
5454
['meta', { name: 'author', content: `${teamMembers.map(c => c.name).join(', ')} and ${vitestName} contributors` }],
55-
['meta', { name: 'keywords', content: 'vitest, vite, test, coverage, snapshot, react, vue, preact, svelte, solid, lit, marko, ruby, cypress, puppeteer, jsdom, happy-dom, test-runner, jest, typescript, esm, tinypool, tinyspy, node' }],
55+
['meta', { name: 'keywords', content: 'vitest, vite, test, coverage, snapshot, react, vue, preact, svelte, solid, lit, marko, ruby, cypress, puppeteer, jsdom, happy-dom, test-runner, jest, typescript, esm, tinyspy, node' }],
5656
['meta', { property: 'og:title', content: vitestName }],
5757
['meta', { property: 'og:description', content: vitestDescription }],
5858
['meta', { property: 'og:url', content: ogUrl }],
@@ -628,6 +628,10 @@ function guide(): DefaultTheme.SidebarItem[] {
628628
},
629629
],
630630
},
631+
{
632+
text: 'Recipes',
633+
link: '/guide/recipes',
634+
},
631635
]
632636
}
633637

docs/advanced/pool.md

Lines changed: 90 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -1,39 +1,44 @@
11
# Custom Pool
22

33
::: warning
4-
This is an advanced and very low-level API. If you just want to [run tests](/guide/), you probably don't need this. It is primarily used by library authors.
4+
This is an advanced, experimental and very low-level API. If you just want to [run tests](/guide/), you probably don't need this. It is primarily used by library authors.
55
:::
66

7-
Vitest runs tests in pools. By default, there are several pools:
7+
Vitest runs tests in a pool. By default, there are several pool runners:
88

99
- `threads` to run tests using `node:worker_threads` (isolation is provided with a new worker context)
1010
- `forks` to run tests using `node:child_process` (isolation is provided with a new `child_process.fork` process)
1111
- `vmThreads` to run tests using `node:worker_threads` (but isolation is provided with `vm` module instead of a new worker context)
1212
- `browser` to run tests using browser providers
1313
- `typescript` to run typechecking on tests
1414

15-
You can provide your own pool by specifying a file path:
15+
::: tip
16+
See [`vitest-pool-example`](https://www.npmjs.com/package/vitest-pool-example) for example of a custom pool runner implementation.
17+
:::
18+
19+
## Usage
20+
21+
You can provide your own pool runner by a function that returns `PoolRunnerInitializer`.
1622

1723
```ts [vitest.config.ts]
1824
import { defineConfig } from 'vitest/config'
25+
import customPool from './my-custom-pool.ts'
1926

2027
export default defineConfig({
2128
test: {
2229
// will run every file with a custom pool by default
23-
pool: './my-custom-pool.ts',
24-
// you can provide options using `poolOptions` object
25-
poolOptions: {
26-
myCustomPool: {
27-
customProperty: true,
28-
},
29-
},
30+
pool: customPool({
31+
customProperty: true,
32+
})
3033
},
3134
})
3235
```
3336

3437
If you need to run tests in different pools, use the [`projects`](/guide/projects) feature:
3538

3639
```ts [vitest.config.ts]
40+
import customPool from './my-custom-pool.ts'
41+
3742
export default defineConfig({
3843
test: {
3944
projects: [
@@ -43,54 +48,100 @@ export default defineConfig({
4348
pool: 'threads',
4449
},
4550
},
51+
{
52+
extends: true,
53+
test: {
54+
pool: customPool({
55+
customProperty: true,
56+
})
57+
}
58+
}
4659
],
4760
},
4861
})
4962
```
5063

5164
## API
5265

53-
The file specified in `pool` option should export a function (can be async) that accepts `Vitest` interface as its first option. This function needs to return an object matching `ProcessPool` interface:
66+
The `pool` option accepts a `PoolRunnerInitializer` that can be used for custom pool runners. The `name` property should indicate name of the custom pool runner. It should be identical with your worker's `name` property.
5467

55-
```ts
56-
import type { ProcessPool, TestSpecification } from 'vitest/node'
68+
```ts [my-custom-pool.ts]
69+
import type { PoolRunnerInitializer } from 'vitest/node'
5770

58-
export interface ProcessPool {
59-
name: string
60-
runTests: (files: TestSpecification[], invalidates?: string[]) => Promise<void>
61-
collectTests: (files: TestSpecification[], invalidates?: string[]) => Promise<void>
62-
close?: () => Promise<void>
71+
export function customPool(customOptions: CustomOptions): PoolRunnerInitializer {
72+
return {
73+
name: 'custom-pool',
74+
createPoolWorker: options => new CustomPoolWorker(options, customOptions),
75+
}
6376
}
6477
```
6578

66-
The function is called only once (unless the server config was updated), and it's generally a good idea to initialize everything you need for tests inside that function and reuse it when `runTests` is called.
79+
In your `CustomPoolWorker` you need to define all required methods:
80+
81+
```ts [my-custom-pool.ts]
82+
import type { PoolOptions, PoolWorker, WorkerRequest } from 'vitest/node'
6783

68-
Vitest calls `runTest` when new tests are scheduled to run. It will not call it if `files` is empty. The first argument is an array of [TestSpecifications](/advanced/api/test-specification). Files are sorted using [`sequencer`](/config/#sequence-sequencer) before `runTests` is called. It's possible (but unlikely) to have the same file twice, but it will always have a different project - this is implemented via [`projects`](/guide/projects) configuration.
84+
class CustomPoolWorker implements PoolWorker {
85+
name = 'custom-pool'
86+
private customOptions: CustomOptions
6987

70-
Vitest will wait until `runTests` is executed before finishing a run (i.e., it will emit [`onTestRunEnd`](/advanced/reporters) only after `runTests` is resolved).
88+
constructor(options: PoolOptions, customOptions: CustomOptions) {
89+
this.customOptions = customOptions
90+
}
7191

72-
If you are using a custom pool, you will have to provide test files and their results yourself - you can reference [`vitest.state`](https://github.com/vitest-dev/vitest/blob/main/packages/vitest/src/node/state.ts) for that (most important are `collectFiles` and `updateTasks`). Vitest uses `startTests` function from `@vitest/runner` package to do that.
92+
send(message: WorkerRequest): void {
93+
// Provide way to send your worker a message
94+
}
7395

74-
Vitest will call `collectTests` if `vitest.collect` is called or `vitest list` is invoked via a CLI command. It works the same way as `runTests`, but you don't have to run test callbacks, only report their tasks by calling `vitest.state.collectFiles(files)`.
96+
on(event: string, callback: (arg: any) => void): void {
97+
// Provide way to listen to your workers events, e.g. message, error, exit
98+
}
7599

76-
To communicate between different processes, you can create methods object using `createMethodsRPC` from `vitest/node`, and use any form of communication that you prefer. For example, to use WebSockets with `birpc` you can write something like this:
100+
off(event: string, callback: (arg: any) => void): void {
101+
// Provide way to unsubscribe `on` listeners
102+
}
77103

78-
```ts
79-
import { createBirpc } from 'birpc'
80-
import { parse, stringify } from 'flatted'
81-
import { createMethodsRPC, TestProject } from 'vitest/node'
104+
async start() {
105+
// do something when the worker is started
106+
}
82107

83-
function createRpc(project: TestProject, wss: WebSocketServer) {
84-
return createBirpc(
85-
createMethodsRPC(project),
86-
{
87-
post: msg => wss.send(msg),
88-
on: fn => wss.on('message', fn),
89-
serialize: stringify,
90-
deserialize: parse,
91-
},
92-
)
108+
async stop() {
109+
// cleanup the state
110+
}
111+
112+
deserialize(data) {
113+
return data
114+
}
93115
}
94116
```
95117

96-
You can see a simple example of a pool made from scratch that doesn't run tests but marks them as collected in [pool/custom-pool.ts](https://github.com/vitest-dev/vitest/blob/main/test/cli/fixtures/custom-pool/pool/custom-pool.ts).
118+
Your `CustomPoolRunner` will be controlling how your custom test runner worker life cycles and communication channel works. For example, your `CustomPoolRunner` could launch a `node:worker_threads` `Worker`, and provide communication via `Worker.postMessage` and `parentPort`.
119+
120+
In your worker file, you can import helper utilities from `vitest/worker`:
121+
122+
```ts [my-worker.ts]
123+
import { init, runBaseTests } from 'vitest/worker'
124+
125+
init({
126+
post: (response) => {
127+
// Provide way to send this message to CustomPoolRunner's onWorker as message event
128+
},
129+
on: (callback) => {
130+
// Provide a way to listen CustomPoolRunner's "postMessage" calls
131+
},
132+
off: (callback) => {
133+
// Optional, provide a way to remove listeners added by "on" calls
134+
},
135+
teardown: () => {
136+
// Optional, provide a way to teardown worker, e.g. unsubscribe all the `on` listeners
137+
},
138+
serialize: (value) => {
139+
// Optional, provide custom serializer for `post` calls
140+
},
141+
deserialize: (value) => {
142+
// Optional, provide custom deserializer for `on` callbacks
143+
},
144+
runTests: state => runBaseTests('run', state),
145+
collectTests: state => runBaseTests('collect', state),
146+
})
147+
```

0 commit comments

Comments
 (0)