[RFC] Storybook Vitest integration

[yannbf]

RFC: Testing stories with Vitest via a plugin

Status: Draft

1. Summary

This RFC proposes a new method for testing stories in Storybook using a Vitest plugin. The plugin will transform story files into test files using the portable stories API, providing a seamless integration with Vitest which will improve the developer experience for Vite users.

As a result, you get:

• Fast tests, based on Vitest speed
• Seamless integration in your Vitest setup, having a single testing environment instead of multiple (e.g. Jest + Vitest)
• Stories are turned into Vitest tests without any boilerplate
• Vitest VSCode extension integration
• Compatibility with Vitest features (e.g. code coverage, reports, browser mode, etc)

2. The Problem

The current Storybook test-runner is based on a Jest + Playwright integration, which, while effective, may not align with the development tools and workflows used by all teams. It depends on jest-playwright, a package that has been deprecated since 2021 and is no longer maintained.

The Vite ecosystem has grown a lot since the Storybook test-runner was released, and nowadays many Storybook projects are Vite based. For these projects, there are common requests/complaints from users:

• Lacking a seamless integration with Vitest. Users would prefer the "stories tests" to be part of their current testing setup, not a separate one with different tools.
• Having to install and configure Jest while they are using Vitest for their non-Storybook tests, or install extra Jest dependencies like jest-image-snapshot, jest-junit and others for additional features instead of using solutions they might already have for Vitest.
• Having to use a special Storybook addon to compute code coverage depending on the user’s framework, while such feature is built-in in Vitest.
• Having to run (or build and serve) Storybook in separate commands instead of having a seamless integration in a single "test" command (e.g. using Vitest's workspace configuration).
• Having slower test times because it requires Storybook to be running in dev mode (slow to test), or to be built and served (slow to build).

3. Non-goals

• Replacing the existing Storybook test-runner. It is still relevant, especially for non-Vite projects.
• Extending support to Webpack or RSpack based projects at this stage.
• Supporting renderers beyond those currently available in the portable stories API (React, Vue3, and Svelte).

4. Proposed Solution

The proposed solution involves creating a Vitest plugin that leverages the portable stories API to transform story files into test files.

Registering the plugin

You can choose one of the two possible ways:

1. In a Vitest config file. This will mix the plugin configuration with the rest of your test:

// vitest.config.ts
import { defineConfig, mergeConfig } from 'vitest/config'
import storybookPlugin from '<the-storybook-vitest-plugin>'

import viteConfig from './vite.config'

export default mergeConfig(
  viteConfig,
  defineConfig({
    plugins: [
      storybookPlugin(),
    ],
    test: {
      // whatever overrides you like here
    },
  })
)

This will probably not be the most recommended way, because you might want to test your stories slightly differently than your unit tests, for instance test stories in Browser mode while running unit tests in Node.
You could also set it up in a separate config file e.g. vitest.storybook.config.ts which is passed on your test command e.g. vitest -c vitest.storybook.config.ts. However if you are planning on creating separate test environments, you should instead use Vitest workspaces.

2. In a Vitest workspace file:

// vitest.workspace.ts
import { defineWorkspace } from 'vitest/config'
import storybookPlugin from '<the-storybook-vitest-plugin>'

export default defineWorkspace([
  {
    extends: './vite.config.ts',
    plugins: [
      storybookPlugin(),
    ],
    test: {
      setupFiles: './src/setupTests.ts',
      environment: 'happy-dom',
    },
  },
])

This will create a Vitest project specific to the Storybook plugin, which can be configured separately and won't impact other test configuration.

As a result, every story you have in your codebase is turned into tests automatically, which will be tested as part of your existing test command.

5. Implementation

Vitest plugin: Prerequisites

In order to use the plugin, we will assume that your project matches the following requisites:

• You use Storybook 8.x or higher
• You have Testing Library installed
• You have Vitest installed and configured

prelude: Portable stories API

Before talking about the implementation, it's important to understand about the portable stories API.
Users can use this API to reuse stories in separate environments, such as unit tests. To set it up, users have to follow three steps:

1. Define a set up file to register the project annotations:

// setupTest.ts
import { beforeAll } from 'vitest'
import { setProjectAnnotations } from '@storybook/react'
import { render } from '@testing-library/react'
import * as storybookAnnotations from './.storybook/preview'

const annotations = setProjectAnnotations([
  // add all the global annotations (args, parameters, loaders, decorators, etc.)
  storybookAnnotations,
  {
    // provide the mounting mechanism to be used by portable stories
    testingLibraryRender: render
  }
])

// optional, in case your preview file has a beforeAll (available in Storybook 8.2)
beforeAll(annotations.beforeAll)

2. Add that setup file to your test configuration:

// vitest.config.ts
import { defineConfig } from 'vitest/config'

export default defineConfig({
  // ...rest of the config
  test: {
    setupFiles: ['./setupTest.ts'],
  },
})

3. Compose stories in your test file:

// ./Button.test.tsx
import { describe, test } from "vitest";
import { render } from "@testing-library/react";
import { composeStories } from "@storybook/react";
import * as stories from "./Button.stories.tsx";

const { Primary } = composeStories(stories);

describe("Button", () => {
  test("Primary", async () => {
    await Primary.play();
  });
});

By running the Button tests, this API allows for:

• Composing and executing stories with all their annotations (args, parameters, loaders, decorators, etc.).
• Running smoke tests for stories without play functions.
• Executing tests for stories with play functions.

This essentially recreates the behavior of Storybook (of preparing and testing the story), however in an external environment, without having to run Storybook.

Vitest plugin: Implementation

The Vitest plugin will leverage the portable stories API and essentially provide all the steps (described above) for setting it up, without you having to do any of them yourself. It will either receive the renderer (e.g. react, vue3) as an option, or it will try to figure it out based on your Storybook config file.

1. Virtual file for setting project annotations and hooks

The plugin will create and register virtual setup file that includes all the boilerplate mentioned previously, which will only exist virtually during the test run.

2. Composing stories in a test file

The plugin will provide all the boilerplate for composing stories without having to write the test file yourself.

Here is an example transformation, based on the following story file:

import { Button } from "./Button";

const meta = {
  title: "Example/Button",
  component: Button,
};
export default meta;

export const Primary = {
  args: {
    primary: true,
    label: "Button",
  },
  play: async () => {
    // your tests and assertions
  },
};

export const Secondary = {
  args: {
    ...Primary.args,
    primary: false,
  },
};

The Vitest plugin will transform it into:

// <-- Original content of the file (removed for brevity) plus the lines below -->
import { describe, test } from "vitest";
// the react specific imports would be replaced by vue or svelte, based on user configuration
import { render } from "@testing-library/react";
import { composeStories } from "@storybook/react";
import * as stories from "/absolute/path/to/Button.stories.tsx";

const { Primary: PrimaryStory, Secondary: SecondaryStory } = composeStories(stories);

describe("Button", () => {
  test("Primary", async () => {
    // Storybook 8.2 portable stories api which executes all loaders/hooks,
    // mounts the story and executes the play function
    await PrimaryStory.play();
  });

  test("Secondary", async () => {
    await SecondaryStory.play();
  });
});

3. Spawning Storybook in background + "click to debug" experience

When running tests in watch mode, the plugin should spawn Storybook. The idea is to provide a "click to debug" experience where if your tests fail, the error message should include something like:

Click to debug the error directly in Storybook: http:<storybook-url-with-deep-link-to-story>

To make it simpler, the plugin could receive a command to execute instead of having to figure out how to run Storybook in the user's codebase (as it could be highly customized):

// vitest.config.ts
storybookPlugin({
  storybookScript: 'yarn storybook'
}),

Vitest reporters can be used to implement this behavior.

4. Automatic viewport dimension

If users are using @storybook/addon-viewport and running Vitest in Browser mode, the plugin should leverage the page object from the Browser context to set up the viewports correctly. This is necessary for tests to pass if they assert behavior that is container size dependenent.

5. Tags filtering

The plugin should support tags filtering via options, the same way the test-runner supports, e.g.:

// vitest.config.ts
storybookPlugin({
  tags: {
    include: ['foo'],
    exclude: ['bar'],
    skip: ['baz', 'skip']
  }
}),

it could use Vitest's skipIf and runIf APIs for that.

6. Prior Art

• Storybook Test-Runner: The existing Jest + Playwright solution is widely used but operates differently.
• Portable Stories API: Already in use within Storybook for rendering stories across different frameworks.

7. Deliverables

• Vitest Plugin: Development and release of the Vitest plugin for Storybook.
• Documentation: Comprehensive documentation on how to use the new testing approach.
• Examples: Sample projects demonstrating the new testing method.

8. Risks

• Compatibility: This project is limited to Vite-based projects, as well as technologies supported by the portable stories API (React, Vue3 and Svelte). We would need to clearly document the scope and limitations.
• Maintenance: There will be two solutions to maintain and keep up to date: Vitest plugin and the Storybook test-runner. We would need to establish a maintenance schedule and community contribution guidelines.

9. Unresolved Questions

• Can we extend support to other frameworks or bundlers in the future?
  Definitely, as long as we improve the portable stories API, we can broaden the support in the Vitest plugin. For non-Vite based projects that will not be the case, however.

• What additional features or improvements can be made to the portable stories API to enhance this integration?
  A big amount of improvements is being done in Storybook 8.2 which will make it easier to support more frameworks in the future.

10. Alternatives considered / Abandoned Ideas

• Maintaining the Status Quo: Continuing only with the existing Jest + Playwright test-runner solution. There is a big desire for a more integrated solution with Vitest, and at the same time there are some risks in only maintaining the Jest based test-runner because of its outdated solution (Jest falling out of preference, jest-playwright-preset being unmaintained).
• Replacing the test-runner from Jest+Playwright to Playwright only: There is an idea to replace the internals of the test-runner so it won't use Jest anymore, and therefore get rid of the jest-playwright-preset package. This means quite some effort because it should still maintain all the functionality of the existing implementation, and we would like to experiment with an innovative solution like the Vitest integration for the time being.

[HFryling]

I'm very interested in using this plugin. Is it still in progress?

[shilman]

This was released in Storybook 8.3. Give it a try!

[HFryling]

Thanks, I appreciate it!

…

________________________________ From: Michael Shilman ***@***.***> Sent: Monday, September 30, 2024 11:15 PM To: storybookjs/storybook ***@***.***> Cc: Heather Fryling ***@***.***>; Comment ***@***.***> Subject: Re: [storybookjs/storybook] [RFC] Storybook Vitest integration (Discussion #28386) This was released in Storybook 8.3<https://storybook.js.org/blog/storybook-8-3/>. Give it a try! — Reply to this email directly, view it on GitHub<#28386 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BJZDBUAVQ3FGBAIGX2MGKHDZZI4Y7AVCNFSM6AAAAABKCAHXJOVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAOBQGUYDGOI>. You are receiving this because you commented.Message ID: ***@***.***>