[@storybook/addon-storyshots] Support for other test runners (vitest)

[AndrewLeedham]

Is your feature request related to a problem? Please describe
@storybook/addon-storyshots is tied to running in a Jest environment, so in projects using other test runners it is hard to integrate.

Describe the solution you'd like
@storybook/addon-storyshots to support other test runners like Vitest.

Describe alternatives you've considered
I tried to replace the globals Jest provides with those from Vitest, but had no luck.

Are you able to assist to bring the feature to reality?
Maybe depends on what solution would fit best in Storybook.

Additional context
Some discussion of options can be seen here: vitest-dev/vitest#700 (comment)

[freakzlike]

I would be glad to see this. Once I tried to build the logic from @storybook/addon-storyshots-puppeteer to use Cypress instead, but failed. Maybe an API structure would be enough to get this working. I don't know if this already exists, but I am thinking of something like this:

import { getAllStories } from '@storybook/addon-storyshots'

getAllStories().forEach(story => {
  it(`should render story ${story.title}`, () => {
    const content = story.render()
    expect(content).toMatchSnapshot()
  })
})

Or maybe for using Cypress with image snapshots

import { getAllStories } from '@storybook/addon-storyshots'

getAllStories().forEach(story => {
  it(`should render story ${story.title}`, () => {
    cy.visit(story.url)
    cy.matchImageSnapshot()
  })
})

So getAllStories will return meta information of all existing stories and story.render() will render the story in the DOM.
This will remove the whole test runner logic from @storybook/addon-storyshots and it will just be a small test utils library.
I can help with the implementation, if necessary.

[AndrewLeedham]

That would make a lot of sense. So it would be something like:
@storybook/addon-storyshots: Core APIs for getting stories/loaders from Storybook. And then:
@storybook/addon-storyshots-jest, @storybook/addon-storyshots-vite, @storybook/addon-storyshots-puppeteer, @storybook/addon-storyshots-cypress which are all adapters for their respective test-runners.

Thoughts @shilman ?

[benbender]

Would love to see @storybook/addon-storyshots-vite & @storybook/addon-storyshots-playwright <3

[lmiller1990]

Also gonna sub to this issue! Vitest is great.

[djindjicdev22]

It's actually not possible to setup snapshot testing for new vanilla vue3 project since they are suggesting to not use jest.

any news on this topic?

[AndrewLeedham]

@djindjicdev22 I am currently working around this by using import.meta.globEager from Vite to get all the story files, using @storybook/testing-react to get the stories and @testing-library/react to render them, then just doing some basic smoke tests. The last 2 are react specific there might be vue alternatives.

In terms of the proposal I don't have time to work on it, so happy for someone to pick it up if they have time?

[jaminellis]

@djindjicdev22 I am currently working around this by using import.meta.globEager from Vite to get all the story files, using @storybook/testing-react to get the stories and @testing-library/react to render them, then just doing some basic smoke tests. The last 2 are react specific there might be vue alternatives.

In terms of the proposal I don't have time to work on it, so happy for someone to pick it up if they have time?

If you're happy to share that I'd have a go at doing something similar for Vue

[AndrewLeedham]

@jaminellis Sure, I have removed a bunch of project specific stuff, but this is the general idea:

/// <reference types="vite/client"/>
import { describe, vi, expect, test } from 'vitest';
import { render } from '@testing-library/react';
import { Meta, StoryFn } from '@storybook/react';
import { composeStories } from '@storybook/testing-react';

type StoryFile = {
  default: Meta;
  [name: string]: StoryFn | Meta;
};

describe('Storybook Smoke Test', async () => {
  const modules = await Promise.all(
    Object.values(
      import.meta.glob<StoryFile>(
        '../**/*.stories.tsx'
      )
    ).map((fn) => fn())
  );

  describe.each(modules)('%s', (module) => {
    test.each(
      Object.values(composeStories(module)).map((Story) => [
        Story.storyName!,
        Story,
      ])
    )('%s', (_, Story) => {
      const { container } = render(<Story />);
      expect(container).toBeTruthy();
      // We also assert if there were any `console.error` or `console.warn` calls here.
    });
  });
});

[MichaelBitard]

Thanks @AndrewLeedham!

For those searching for vue3, here is what we use:

import { describe, expect, test } from 'vitest'
import type { Meta, StoryFn } from '@storybook/vue3'
import { render } from '@testing-library/vue'
import { composeStories } from '@storybook/testing-vue3';
import type { ContextedStory } from '@storybook/testing-vue3/dist/types';


type StoryFile = {
  default: Meta;
  [name: string]: StoryFn | Meta;
};

describe('Storybook Smoke Test', async () => {
  const modules = await Promise.all(
    Object.values(
      import.meta.glob<StoryFile>(
        '../**/*.stories.ts'
      )
    ).map((fn) => fn())
  );
  describe.each(modules.map(module => ({ name: module.default.title, module })))('$name', ({ name, module }) => {
    test.skipIf(name?.includes("NoStoryshots")).each(Object.entries<ContextedStory<unknown>>(composeStories(module)).map(([name, story]) => ({ name, story })))('$name', ({ story }) => {
      const mounted = render(story());
      expect(mounted.html()).toMatchSnapshot();
    });
  });
});

[djindjicdev22]

@MichaelBitard can you share some example setup as a whole repository? I would like to run it also with vitest, but nor sure:

1. where to put your snippet code
2. how to generate snapshots

[MichaelBitard]

@djindjicdev22 sure, here is our exact code source in our repo: https://github.com/MTES-MCT/camino/blob/master/packages/ui/src/storybook.spec.ts

let me know if you need more information :)

[lucasriondel]

is there anything ongoing for a react storybook ?

[dohooo]

Are there any updates?

Maybe adding another option that gives a testing runner (vitest) is one solution?

import * as vitest from 'vitest';

shots({
testRunner: vitest
})

Because they have the same APIs.

[IsaiahByDayah]

@jaminellis Sure, I have removed a bunch of project specific stuff, but this is the general idea:

/// <reference types="vite/client"/>
import { describe, vi, expect, test } from 'vitest';
import { render } from '@testing-library/react';
import { Meta, StoryFn } from '@storybook/react';
import { composeStories } from '@storybook/testing-react';

type StoryFile = {
  default: Meta;
  [name: string]: StoryFn | Meta;
};

describe('Storybook Smoke Test', async () => {
  const modules = await Promise.all(
    Object.values(
      import.meta.glob<StoryFile>(
        '../**/*.stories.tsx'
      )
    ).map((fn) => fn())
  );

  describe.each(modules)('%s', (module) => {
    test.each(
      Object.values(composeStories(module)).map((Story) => [
        Story.storyName!,
        Story,
      ])
    )('%s', (_, Story) => {
      const { container } = render(<Story />);
      expect(container).toBeTruthy();
      // We also assert if there were any `console.error` or `console.warn` calls here.
    });
  });
});

Thank you @AndrewLeedham for sharing this! Im pretty new to vite / vitest / "manual" snapshot testing but this helped immensely!

For anyone using this for storybook (v6.5.15 tested) + vite + vitest + react, I did have to make the following adjustments:

• install and setup a browser testing env (personally went with happy-dom to give it a try and like it but suspect jsdom would also work)
• update some of the module/test naming (building off what @MichaelBitard shared for their vue solution)
• actually trigger snapshot testing in the test

After all said and done, this was my final working file:

// snapshots.test.tsx

/// <reference types="vite/client"/>
import { Meta, StoryFn } from "@storybook/react"
import { composeStories } from "@storybook/testing-react"
import { render } from "@testing-library/react"
import { describe, expect, test } from "vitest"

type StoryFile = {
  default: Meta
  [name: string]: StoryFn | Meta
}

describe("Storybook Snapshots", async () => {
  const modules = await Promise.all(
    Object.values(import.meta.glob<StoryFile>("../**/*.stories.tsx")).map(
      (fn) => fn()
    )
  )

  describe.each(
    modules.map((module) => ({ name: module.default.title, module }))
  )("$name", ({ module }) => {
    test.each(
      Object.values(composeStories(module)).map((Story) => [
        Story.storyName!,
        Story,
      ])
    )("%s", (_, Story) => {
      const { container } = render(<Story />)
      expect(container).toBeTruthy()
      expect(container).toMatchSnapshot()
    })
  })
})

[curtvict]

Oh boy. Thanks @IsaiahByDayah and @AndrewLeedham for the starter packs you've posted. This is my iteration:

// snapshots.test.tsx
/// <reference types="vite/client"/>
import { configureStore } from '@reduxjs/toolkit';
import { Meta, StoryFn } from '@storybook/react';
import { composeStories } from '@storybook/testing-react';
import { render } from '@testing-library/react';
import { Provider } from 'react-redux';
import { MemoryRouter } from 'react-router-dom';
import { ThemeProvider } from 'styled-components';
import { describe, expect, test } from 'vitest';

import { reducer } from 'redux/store';
import { GlobalStyles, TypographyStyles } from 'storybook/stories/globalStyles';
import theme from 'storybook/stories/theme';

type StoryFile = {
  default: Meta;
  [name: string]: StoryFn | Meta;
};

const store = configureStore({ reducer });

describe('Storybook Snapshots', async () => {
  const modulesByStoryName = await Promise.all(
    Object.entries(import.meta.glob<StoryFile>('./stories/**/*.stories.tsx')).map(([key, fn]) => {
      const chunks = key.split('/');
      const filename = chunks[chunks.length - 1];

      let storyName;

      if (filename.match(/index\.stories/)) {
        storyName = chunks[chunks.length - 2];
      } else {
        storyName = filename.replace(/\.stories\..*/, '');
      }

      return [storyName, fn()];
    })
  );

  describe.each(modulesByStoryName.map(([name, module]) => ({ name, module })))(
    '$name',
    async ({ module }) => {
      test.each(
        Object.values(composeStories(await (module as Promise<StoryFile>))).map((Story) => [
          Story.storyName!,
          Story,
        ])
      )('%s', (_, Story) => {
        const { container } = render(
          <Provider store={store}>
            <ThemeProvider theme={theme}>
              <GlobalStyles />
              <TypographyStyles />
              <MemoryRouter>
                <Story />
              </MemoryRouter>
            </ThemeProvider>
          </Provider>
        );
        expect(container).toBeTruthy();
        expect(container).toMatchSnapshot();
      });
    }
  );
});

We don't explicitly set the title attribute on our stories, since it's optional and can be automatically grokked by Storybook based on the directory structure

We're relying on an Atomic design directory structure in our Storybook, so we know that component names might be at the third position in the path. I say "might" because of this type of structure we're adhering to:

src/storybook/stories/atoms/MyAtom.stories.tsx
# or
src/storybook/stories/molecules/MyMolecule/index.stories.tsx

In the latter case we want to grab the parent folder name instead of the actual story file name, hence the conditional during the async import function. I'm returning a tuple but this probably could be simplified as an object reduction or an array of objects.

Finally our stories rely on some decoration. They also rely on some portals being set, which I'm handling in my Vitest setup file:

const setupReactPortals = () => {
  const alertsRoot = window.document.createElement('div');
  const popperRoot = window.document.createElement('div');

  alertsRoot.setAttribute('id', 'alerts-root');
  popperRoot.setAttribute('id', 'popper-root');

  window.document.body.appendChild(alertsRoot);
  window.document.body.appendChild(popperRoot);
};

beforeAll(() => {
  setupReactPortals();
  ...
});

It would be awesome if Storybook would release a plugin for first-class Vitest support though. Curious if @shilman feels like this might be roadmap-worthy. Supporting Vitest feels trivial since their APIs are 99% congruent with Jest.

[ndelangen]

The future of storyshots is the test-runner:
https://storybook.js.org/docs/react/writing-tests/test-runner#page-top

And use the play function for expectations:
https://storybook.js.org/docs/react/writing-stories/play-function#page-top

We will not be making any improvement / changes to storyshots.

[jfstephe]

There's a lot of shouts for vitest etc. I'd like to add my voice for other test runners such as for cucumberJs with PlayWright.

IMHO the testing frameworks mentioned above are very unit test-y, and less focused on the user journeys/don't serve as end-user/tester readable documentation on the behaviour of the components. I mention 'tester' there because the tests written in gherkin syntax don't need to be automated, and so these could be being run by a manual tester, which is another added benefit.

Just need something that's easy to integrate with please :-). Happy to put the work in!

[slugmandrew]

For anyone else coming here looking for a way to replicate storyshots using vitest, here's my solution, that builds upon the great solutions offered by @MichaelBitard, @AndrewLeedham, @IsaiahByDayah and @curtvict.

We make heavy use of styled-components, so needed a way to extract the styles into something consistent. This essentially replicates jest-styled-components using vitest.

I'm sure there may be a nicer way to do this, but this what what I hacked together. Hope it saves somebody some time :)

/// <reference types="vite/client"/>
import { Meta, StoryFn } from "@storybook/react"
import { composeStories } from "@storybook/testing-react"
import { render, waitFor } from "@testing-library/react"
import { describe, expect, test } from "vitest"
import { SessionContextProvider } from "hooks/useSessionInfo"
import { server } from "storybook/setup"
import { css, html } from "js-beautify"
import { ServerStyleSheet, StyleSheetManager } from "styled-components"

type StoryFile = {
  default: Meta
  [name: string]: StoryFn | Meta
}

/**
 * Replace the class names in the markup with the shorter placeholder names
 * @param markup the markup to replace the class names in
 * @param stylesMap a map of the original class names to the style rules
 * @returns the replaced markup and a map of the original class names to the shorter placeholder names
 */
const replaceClassNames = (markup: string, stylesMap: Map<string, string>): { replacedMarkup: string; placeholderMap: Record<string, string> } => {
  let index = 0
  const placeholderMap: Record<string, string> = {}

  const replacedMarkup = markup.replace(/class="([\w-\s]+)"/g, (match, classNames: string) => {
    const replacedClassNames = classNames
      .split(/\s+/)
      .map((className) => {
        // Replace the hashed class names
        if (stylesMap.has(className)) {
          if (!placeholderMap[className]) {
            // Use a shorter placeholder, e.g., sc0, sc1, etc.
            placeholderMap[className] = `c${index++}` // hc for hashed class
          }
          return placeholderMap[className]
        }
        // Replace the sc- prefixed class names
        if (className.startsWith("sc-")) {
          if (!placeholderMap[className]) {
            placeholderMap[className] = `sc`
          }
          return placeholderMap[className]
        }
        return className // if className is not in stylesMap, leave it unchanged
      })
      .join(" ")
    return `class="${replacedClassNames}"`
  })
  return { replacedMarkup, placeholderMap }
}

/**
 * Replace the class names in the CSS with the shorter placeholder names
 * @param stylesMap a map of the original class names to the style rules
 * @param classNamesMap a map of the original class names to the shorter placeholder names
 * @returns the replaced CSS text
 */
const replaceClassNamesInCss = (stylesMap: Map<string, string>, classNamesMap: Record<string, string>): string => {
  let replacedCssText = ""

  // Loop through each entry in classNamesMap
  for (const [originalClassName, newClassName] of Object.entries(classNamesMap)) {
    // Retrieve the style rules for the original class name from stylesMap
    const styleRules = stylesMap.get(originalClassName)

    // If there are style rules for this class name, append them to the CSS text
    if (styleRules) {
      replacedCssText += `.${newClassName} { ${styleRules} }\n`
    }
  }

  return replacedCssText
}

/**
 * Perform snapshot tests on all of our stories
 */
describe("Storybook Snapshots", async () => {
  const modules = await Promise.all(Object.values(import.meta.glob<StoryFile>("../**/*.stories.tsx")).map((fn) => fn()))

  describe.each(modules.map((module) => ({ name: module.default.title, module })))("$name", ({ module }) => {
    test.each(Object.values(composeStories(module)).map((Story) => [Story.storyName, Story]))("%s", async (_, Story) => {
      if (!Story) {
        throw new Error("Story is undefined")
      }

      console.log(`Starting snapshot test for ${Story.storyName!}`)

      // grab any story-specific handlers
      const endpoints = Story.parameters?.msw?.handlers ?? []
      for (const group in endpoints) {
        if (endpoints[group]) {
          server.use(...endpoints[group])
        }
      }

      // Sheet used to extract the CSS from the styled components
      const sheet = new ServerStyleSheet()

      // Wait for the story to fully render
      const { container } = await waitFor(() =>
        render(
          <StyleSheetManager sheet={sheet.instance}>
            <SessionContextProvider>
              <Story />
            </SessionContextProvider>
          </StyleSheetManager>
        )
      )
      // Ensure we have something
      expect(container).toBeTruthy()

      // Custom Serializer
      expect.addSnapshotSerializer({
        serialize(val, config, indentation, depth, refs, printer) {
          // extract the styles into a map
          const stylesMap = new Map()
          const styleRegex = /\.([a-zA-Z0-9-]+)\{([^}]+)}/gs
          let match
          while ((match = styleRegex.exec(sheet.getStyleTags())) !== null) {
            stylesMap.set(match[1], match[2].trim())
          }

          // Replace class names in the HTML markup
          const { replacedMarkup, placeholderMap } = replaceClassNames(container.outerHTML, stylesMap)

          // Replace class names in the CSS text
          const replacedCssText = replaceClassNamesInCss(stylesMap, placeholderMap)

          // combine and prettify
          return (
            html(replacedMarkup, {
              indent_size: 2,
              indent_char: " ",
              wrap_line_length: 0,
              unformatted: ["svg", "path"], // don't format the internals of these tags, as it takes up too many lines
            }) +
            "\n\n" +
            css(replacedCssText)
          )
        },
        test(val) {
          // Serializer only runs if this condition is true
          return val instanceof HTMLElement
        },
      })
      // Actually check the snapshot
      await expect(container).toMatchFileSnapshot(`./__snapshots__/${module.default.title}-${Story.storyName}.snap`)
    })
  })
})