Builder breaks with package managers that don't hoist (e.g. pnpm)

[Pinpickle]

I recently tried to upgrade from version 0.0.8 to 0.0.10 on my repo, which is using pnpm as its package manager.

pnpm does not hoist dependencies (so, for example, storybook's dependencies are not available in the project's top level node_modules).

When starting a project with storybook-builder-vite and pnpm, we see this error: Error: Failed to resolve force included dependency: airbnb-js-shims, which is one of the forced optimized dependencies here: https://github.com/eirslett/storybook-builder-vite/blob/main/packages/storybook-builder-vite/optimizeDeps.js

I'm confused because:

• this used to work in 0.0.8 and airbnb-js-shims was in that list there
• require.resolve("airbnb-js-shims") shouldn't work from this file in pnpm, based on my understanding of node module resolution and pnpm. It is resolving, so something is wrong with my understanding

Manually installing airbnb-js-shims fixes this error, but it is replaced with the next one down the list.

Reproduction here: https://github.com/Pinpickle/storybook-builder-vite-0.0.10-pnpm-bug (you can run it under pnpm and yarn to see it breaking and working)

[schalk-b]

Should these be added in peer dependencies?

[eirslett]

Could it be because of this change? #42

[IanVS]

Should these be added in peer dependencies?

Peer dependencies are a dumpster fire, it would be great to avoid them.

I think part of the issue is that we want this builder to work with every framework, but we don't want to bundle all of the various framework plugins with it, some of which can break others (the issue #42 was trying to fix). I'm wondering more and more whether we need some kind of architecture which supports plugins to the builder, for the various frameworks, similar to how storybook itself works. It causes an explosion in the number of packages, but it would be much more reliable, I think.

[IanVS]

Why is airbnb-js-shims included in the list of optimize deps? Do we know what is trying to pull it in?

[eirslett]

It's used by @storybook/core-client. (You'll find it in the lockfile.)

[eirslett]

I tagged this with "help wanted"; I don't use pnpm myself, so it would be nice if anybody who uses pnpm in their daily work has time to have a look!

[Pinpickle]

I had a quick look today and it was a bit of a headache. Issue is, it's difficult to use optimizeDeps.include relative to another dep, it's only relative to the root. I managed to do resolution in optimizeDeps relative to the project root (using the package resolve) but then it just filters out heaps of packages and the browser breaks when trying to import.

So this is all still broken. Are there any vite maintainers we can bring on to help?

const resolve = require('resolve');

// We need Vite to precompile these dependencies, because they contain non-ESM code that would break
// if we served it directly to the browser.
module.exports.optimizeDeps = function optimizeDeps(options) {
    return {
    include: [
        'airbnb-js-shims',
        // etc...
    ].filter((m) => {
        try {
            resolve.sync(m, { basedir: options.configDir });
            return true;
        } catch (err) {
            return false;
        }
    }),
}};

I also wrote a plugin to make resolution of @storybook/client-api relative to the framework installed at the root, although this was never broken:

const { resolveRecursively } = require('./resolveRecursively');

const importsToMap = {
    '@storybook/core-client': ['@storybook/core'],
    '@storybook/client-logger': ['@storybook/core', '@storybook/core-client'],
    '@storybook/client-api': ['@storybook/core', '@storybook/core-client'],
}

module.exports.storybookImports = function storybookImports(options) {
    const { framework, frameworkPath, configDir } = options;
    const frameworkImportPath = frameworkPath || `@storybook/${framework}`;
    return {
        name: 'storybook-imports',
        config(id) {
            return {
                resolve: {
                    alias: Object.entries(importsToMap).map(([key, value]) => ({ find: key, replacement: resolveRecursively(configDir, frameworkImportPath, ...value, key, ) }))
                },
                optimizeDeps: {
                    include: Object.keys(importsToMap),
                }
            }
        },
    };
};

Where this is resolveRecursively.js

const readPkgUp = require('read-pkg-up');
const { dirname } = require('path');
const resolve = require('resolve');

/**
 * Continuously resolves packages relative to the previous module, starting from base
 */
const resolveRecursively = (base, ...entries) => {
    const absolutePath = entries.reduce(
        (basedir, entry) =>
            resolve.sync(entry, { basedir, preserveSymlinks: false }),
        base
    );

    // we get the location of the package.json so vite can use it's resolution,
    // as opposed to Node's which will pick up the commonjs bundle (as opposed to esm if it exists)
    const packageJson = readPkgUp.sync({ cwd: absolutePath });

    return dirname(packageJson.path);
};

module.exports = { resolveRecursively };

[eirslett]

Can you try running pnpm install --shamefully-hoist, does that work?

[Pinpickle]

@eirslett that works, yep

This makes sense, given that it's consistent with yarn/npm's hoisting behaviour. It's not really a viable workaround, as removing hoisting is one of the benefits of pnpm.

[Pinpickle]

I found the offending commit: dde6b5f, AKA #28

Basically, as the root used to be inside storybook-builder-vite, the resolutions in optimizeDeps are relative to the root. Now that the root is in the project itself, these resolutions aren't necessarily correct.

So we want some optimized deps/import to be relative to storybook-builder-vite and the rest be relative to the project root.

Not sure how to accomplish that, though 😅

---

In the meantime, I've added this to my viteFinal to basically undo that commit:

const { dirname } = require("path");

// https://github.com/eirslett/storybook-builder-vite/issues/55
config.root = dirname(require.resolve("storybook-builder-vite"));
config.server.fsServe = undefined;

And it works!

[Bjodol]

Just to attach some complexity to this issue, if you are using pnpm with (rushjs)[https://rushjs.io/pages/intro/get_started/] you don't have the option to use --shamefully-hoist

[eirslett]

Does it work with Vite 2.4.1 (or newer)?

[Pinpickle]

Does it work with Vite 2.4.1 (or newer)?

@eirslett I just tried Vite 2.4.1 and I get the same issue. I've updated the reproduction repo with the latest version of Vite and Storybook now.

Just to attach some complexity to this issue, if you are using pnpm with (rushjs)[https://rushjs.io/pages/intro/get_started/] you don't have the option to use --shamefully-hoist

@Bjodol I'm in the same boat with Rush, you probably don't want to enable shamefully-hoist even if you could. I'd suggest the workaround in #55 (comment)

[vikingair]

@Pinpickle Your workaround seems not to work within pnpm workspaces. Transforming this repository into pnpm workspace and adding the above workaround allows to reproduce the following issue. (See #92)

When running cd packages/example-react && pnpm storybook you get the following error:

info => Loading presets
ERR! Error: [vite-plugin-mdx] "react" must be installed

Do you have any idea for an easy workaround?

[Pinpickle]

@fdc-viktor-luft that doesn't sound good! Truth be told, I've stopped using storybook because even with this working, it was still painfully slow.

I'm not sure why this error would come up now, when it didn't come up before. I think the real solution here would be to fix this builder, so the root can correctly be set to the project root (and React be installed).

---

On another note, Vite's support of nested optimizeDeps.include (landed in 2.5.2, vitejs/vite#4634) should make this easier to fix properly.

[vikingair]

@Pinpickle Thanks for your response.

I've stopped using storybook

Do you have a working alternative or did you just do a simple "storybook" yourself instead?

[Pinpickle]

@fdc-viktor-luft I rolled my own, yeah. I only used storybook for development so I just needed an easy way to look at components with specific props.

My code is pretty unique to my setup, but the main magic is using glob imports, i.e. import.meta.glob("./**/*.stories.tsx")

[vikingair]

@Pinpickle Nice 👍 Thanks for the hint 🤩

Sadly we're using Storybook in production, too, and I wanted to migrate all of our Frontend-Setup from CRA to Vite. The storybook is currently the only blocker 😖 I think I'll need to run CRA for Storybook and use Vite everywhere else as long as this will not be resolved.

[zheeeng]

@Pinpickle Thx a lot for your suggestion.

[robcaldecott]

I'm still struggling with this. Even a simple pnpm project (without using workspaces) fails to work without complaining about @mdx-js/react even with these various workarounds via viteFinal. It doesn't even appear to be running anything inside viteFinal before it gives up btw.

Turning on node-linker=hoisted in .npmrc fixes it but it would be good to find another workaround.

Here's another example: https://github.com/robcaldecott/pnpm-storybook-builder-vite

[IanVS]

I'm not a pnpm expert by any means, but I've been working on getting the original reproduction above working, and after a lot of hacking around, I think I've done it. I pushed up the changes to a new repo: https://github.com/IanVS/vite-storybook-pnpm-shame

This was a combination of adding a lot of new optimizeDeps.include that are sub-dependencies (which we can add to our own list in this project so others don't need to), but a few of them required adding the dependencies directly to the repo, which I don't understand and don't like. I'm not sure if this is a vite bug, or weird pnpm thing, or what. But at least it seems possible to get this working without hoisting.

[virtuoushub]

I'm not a pnpm expert by any means, but I've been working on getting the original reproduction above working, and after a lot of hacking around, I think I've done it. I pushed up the changes to a new repo: https://github.com/IanVS/vite-storybook-pnpm-shame

This was a combination of adding a lot of new optimizeDeps.include that are sub-dependencies (which we can add to our own list in this project so others don't need to), but a few of them required adding the dependencies directly to the repo, which I don't understand and don't like. I'm not sure if this is a vite bug, or weird pnpm thing, or what. But at least it seems possible to get this working without hoisting.

I think it is in some part due to Storybook and it's use of peerDependencies. I hear Storybook 7.0 is planning to address this.

[shilman]

I've fixed addon-docs peer dependencies (the major culprit, but possibly not everything?) in 6.5 if anybody wants to take another look at this.

npx sb upgrade --prerelease

[kaminskypavel]

I've fixed addon-docs peer dependencies (the major culprit, but possibly not everything?) in 6.5 if anybody wants to take another look at this.

npx sb upgrade --prerelease

just tested it, and it seems to work.
unfortunately, it creates a new issue related to deduping @storybook/client-api.
the solution here seems to help though

[vikingair]

That might be a suitable work around, but not a real solution 🤔

https://pnpm.io/npmrc#public-hoist-pattern

This setting is useful when dealing with some flawed pluggable tools that don't resolve dependencies properly.

[IanVS]

FWIW, we are working on improved pnpm support in 7.0, and our goal is to not require any hoisting or workarounds.

[rogerleung0411]

FWIW, we are working on improved pnpm support in 7.0, and our goal is to not require any hoisting or workarounds.

can't wait to see this final solution shipped with a release version!

[williamgoulois]

This single line in .npmrc fixed my problems with storybook and npm:

public-hoist-pattern[]=*storybook*

🎉 Thanks a lot @srflp for this solution ! works perfectly without clogging package.json or vite config.optimizeDeps.include with a bunch of storybook packages
Don't forget to run pnpm install to update node_modules structure after your changes

config.root = dirname(require.resolve("storybook-builder-vite"));

This solution to change root appeared to work for me in the first time but was actually breaking @storybook/addon-postcss.
The CSS file that i was importing in preview.tsx with @tailwind directives was picked up but tailwind styles were not applied.
HMR on modification of a component classname did not trigger an update of the CSS file too.

[RIP21]

@IanVS v7 is almost there as it's in Beta already? Is it going to be fixed for the release?

[IanVS]

Yes, the 7.0 beta works correctly with pnpm and yarn pnp. I'd recommend trying it out. You can upgrade with npx sb@next upgrade --prerelease.

[RIP21]

Doesn't much for me. For some reason I have issues with type exports all over the place. Thought that's the reason.

[IanVS]

@RIP21 feel free to hop into discord.gg/storybook and tag me there, or find me on mastodon at @IanVS@fosstodon.org. Or open an issue in the storybook repo and tag me there, and we can dig in.

[kaminskypavel]

@RIP21 It works perfectly fine for me, have a look here https://github.com/kaminskypavel/fullpower-stack/tree/master/packages/ui

@IanVS did a great job !👏

[RIP21]

@kaminskypavel you don't have any type export it seems in it.
I have 1:1 config to yours aside from vite-tsconfig-paths. But even without it and fixed paths, it fails with exactly the same issue.
For reference here is the repro
https://github.com/RIP21/sb7-vite-bug-repo
SyntaxError: The requested module '/src/components/tooltip/index.ts' does not provide an export named 'TooltipProps'
Error is this one.

[omarelb]

Yes, the 7.0 beta works correctly with pnpm and yarn pnp. I'd recommend trying it out. You can upgrade with npx sb@next upgrade --prerelease.

Hi! It's still not working for me after running npx sb@next upgrade --prerelease. I'm getting the following error:

[vite] Internal server error: Failed to resolve import "@storybook/preview-api" from "../../../../../../../../virtual:/@storybook/builder-vite/vite-app.js". Does the file exist?
  Plugin: vite:import-analysis
  File: /virtual:/@storybook/builder-vite/vite-app.js
  1  |  import { composeConfigs, PreviewWeb, ClientApi } from '@storybook/preview-api';
     |                                                         ^
  2  |    import '/virtual:/@storybook/builder-vite/setup-addons.js';
  3  |    import { importFn } from '/virtual:/@storybook/builder-vite/storybook-stories.js';

I'm running pnpm in a monorepo and trying to get storybook running for one of the subpackages. In a pnpm monorepo the actual node_modules assets are located in the root folder. Maybe that has something to do with the issues? Here are the dependencies:

    "@storybook/addon-actions": "^7.0.0-beta.30",
    "@storybook/addon-essentials": "^7.0.0-beta.30",
    "@storybook/addon-links": "^7.0.0-beta.30",
    "@storybook/web-components": "^7.0.0-beta.30",
    "@storybook/web-components-vite": "^7.0.0-beta.30",
    "storybook": "^7.0.0-beta.30",
    "@babel/core": "^7.20.12",
    "babel-loader": "^8.3.0"

Using public-hoist-pattern[]=*storybook* as suggested by others does work.

[Dschungelabenteuer]

@omarelb I think this may be a regression from v7.0.0-beta.30. I'm having this issue as well, but it was working perfectly before that prerelease. See this and original issue. You could downgrade to v7.0.0-beta.29, but as mentioned in the original issue, there are some issues with HMR on Vite-based Storybooks (I'd personally downgrade to v7.0.0-beta.25)

[vikingair]

Thanks for the info @Dschungelabenteuer and FYI @omarelb, this is also working and a little less generic:

public-hoist-pattern[]=@storybook/*

[abencun-symphony]

This seems to have been resolved with Storybook 7 release for us.

[acusti]

i have tried to get the vite builder working with storybook v7.0.2, v7.0.6, v7.0.7, and v7.1.0-alpha.8, and i still cannot get it to work in a monorepo using yarn v3.3.0 and pnp. first i needed to added packageExtensions to my .yarnrc.yml to resolve Error: @storybook/core-common tried to access @storybook/react-vite, but it isn't declared in its dependencies; this makes the require call ambiguous and unsound. and Error: @storybook/react-dom-shim tried to access react-dom (a peer dependency) but it isn't provided by its ancestors; this makes the require call ambiguous and unsound.:

packageExtensions:
    '@storybook/core-common@*':
        dependencies:
            '@storybook/react-vite': '^7.1.0-alpha.8'
            'react-dom': '^18.2.0'

after adding that, running yarn, then running storybook dev, i am able to get the info => Starting manager... log with the info box saying that Storybook started and showing URLs to access it before i see the following error:

[vite] Internal server error: Failed to resolve import "@storybook/react/preview" from "../../../../../../virtual:/@storybook/builder-vite/vite-app.js". Does the file exist?
  Plugin: vite:import-analysis
  File: /virtual:/@storybook/builder-vite/vite-app.js:7:47
  5  |
  6  |    const getProjectAnnotations = async () => {
  7  |      const configs = await Promise.all([import('@storybook/react/preview'),
     |                                                ^
  8  |  import('@storybook/addon-essentials/docs/preview'),
  9  |  import('@storybook/addon-essentials/actions/preview'),

i also tried adding a viteFinal item to my .storybook/main.ts config (and a direct devDependency on @storybook/builder-vite v0.4.2):

import { dirname } from 'path';
import { mergeConfig } from 'vite';

import type { StorybookConfig } from '@storybook/react-vite';

const config: StorybookConfig = {
    framework: {
        name: '@storybook/react-vite',
        options: {},
    },
    // ...
    async viteFinal(config) {
        // Merge custom configuration into the default config
        return mergeConfig(config, {
            root: dirname(require.resolve('@storybook/builder-vite')),
            server: { fsServe: undefined },
        });
    },
};

export default config;

when i run storybook dev with that config, i see the info => Starting manager... log, then (before the Storybook started info box):

Failed to resolve dependency: @mdx-js/react, present in 'optimizeDeps.include'
Failed to resolve dependency: @storybook/addon-docs, present in 'optimizeDeps.include'
Failed to resolve dependency: @storybook/addon-essentials/docs/mdx-react-shim, present in 'optimizeDeps.include'
Failed to resolve dependency: react-dom, present in 'optimizeDeps.include'
Failed to resolve dependency: react-dom/client, present in 'optimizeDeps.include'
Failed to resolve dependency: react, present in 'optimizeDeps.include'
Failed to resolve dependency: react/jsx-runtime, present in 'optimizeDeps.include'
Failed to resolve dependency: react/jsx-runtime, present in 'optimizeDeps.include' (x2)
Failed to resolve dependency: react/jsx-dev-runtime, present in 'optimizeDeps.include'
Failed to resolve dependency: react, present in 'optimizeDeps.include'

so i’m guessing the viteFinal function that i tried to use isn’t working and is breaking things more.

[IanVS]

@acusti would you be willing to create a minimal reproduction and open a new issue so we can dig into it more there?

[acusti]

@IanVS ok, i can do that when i get a free moment

[acusti]

@IanVS sorry i never got around to working on a minimal repro, but i did return to trying to upgrade my yarn v3.x monorepo to storybook v7.x and the vite builder, and i wound up getting everything to work. it turned out that the errors i was seeing were mostly just symptoms of missing dependencies. in the end, i just needed to thoroughly configure the packageExtensions in my .yarnrc.yml file and use the getAbsolutePath helper in my .storybook/main.ts file. here’s all of the necessary pieces (i’m using v7.5.0-alpha.5 and react-docgen v6.x):

storybook workspace’s package.json:

    "devDependencies": {
        "@babel/core": "^7.20.5",
        "@storybook/addon-docs": "7.5.0-alpha.5",
        "@storybook/addon-essentials": "7.5.0-alpha.5",
        "@storybook/addon-interactions": "7.5.0-alpha.5",
        "@storybook/addon-links": "7.5.0-alpha.5",
        "@storybook/addon-onboarding": "^1.0.8",
        "@storybook/blocks": "7.5.0-alpha.5",
        "@storybook/manager-api": "7.5.0-alpha.5",
        "@storybook/react": "7.5.0-alpha.5",
        "@storybook/react-vite": "7.5.0-alpha.5",
        "@storybook/testing-library": "^0.2.1",
        "@types/react": "^18.0.25",
        "core-js": "^3.26.1",
        "react": "^18.2.0",
        "react-dom": "^18.2.0",
        "storybook": "7.5.0-alpha.5",
        "typescript": "^5.1.6",
        "vite": "^4.4.9"
    },

root .yarnrc.yml:

plugins:
  - path: .yarn/plugins/@yarnpkg/plugin-workspace-tools.cjs
    spec: "@yarnpkg/plugin-workspace-tools"

yarnPath: .yarn/releases/yarn-3.6.3.cjs

packageExtensions:
    'eslint-config-react-app@*':
        dependencies:
            '@babel/plugin-syntax-flow': '^7.18.6'
            '@babel/plugin-transform-react-jsx': '^7.18.6'
    '@storybook/builder-vite@*':
        dependencies:
            '@storybook/addon-docs': '7.5.0-alpha.5'
            'react': '^18.2.0'
            'react-dom': '^18.2.0'
    '@storybook/docs-tools@*':
        dependencies:
            'react': '^18.2.0'
            'react-dom': '^18.2.0'
    '@storybook/core-common@*':
        dependencies:
            '@storybook/react-vite': '7.5.0-alpha.5'
            'react': '^18.2.0'
            'react-dom': '^18.2.0'
            'vite': '^4.4.9'

.storybook/main.ts:

import { dirname, join } from 'path';

import type { StorybookConfig } from '@storybook/react-vite';

/**
 * This function is used to resolve the absolute path of a package.
 * It is needed in projects that use Yarn PnP or are set up within a monorepo.
 */
function getAbsolutePath(value: string): any {
    return dirname(require.resolve(join(value, 'package.json')));
}

const config: StorybookConfig = {
    addons: [
        getAbsolutePath('@storybook/addon-links'),
        getAbsolutePath('@storybook/addon-essentials'),
        getAbsolutePath('@storybook/addon-onboarding'),
        getAbsolutePath('@storybook/addon-interactions'),
    ],
    docs: {
        autodocs: 'tag',
    },
    framework: {
        name: getAbsolutePath('@storybook/react-vite'),
        options: {},
    },
    stories: [
        '../stories/Introduction.stories.mdx',
        '../stories/**/*.stories.mdx',
        '../stories/**/*.stories.@(js|jsx|ts|tsx)',
    ],
    typescript: {
        reactDocgen: 'react-docgen',
    },
};

export default config;

[Pinpickle]

Two years later I've finally tried this again and it works out-of-the-box with my Vite and pnpm setup. Storybook team: thanks for the hard work improving compatibility!