Yarn 2: Support Yarn v2 "Berry" #9527

crubier · 2020-01-18T13:11:09Z

Describe the bug

If you use Yarn v2, then you can't add Storybook to a simple Create React App, it crashes.

And the output here:

https://github.com/crubier/berry/commit/78acb633a0ad411ed173dae0a0037ab82fe346ad/checks?check_suite_id=407781418

To Reproduce

The following process is used in the workflow to reproduce this bug:

Install yarn v2: https://next.yarnpkg.com/getting-started/install
yarn dlx create-react-app my-cra && cd my-cra to init a crat react app
yarn dlx -p @storybook/cli sb init to init storybook, but it fails here
Then I would do yarn storybook and that would be it.... ideally..

Expected behavior

Success

Screenshots

Error: Cannot find module 'commander'
Require stack:
- /Users/vincent/.yarn/berry/cache/@storybook-cli-npm-5.3.6-c196adf18e-1.zip/node_modules/@storybook/cli/bin/generate.js
- /Users/vincent/.yarn/berry/cache/@storybook-cli-npm-5.3.6-c196adf18e-1.zip/node_modules/@storybook/cli/bin/index.js
Require stack:
- /Users/vincent/.yarn/berry/cache/@storybook-cli-npm-5.3.6-c196adf18e-1.zip/node_modules/@storybook/cli/bin/generate.js
- /Users/vincent/.yarn/berry/cache/@storybook-cli-npm-5.3.6-c196adf18e-1.zip/node_modules/@storybook/cli/bin/index.js
    at Proxy.module_1.Module._resolveFilename (/private/var/folders/2g/yhlpschd00n71jykx6bgbp1w0000gn/T/tmp-69185QMHt9oBf5169/dlx-69185/.pnp.js:18118:54)
    at Proxy.module_1.Module._load (/private/var/folders/2g/yhlpschd00n71jykx6bgbp1w0000gn/T/tmp-69185QMHt9oBf5169/dlx-69185/.pnp.js:18001:40)
    at Object.<anonymous> (/Users/vincent/.yarn/berry/cache/@storybook-cli-npm-5.3.6-c196adf18e-1.zip/node_modules/@storybook/cli/bin/generate.js:1)
    at Generator.next (<anonymous>)

System:

➜ npx -p @storybook/cli@next sb info

Environment Info:

  System:
    OS: macOS 10.15.2
    CPU: (12) x64 Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
  Binaries:
    Node: 10.15.3 - ~/.nvm/versions/node/v10.15.3/bin/node
    Yarn: 2.0.0-rc.23 - /usr/local/bin/yarn
    npm: 6.4.1 - ~/.nvm/versions/node/v10.15.3/bin/npm
  Browsers:
    Chrome: 79.0.3945.117
    Firefox: 71.0
    Safari: 13.0.4

Additional context
Trying to make Storybook work with yarn v2

The text was updated successfully, but these errors were encountered:

gaetanmaisse · 2020-01-25T11:56:58Z

I will take a look at this one and try to make things work :)
@ndelangen @shilman do you know if another member already planned to work on Yarn v2 compatibility?

ndelangen · 2020-01-25T13:22:24Z

I was trigged to do it too, but I'm quite swamped; it'd be awesome if you could take this on!

shilman · 2020-01-25T14:02:10Z

@igor-dv you were looking at yarn-pnp at one point, weren't you?

gaetanmaisse · 2020-01-25T17:44:54Z

Current situation about the compatibility with Yarn 2 (a.k.a Berry)

@crubier created a repo explaining how to make SB works with Yarn Berry: https://github.com/crubier/yarnv2-storybook, the workaround is to:

Unplug @storybook/core, for details see https://next.yarnpkg.com/cli/unplug -> yarn unplug @storybook/core
Create a node_modules folder to be able to create SB Babel cache folder -> mkdir node_modules
Create a preset to define a custom webpack config that uses Plug'n'Play resolver for Webpack.

Based on this repo and comments, multiple points need to be addressed to make SB works with Yarn Berry out of the box. There are my thoughts about that:

It looks like @storybook/core needs to be unplugged because it writes something inside itself. This is working with classical/legacy yarn because
node_modules/@storybook/core is a folder with writable rights. In Yarn Berry, packages are read-only so no more writings allowed:

Packages are now kept within their cache archives. For safety and to prevent cache corruptions, those archives are mounted as read-only drives and cannot be modified under normal circumstances

So we need to find which part of @storybook/core is writing something and update this behavior. Sadly for us @storybook/core is (one of) the last packages still in JS.
As there is no node_modules folder anymore, we need to find another place to put SB Babel cache. The file managing this is https://github.com/storybookjs/storybook/blob/next/lib/core/src/server/common/babel-cache-preset.js and it should be pretty easy to fallback/move to another cache folder.
However, I didn't find any wide used convention for where should be the cache folder for now (some package like Babel have a fallback to os.tmp(), webpack has changed it here...), so I think we need to discuss that.
Update the base webpack config to include Plug'n'Play resolver if the user is using Yarn Berry (in a more global point of view is using Plug'n'Play i.e. there is a pnp.js file in its project)

OR

Create a preset to handle this out of @storybook/core as it is affecting only a part of SB users. Something like what @igor-dv and @crubier started there: Pnp continue presets#18 (which is only working for TS users).

My personal opinion is that we should consider adding the Plug'n'Play resolver in SB core as Yarn Berry is now the "stable" version (yarn as we are used to is now legacy). More and more SB users will be using Yarn Berry and SB should work with it without any additional setup.
According to Yarn 2 support nodejs/docker-node#1180, Yarn Berry will/should be the default version ship with Node 14 Docker Image.

Here is a link of Introducing Yarn 2 article written by its main maintainer.

Any feedback on all this is welcome! @crubier @igor-dv @ndelangen @shilman and anyone else :)

shilman · 2020-01-26T14:21:36Z

@gaetanmaisse Thanks for this detailed explanation and for taking this on. You're awesome!!

One extra bit of information: @tmeasday was recently talking to me about wanting to rewrite story_store, client_api, and some other key files in lib/core. I wonder if it might be the right time to port it to typescript and fix whatever is causing it to write itself. WDYT?

gaetanmaisse · 2020-01-26T16:19:01Z

Are you talking about the usages of story_store and client_api in core? because they are already written in TS in lib/client-api.

I just investigated a bit more and the answer to "What core does write inside itself?" was... 🥁 ready? 🥁 ... the bundle generated by webpack 😄 ! See there, with no custom outputDir we are using ../public folder and so it ends in node_modules/@storybook/core/dist/public. So the solution to this will be to use a folder outside node_modules. One more folder to discuss (+ cache folder).

About "3. Update the base webpack config to include Plug'n'Play", it looks like Plug'n'Play will be supported out of the box by Webpack 5. But it is still in beta for now!
-- Note for the future, there is a constant to check if the user is running Pnp, can be useful if we have to add the Plug'n'Play resolver in Webpack SB base config.

In the next days, I will try to open a PR with as few changes as possible to make this works, this way we will have some code to discuss. 😉

shilman · 2020-01-26T16:29:13Z

LOL I guess rewriting that wouldn't help things then! 🙈 But I think the rewrite was meant to extend to at least some of lib/core, including start.js, which is one of the more complex files in our codebase.

Sounds like you don't need any of that to move forward. Thanks for leading the charge here!

ndelangen · 2020-01-27T13:22:41Z

We can move the cache dir to <cwd>/.cache/storybook (creating the folder if not present) for both the babel cache and the output dir for webpack.

That would solve these issues?

arcanis · 2020-01-28T09:53:38Z

3. Update the base webpack config to include Plug'n'Play resolver if the user is using Yarn Berry (in a more global point of view is using Plug'n'Play i.e. there is a pnp.js file in its project)

Note that you don't need to check whether the user is running under Plug'n'Play, as the plugin already does and is a noop for other environments 🙂This is for example what Gatsby and Next do.

gaetanmaisse · 2020-02-07T13:04:42Z

With Storybook 6.0.0-alpha.6+ SB & Yarn 2 are working together for simple config, as in repro repo provided by @crubier 🎉

I still need to work on:

Making yarn dlx -p @storybook/cli sb init works, to have out of the box support:
- PR: CLI: transpile @storybook/cli to CJS #9807
- PR: CLI: Add Yarn 2 compatibility #9866
Updating @storybook/preset-create-react-app. it's currently resolving react-script in node_modules/.bin/ directory, so I need to add one more check for SB PnP user and access webpack config file directly in the .zip file with PnP API.
- PR: Make preset-create-react-app compatible with Yarn PnP/Yarn 2 presets#104
Checking that all addons are working with the latest alpha of SB 6:
- PR : Yarn 2: Fix dependencies issues for compatibility #10613

    - ✅ A11y
    - ✅ Actions
    - ✅ Backgrounds
    - ✅ CSS Resources
    - ✅ Design Assets
    - 🚧 Docs
    - 🚧 Essential
    - ✅ Events
    - ✅ Google Analytics
    - ✅ GraphQL
    - ✅ Jest
    - ✅ Knobs
    - ✅ Links
    - ✅ Query Params
    - 🤷‍♂ Storyshots
    - ✅ Storysource
    - ✅ Toolbars
    - ✅ Viewport

Finding a way of adding examples with Yarn 2 inside SB monorepo. Maybe creating some GitHub actions as it is done on Yarn 2 repo can be a good idea because it really mimics the users' behaviors: https://github.com/yarnpkg/berry/blob/master/.github/workflows/e2e-jest-workflow.yml
- PR: CLI: Add Yarn 2 compatibility #9866
- Yarn 2 will be fully integrated into the new E2E tests: Tech/ci examples overhaul #10447
Handle .storybook/preview.js file properly . See Yarn 2: SB not working when using a .storybook/preview.js file #10094
- PR: Yarn 2: rework imports in webpack preview virtual module to fix compatibility #10305

@ndelangen For the cache folder, I put it in node_modules/.cache/@storybook for now. An enhancement issue has been created by @arcanis in Yarn Org yarnpkg/berry#918 to discuss the usefulness, pros, and cons of creating and maintaining a cache pkg in the Yarn org. If it ends on the creation of a such package we will surely use it instead of our custom resolve-path-in-sb-cache.js.

shilman · 2020-02-07T14:26:56Z

Great job @gaetanmaisse !!! 💯

missing1984 · 2020-04-12T17:43:49Z

Love the efforts here! I ran into #7445 in yarn pnp. Not sure if someone caught this before.

Looks like corejs-upgrade-webpack-plugin is not compatible with yarn pnp. Maybe we need to disable it in pnp mode?

kubijo · 2020-04-14T17:25:55Z

With Storybook 6.0.0-alpha.6+ SB & Yarn 2 are working together for simple config, as in repro repo provided by @crubier

Hi.

Thanks for your great efforts!
I've managed to get our storybook instance to run with alpha.33 after configs migration & I've also had to unplug @storybook/client-api since otherwise it wouldn't be resolved from somewhere deep.

Here's the error message:

ERROR in ./conf/storybook/preview.js-generated-config-entry.js
Module not found: Error: Can't resolve '…/.yarn/$virtual/@storybook-client-api-virtual-4ff893b6d5/0/cache/@storybook-client-api-npm-6.0.0-alpha.33-bfa9fe6573-2.zip/node_modules/@storybook/client-api/dist/index.js' in '…/conf/storybook'
 @ ./conf/storybook/preview.js-generated-config-entry.js 52:15-225
 @ multi ./.yarn/$$virtual/@storybook-core-virtual-2edb463070/0/cache/@storybook-core-npm-6.0.0-alpha.33-85dfb5f3b0-2.zip/node_modules/@storybook/core/dist/server/common/polyfills.js ./.yarn/$$virtual/@storybook-core-virtual-2edb463070/0/cache/@storybook-core-npm-6.0.0-alpha.33-85dfb5f3b0-2.zip/node_modules/@storybook/core/dist/server/preview/globals.js ./conf/storybook/storybook-init-framework-entry.js ./conf/storybook/preview.js-generated-config-entry.js ./conf/storybook/generated-stories-entry.js ./.yarn/cache/webpack-hot-middleware-npm-2.25.0-b59f3763d7-2.zip/node_modules/webpack-hot-middleware/client.js?reload=true&quiet=false

Also I get this one (don't mind that): Could not open http://localhost:6006/ inside a browser. If you're running this command inside a docker container or on a CI, you need to pass the '--ci' flag to prevent opening a browser by default.

gaetanmaisse · 2020-04-14T20:11:17Z

@kubijo thanks for your feedback! The issue you're facing will be fixed with #10342, I would like to merge it ASAP to have the fix included in the next alpha (in the next few days 🤞).

gaetanmaisse · 2020-04-15T17:01:25Z

@kubijo alpha.35 is out and should fix many of the Module not found errors. 🚀
Can you give it a try and check if you still face issues with Yarn 2?

kubijo · 2020-04-15T17:15:00Z

@gaetanmaisse alpha.34 indeed seems to work 👍

DimitrK · 2020-04-15T18:37:43Z

@gaetanmaisse great news, alpha.35 works for me. Still, alpha.34 @kubijo is mentioning unfortunately didn't work for me.

kubijo · 2020-04-15T18:40:39Z

I might have been a little to quick as 35 didn't resolve ... So I thought that it's a typo, but maybe I just raced CDNs

edit: nevermind, I think that just IDE didn't see it because it is running from a.35

shilman · 2020-05-02T14:21:41Z

Good golly!! I just released https://github.com/storybookjs/storybook/releases/tag/v6.0.0-beta.1 containing PR #10613 that references this issue. Upgrade today to try it out!

You can find this prerelease on the @next NPM tag.

Closing this issue. Please re-open if you think there's still more to do.

jon301 · 2020-11-01T07:08:01Z

Hi Storybook Team !
I'm currently trying the latest version of Storybook (6.0.28) with Yarn Berry, and I'm encountering the following issues.

Here are the steps I followed:
yarn dlx -p @storybook/cli sb init
yarn storybook

core-js is not found

ERROR in ./.storybook/preview.js-generated-config-entry.js
Module not found: Error: Your application tried to access core-js, but it isn't declared in your dependencies; this makes the require call ambiguous and unsound.

Required package: core-js (via "core-js/modules/web.dom-collections.for-each")
Required by: /Users/jonjon/work/onitech-mono/packages/libs/my-lib1/.storybook/preview.js-generated-config-entry.js

...

WARN Broken build, fix the error above.
WARN You may need to refresh the browser.

Essential Addon not working

After adding core-js manually to dev dependencies, Storybook starts correctly but the Essentials Addon does not work (there's no Addon panel in the UI, even if Show addons is checked)

There's also an error in the console:

info => Loading presets
WARN   Failed to load preset: {"name":"@storybook/addon-essentials","type":"presets"} on level 1
ERR! Error: @storybook/core tried to access @storybook/addon-essentials, but it isn't declared in its dependencies; this makes the require call ambiguous and unsound.
ERR!
ERR! Required package: @storybook/addon-essentials (via "@storybook/addon-essentials")
ERR! Required by: @storybook/core@virtual:3ced7c9a1cb7b9c553fcb23a084cf86bc2b7b3c18fb1453f5d7e4e78ef02cf878b49023eb9dceaf9c0ff10242ebfc55e1b462bbf81d54d60fb71b1a18460aa06#npm:6.0.28 (via /Users/jonjon/work/onitech-mono/.yarn/$$virtual/@storybook-core-virtual-5e0cc5b3d0/0/cache/@storybook-core-npm-6.0.28-47dac5a288-ee78dbe2de.zip/node_modules/@storybook/core/dist/server/)

Not sure what to do here as@storybook/addon-essentials is installed correctly :(

Warning in `Introduction.stories.mdx`

WARNING in ./src/stories/Introduction.stories.mdx 11:0
Module parse failed: Unexpected token (11:0)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
| import StackAlt from './assets/stackalt.svg';
|
> <Meta title="Example/Introduction" />
|
| <style>{`
 @ \.)(?=.)[^/]*?\.stories\.mdx)$ (./src sync ^\.(?:(?:^|\/|(?:(?:(?!(?:^|\/)\.).)*?)\/)(?!\.)(?=.)[^/]*?\.stories\.mdx)$) ./stories/Introduction.stories.mdx
 @ ./.storybook/generated-stories-entry.js

shilman · 2020-11-01T07:09:00Z

cc @gaetanmaisse @merceyz

jon301 · 2020-11-01T09:04:04Z

For info, I digged a bit and created a repro case with some useful information.
Thank you for your help.

https://github.com/jon301/storybook-yarn-berry-monorepo

merceyz · 2020-11-01T11:16:51Z

Fixed in #11753 which was released in v6.1.0-alpha.8

jon301 · 2020-11-01T13:33:24Z

Thanks @merceyz for the quick answer !

Updating to 6.1.0-alpha.34 has made errors disappear for the basic Storybook configuration.
However when adding a custom webpack config with SASS support, new errors appear:

Module not found: Error: @storybook/core tried to access sass-loader, but it isn't declared in its dependencies; this makes the require call ambiguous and unsound.

Required package: sass-loader (via "sass-loader")
Required by: @storybook/core@npm:6.1.0-alpha.34 (via /Users/jonjon/tmp/storybook-yarn-berry-monorepo/.yarn/cache/@storybook-core-npm-6.1.0-alpha.34-a93cd16488-2d6e16aa2a.zip/node_modules/@storybook/core/)

I've updated the repro case : https://github.com/jon301/storybook-yarn-berry-monorepo

merceyz · 2020-11-01T14:11:15Z

You need to require.resolve those loaders, otherwise you're telling storybook to resolve them for you, which it can't since it doesn't declare them. Or you can use the scss preset that does this for you https://github.com/storybookjs/presets/tree/master/packages/preset-scss

crubier mentioned this issue Jan 18, 2020

[Feature] Add Storybook to the "Supported Tooling Table" yarnpkg/berry#592

Closed

2 tasks

ndelangen added compatibility with other tools cra Prioritize create-react-app compatibility yarn / npm Yarn / npm acting weird dependencies:update labels Jan 25, 2020

gaetanmaisse self-assigned this Jan 25, 2020

gaetanmaisse mentioned this issue Jan 28, 2020

Core: Add Yarn 2 compatibility #9667

Merged

hydrosquall mentioned this issue Feb 3, 2020

Yarn PnP compatibility react-cosmos/react-cosmos#946

Closed

shilman added this to the 6.0.0 milestone Feb 5, 2020

This was referenced Feb 10, 2020

CLI: transpile @storybook/cli to CJS #9807

Merged

CLI: Add Yarn 2 compatibility #9866

Merged

This was referenced Feb 24, 2020

Make preset-create-react-app compatible with Yarn PnP/Yarn 2 storybookjs/presets#104

Merged

Pnp continue storybookjs/presets#18

Closed

This was referenced Mar 9, 2020

Misc: Add some missing dependencies #10012

Merged

Yarn 2: preset-create-react-app doesn't work with Yarn v2 monorepo + hoisting/PnP #10078

Closed

Yarn 2: SB not working when using a .storybook/preview.js file #10094

Closed

gaetanmaisse changed the title ~~Support Yarn v2 "Berry"~~ Yarn 2: Support Yarn v2 "Berry" Mar 10, 2020

shilman mentioned this issue Mar 23, 2020

Storybook 6.0 Release 🏆 #9311

Closed

gaetanmaisse mentioned this issue Apr 3, 2020

Yarn 2: rework imports in webpack preview virtual module to fix compatibility #10305

Merged

gaetanmaisse mentioned this issue May 1, 2020

Yarn 2: Fix dependencies issues for compatibility #10613

Merged

shilman closed this as completed May 2, 2020

gaetanmaisse mentioned this issue Jul 12, 2020

Yarn 2 support #11511

Closed

pringshia mentioned this issue Sep 20, 2021

JS package reorganization facebookresearch/Mephisto#565

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Yarn 2: Support Yarn v2 "Berry" #9527

Yarn 2: Support Yarn v2 "Berry" #9527

crubier commented Jan 18, 2020

gaetanmaisse commented Jan 25, 2020

ndelangen commented Jan 25, 2020

shilman commented Jan 25, 2020

gaetanmaisse commented Jan 25, 2020 •

edited

Loading

shilman commented Jan 26, 2020 •

edited

Loading

gaetanmaisse commented Jan 26, 2020

shilman commented Jan 26, 2020

ndelangen commented Jan 27, 2020 •

edited

Loading

arcanis commented Jan 28, 2020

gaetanmaisse commented Feb 7, 2020 •

edited

Loading

shilman commented Feb 7, 2020

missing1984 commented Apr 12, 2020

kubijo commented Apr 14, 2020 •

edited

Loading

gaetanmaisse commented Apr 14, 2020

gaetanmaisse commented Apr 15, 2020

kubijo commented Apr 15, 2020

DimitrK commented Apr 15, 2020 •

edited

Loading

kubijo commented Apr 15, 2020 •

edited

Loading

shilman commented May 2, 2020

jon301 commented Nov 1, 2020

shilman commented Nov 1, 2020

jon301 commented Nov 1, 2020

merceyz commented Nov 1, 2020

jon301 commented Nov 1, 2020 •

edited

Loading

merceyz commented Nov 1, 2020 •

edited

Loading

Yarn 2: Support Yarn v2 "Berry" #9527

Yarn 2: Support Yarn v2 "Berry" #9527

Comments

crubier commented Jan 18, 2020

gaetanmaisse commented Jan 25, 2020

ndelangen commented Jan 25, 2020

shilman commented Jan 25, 2020

gaetanmaisse commented Jan 25, 2020 • edited Loading

Current situation about the compatibility with Yarn 2 (a.k.a Berry)

shilman commented Jan 26, 2020 • edited Loading

gaetanmaisse commented Jan 26, 2020

shilman commented Jan 26, 2020

ndelangen commented Jan 27, 2020 • edited Loading

arcanis commented Jan 28, 2020

gaetanmaisse commented Feb 7, 2020 • edited Loading

shilman commented Feb 7, 2020

missing1984 commented Apr 12, 2020

kubijo commented Apr 14, 2020 • edited Loading

gaetanmaisse commented Apr 14, 2020

gaetanmaisse commented Apr 15, 2020

kubijo commented Apr 15, 2020

DimitrK commented Apr 15, 2020 • edited Loading

kubijo commented Apr 15, 2020 • edited Loading

shilman commented May 2, 2020

jon301 commented Nov 1, 2020

core-js is not found

Essential Addon not working

Warning in Introduction.stories.mdx

shilman commented Nov 1, 2020

jon301 commented Nov 1, 2020

merceyz commented Nov 1, 2020

jon301 commented Nov 1, 2020 • edited Loading

merceyz commented Nov 1, 2020 • edited Loading

gaetanmaisse commented Jan 25, 2020 •

edited

Loading

shilman commented Jan 26, 2020 •

edited

Loading

ndelangen commented Jan 27, 2020 •

edited

Loading

gaetanmaisse commented Feb 7, 2020 •

edited

Loading

kubijo commented Apr 14, 2020 •

edited

Loading

DimitrK commented Apr 15, 2020 •

edited

Loading

kubijo commented Apr 15, 2020 •

edited

Loading

Warning in `Introduction.stories.mdx`

jon301 commented Nov 1, 2020 •

edited

Loading

merceyz commented Nov 1, 2020 •

edited

Loading