Different results between Yarn & npm

[kopax]

Doing yarn and npm install produce different node_modules (it can be tested on package named react-styleguidist)

What is the current behavior?

With yarn, ls node_modules/react-styleguidist/node_modules/.bin:

acorn -> ../../../acorn/bin/acorn
webpack-dev-server -> ../../../webpack-dev-server/bin/webpack-dev-server.js
react-docgen -> ../../../react-docgen/bin/react-docgen.js
findup -> ../../../findup/bin/findup.js
esgenerate -> ../../../escodegen/bin/esgenerate.js
escodegen -> ../../../escodegen/bin/escodegen.js
buble -> ../../../buble/bin/buble

With npm, ls node_modules/react-styleguidist/node_modules/.bin:

acorn -> ../../../acorn/bin/acorn

If the current behavior is a bug, please provide the steps to reproduce.

mkdir -p ~/tmp
cd ~/tmp
git clone git@github.com:bootstrap-styled/rsg-components.git
cd rsg-components
git checkout testing-broken-v9
npm install # or `yarn`
npm run styleguide

What is the expected behavior?

We expect to have the same result between npm and yarn.

Versions

• yarn -v 1.13.0
• npm -v 6.8.0
• Linux dev-11 4.9.0-8-amd64 Add throttling to tarball requests #1 SMP Debian 4.9.130-2 (2018-10-27) x86_64 GNU/Linux

Related issues

• How can I import 'antd' or any other lib ? styleguidist/react-styleguidist#1278
• [Regression v8 to v9] Example can't import itself anymore. moduleAliases/WebpackConfig.resolve.alias is broken => import or require() statements can be added only by editing a Markdown example file styleguidist/react-styleguidist#1310
• Error: import or require() statements can be added only by editing a Markdown styleguidist/react-styleguidist#1312

[arcanis]

We expect to have the same result between npm and yarn.

No we don't? This in itself is not a reason for something to be described a bug

What is the current behavior?

It seems to me Yarn is correct here - you depend on webpack-dev-server so you have access to its binary link (similar to how acorn is exposed, since you depend on it as well). I don't see why that would cause a problem with react-styleguidist.

Npm might link things slightly differently but that hardly matters since the way things are laid out in the node_modules folder (including the .bin folder) is entirely unspecified and implementation-specific. The only guarantee we (and any other package manager, whether it's Yarn, npm, or pnpm) make is that yarn run and require will work as expected when called from your packages.

[kopax]

How can you explain that the code execution result in different dependency being imported? I am trying hard to fix our npm installation and so far yarn is our source of truth. How could it be?

[arcanis]

I can't investigate your code, but it's possible that npm locked your dependencies to a different set from what Yarn did (we have different resolution algorithms with different heuristics, so the end result isn't necessarily the same, while always correct). Maybe one of your dependencies introduced a breaking change without respecting semver, and you got burned by it.

Maybe there's also a bug in npm for all I know - I think they have problems with peer dependencies and hoisting, so it might be it.

[kopax]

I can't investigate your code

Fair enough, what we need is to understand how we can make npm goes the same result without using yarn (or what make yarn produce a different node_modules for react-styleguidist dependency). We have identified that acorn and buble are show differences depending on the package manager used.

npm locked your dependencies to a different set from what Yarn did (we have different resolution algorithms with different heuristics, so the end result isn't necessarily the same, while always correct)

What do you mean by locked your dependencies? We do not use lock file except in application.

Maybe one of your dependencies introduced a breaking change without respecting semver, and you got burned by it.

With npm, doing npm ls acorn show │ └── UNMET PEER DEPENDENCY acorn@6.1.1 next to the dependency.

With yarn it shows: │ ├── acorn@6.1.1

Maybe there's also a bug in npm for all I know - I think they have problems with peer dependencies and hoisting, so it might be it.

We do not use peerDependencies and hosting in that chain.

[arcanis]

npm ls acorn show │ └── UNMET PEER DEPENDENCY acorn@6.1.1 next to the dependency

We do not use peerDependencies and hosting in that chain.

See the problem? 😛

Again, it seems to me that you've hit the problem described here - they even specifically mention acorn as being the affected package. From my understanding npm needs to refactor its internals to solve this. I'm not an npm expert tho, so I can't really offer you advices apart from waiting for the fix or switching to Yarn 🙂

[kopax]

I see, thanks a lot for the explanation. We prefer to support the official npm package manager for the moment.

One question before closing, if yarn > npm, why don't you merge the effort in one project ?

[arcanis]

One question before closing, if yarn > npm, why don't you merge the effort in one project ?

There are a lot of reasons and you probably can understand why I prefer not to answer it myself.

I just want to point out that I find it a bit interesting that we get this question every once in a while, even though noone would ever ask the same thing of say Webpack vs Parcel, React vs Vue, Gatsby vs Next, etc. As open-source maintainers, whether it's the Yarn or npm folks, we all put our efforts into the projects that we like to work with, and that's how it should work.

[kopax]

I like to work with things that work 🤓

That lead me to a last question, which stopped me using yarn even after tones of reading. npm is the official package manager of Node, for this reason, it can't die and using today npm within code/scripts will never be obsolete.

How can yarn succeed on long term support?

[arcanis]

When I started working on Yarn (February 2017) we were at roughly 20% market share based on the number of downloads from the npm registry (ie 20% of all the downloads came from Yarn). Shortly after npm's infrastructure change break our stats (August 2018) we were at 48%. Your guess is as good as mine regarding where we are now 🙂

There's a common misconception that Yarn is the outsider. It's too bad we lost access to our stats to disprove it. We've been pinging npm to fix it every now and then, but it's never been their top priority.

How can yarn succeed on long term support?

Doing what we do best, coming up with new solutions to existing problems. I think we'll eventually have to speak to the Node project to solve this "npm is the only default" thing, it makes little sense when both package managers have roughly the same following.