workspaces "bin" symlinks not created for sub-dependencies

[chrisblossom]

Do you want to request a feature or report a bug?
bug

What is the current behavior?
Yarn Workspaces do not create node_modules/.bin symlinks for nested dependencies that have a bin specified in their package.json.

Original Yarn issue: #2874

If the current behavior is a bug, please provide the steps to reproduce.

Create yarn workspace
Create two workspace packages, one and two
Add two as a dependency to one
Add eslint as a dependency to two

Example: chrisblossom/yarn-issue-4964

What is the expected behavior?
one/node_modules/.bin/eslint symlink exists

Please mention your node.js, yarn and operating system version.
node 8.9.1
Yarn 1.3.2
OSX 10.13.1

[rally25rs]

Related: #4543
Caused by #4730

[klaasman]

Is there a temporary workaround for this issue?

[tothandras]

@klaasman You can create a symlink to the root node_modules/.bin as a workaround:

$ cd packages/foo
$ mkdir node_modules
$ ln -s ../../../node_modules/.bin ./node_modules

I hope a fix will come soon though... Whenever I switch to yarn I find some blocking bug that forces me to go back to npm.

[alexeyraspopov]

Another workaround: go back to the root, rm -rf node_modules and do yarn install. This installs everything back and links necessary .bin files to workspaces.

[alexeyraspopov]

Based on this workaround, it seems that the issue is caused by how PackageLinker works during yarn add in a workspace vs root yarn install. Maybe related to hoisting, @rally25rs?

[alexeyraspopov]

And worth mentioning, linking does not happen by just yarn install if node_modules are present.

[GAumala]

Hello, @alexeyraspopov thanks for posting that workaround, it does solve this issue at least partially for me.

I have 4 packages in my workspace, I deleted every single node_modules directory and ran yarn --frozen-lockfile. .bin files are installed for all packages except one. This might be due to the fact that this particular package has only one dependency with a binary, and the workspace has different versions of this package.

Can you please give me any hints on how to debug this by myself? Maybe I should start with src/package-linker.js?

[josh08h]

Any movement on this?

Only direct dependencies have their bin symlinked from local node_modules/.bin to the root node_modules/.bin.

> yarn -v
1.7.0

I would expect yarn to symlink all transitive dependency bins and direct dependency bins to the root level node_modules/.bin dir.

[NE-SmallTown]

Any update on this?

[btakita]

I had to create a postinstall script that traverses all of the packages*/bin/* files in the repo & symlinks the files into the node_modules/.bin directory.

Of course, I'd love for this bug to be fixed, but here's the hack in the meantime...

#/bin/sh
DIR="$(pwd)"
pushd node_modules/.bin
DIR__BIN="$(pwd)"
# In the case of a git submodule which is also a monorepo
for f in $(find $DIR/packages/*/packages/*/bin/*); do
	ln -sf "$(realpath --relative-to="$(pwd)" "$f")"
done
for f in $(find $DIR/packages/*/bin/*); do
	ln -sf "$(realpath --relative-to="$(pwd)" "$f")"
done
popd

[FezVrasta]

I'm trying to understand how is Facebook able to use Yarn in production with these bugs... Is anyone from the team aware of this problem? #4730 doesn't seem to have fixed it

[mattfysh]

This seems like the correct behaviour to me. packages/one has not declared a dependency on eslint and therefore should not be given access to the eslint CLI.

If you want to use eslint in the packages/one folder a dependency must be declared. Otherwise you could create a brittle connection between one and the private, internal implementation of two

Can you point to an example of how this works in a non-workspaces setup?

[dizel3d]

nohoist in a root package.json helps me:

"workspaces": {
  "nohoist": [
    "**/eslint"
  ]
},

The same for a child package.json:

"workspaces": {
  "nohoist": [
    "eslint"
  ]
},

[kirill-konshin]

If you use Lerna simple trick is to add following to your package.json:

{
  "postinstall": "lerna link"
}

P.S. Side note, keep in mind that after this your monorepo packages will be linked twice (assume packageB requires packageA:

• node_modules
  • packageA — first link by Yarn
• packageA — source
• packageB
  • node_modules
    • packageA — second link by Lerna

[trusktr]

@kirill-konshin Interesting. I wonder which sort of symlinking is better? Is the benefit of hoisting only for speed of setup time?

[kirill-konshin]

@trusktr the thing is that not all tools are smart enough to find the binaries at the top level.

[swernerx]

FYI: I build a little tool on top of Lerna to link binaries offered by packages to the top level .bin folder. This makes it possible to use e.g. shared CI tools from a package. (We need this to be implemented that way because other users of the tool outside of the mono repo rely on it as well).

https://github.com/sebastian-software/link-lerna-package-binaries

[eric-burel]

nohoist seems deprecated but the replacing options doesn't give enough control: https://yarnpkg.com/configuration/manifest#installConfig

Any other hint on this? I have a Remix starter in a monorepo and I rely a lot on their CLI, it must stay in the right node_modules

Edit: I've configure my starter app package.json like so:

"installConfig": {
    "hoistingLimits": "dependencies"
  }

This way, I use hoisting for my packages (I have a depcheck to confirm that I didn't miss a dependency before deploying) but I disable it for the Remix starter up, it seems to work ok this way.

I still need to figure how to use the "plug and play" mode to avoid node_modules, if that's even possible.

This works better, however then running binaries with yarn in the starter/remix folder, seems to actually run it at the root. So for instance, yarn run-p will look for scripts at the root package.json level and not at the remix package.json level.

[kenfj]

FYI: I've encountered similar issue like yarn dev should call vite but got command not found: vite
because node_modules/.bin hasn't been created in some of sub-packages.

In my case, I added "@types/node": "^18.8.4", in the devDependencies and it worked 🎉

I have no idea why this worked but I found @types/node was missing in the not-working sub-packages by comparing other working sub-packages in the same workspaces.