Yarn wastefully caches and duplicates file:* dependencies

[mourner]

Do you want to request a feature or report a bug?

Bug.

What is the current behavior?

When using a file:<path> dependency format, Yarn caches the contents of the local directory every time yarn reinstalls packages (e.g. on yarn --force). This is wasteful, because the directory is already present locally and doesn't need caching. But more importantly, it can bloat Yarn cache very quickly if the dependency folder is relatively large, because it always creates a new copy even if the contents are the same.

I discovered this when I found out my Yarn cache directory is 10GB, most of which is dozens of copies of the same folder I had as a file dependency.

If the current behavior is a bug, please provide the steps to reproduce.

// package.json
{
  "name": "yarn-file-cache-bug",
  "version": "1.0.0",
  "dependencies": {
    "dep": "file:dep"
  }
}

// dep/package.json
{
  "name": "dep",
  "version": "1.0.0"
}

Then run yarn --force multiple times. Every run will generate a new copy of the dependency:

$ find ~/Library/Caches/Yarn/v4 -iname "npm-dep*" -maxdepth 1
/Users/mourner/Library/Caches/Yarn/v4/npm-dep-1.0.0-5e2e1343-9d0e-4c2c-831c-bd20816a2c14-1555338013144
/Users/mourner/Library/Caches/Yarn/v4/npm-dep-1.0.0-a47e7ec3-3bfa-4630-9ecc-402e4240e46d-1555338004665
/Users/mourner/Library/Caches/Yarn/v4/npm-dep-1.0.0-17a1c86d-ed38-43b0-80cc-aeb881dd87ae-1555338015639
/Users/mourner/Library/Caches/Yarn/v4/npm-dep-1.0.0-1e267f7b-2839-4e6e-af41-1a4f620ca4de-1555337975522
/Users/mourner/Library/Caches/Yarn/v4/npm-dep-1.0.0-4db913f2-8a1a-4e71-88be-99b7a88f734a-1555338009514

What is the expected behavior?

Yarn should not cache file:<path> dependencies at all.

Please mention your node.js, yarn and operating system version.

OS: macOS Mojave v10.14
Node: v10.15.2
Yarn: 1.15.2

[einnjo]

This looks like a duplicate of #2165, which is closed for whatever reason, but we are still running into the same problem today.

[OKNoah]

I still seem to have this problem although I will have to double check. Had to clear our 20-30GB of Yarn cache a few times in a day and I worry it's gonna wear out my hardware.

Also, the docs say the cache should be in ~/.yarn/cache (or was it the other way around). There is no built-in way to disable it either.

[0xdeafcafe]

Still an issue in 2021 🙃.

[jonni-larjomaa]

Still seems to be an issue 2023

[Davidiusdadi]

wow - yes this is still an issue: just clearing my yarn cache dir which was ~250GB mostly due to file: dependencies 🤯
If the file: dependency is really big - repeated yarn installs can really add up quickly.