yarn misuses cache on git dependencies

[SEAPUNK]

Do you want to request a feature or report a bug?
Bug

What is the current behavior?
misuses cache to install the git dependency at the wrong commit, seems to happen particularly when specifying git branch

If the current behavior is a bug, please provide the steps to reproduce.
sample repo: https://github.com/SEAPUNK/yarn-test-thing

1. yarn add SEAPUNK/yarn-test-thing
2. edit package.json's dependency from SEAPUNK/yarn-test-thing to SEAPUNK/yarn-test-thing#branch
3. edit the yarn.lock file to change the single dependency from what it is to

yarn-test-thing@seapunk/yarn-test-thing#branch:
  version "0.0.1-something"
  resolved "https://codeload.github.com/seapunk/yarn-test-thing/tar.gz/3ea9fe7984b9005f3cc62b14ee4018013ae0db0f"

i.e. what it will be when you run yarn upgrade

4. yarn install
5. inspect node_modules/yarn-test-thing, and see that the files BRANCH_ONE and BRANCH_TWO are not there

What is the expected behavior?
installs the right version at commit 3ea9fe7984b9005f3cc62b14ee4018013ae0db0f

Please mention your node.js, yarn and operating system version.

node.js 8.7.0
yarn 1.2.1
macos high sierra

---

this is particularly annoying when you have committed a commit to the git dependency branch, ran yarn upgrade, check in the new yarn lockfile to the repo you're working on, only for yarn to not install that "resolved" version in the lockfile when someone else checks the code out and runs yarn install

[SEAPUNK]

i'm lazy so i created what i think the tl;dr of the repro steps are, if they don't work, lmk and i'll give you something that should work better

[jwarby]

Also just got bitten by this with a GitHub dependency - CI build failed after pushing, even though yarn.lock was updated with the new commit hash. Worked around the issue for now by making CI pipeline run yarn cache clean after the other steps in my build.

[SEAPUNK]

got bitten by it again; the problem is that yarn is caching the package by its version in package.json, and not by git commit; so it sees in the lockfile that version is 0.0.1-something, looks for the cache for 0.0.1-something of the package, and uses that

the fix for this is to cache these installed-from-git packages by git commit, so in the yarn cache dir it'd look something like git-yarn-test-thing-3ea9fe7984b9005f3cc62b14ee4018013ae0db0f instead of npm-yarn-test-thing-0.0.1-something

[isaachinman]

The methodology outline by @SEAPUNK seems sound, installing from a git repository directly would indicate you are interested in having the latest codebase irrespective of versions/releases.

[cmawhorter]

this seemed to be fixed (in 1.7 i think) but since upgrading to 1.9.4 it is back to broken and worse.

in the prior problematic state, i used to be able to bump the version in the target dep to force yarn to update it, but now that's not working and i've been removing the yarn.lock file to get around the issue.

i admittedly forgot about yarn cache clean so can't comment on that atm.

Edit: yarn clean cache works

[lidel]

Alternative workaround for github dev deps is to define it as URL:

"devDependencies": {
  "yarn-test-thing": "https://github.com/seapunk/yarn-test-thing/tarball/3ea9fe7984b9005f3cc62b14ee4018013ae0db0f/yarn-test-thing.tar.gz",

It works around github-specific handlers and just does a plain HTTP download with a checksum (works fine for me so far).

[FelipeCoimbra]

Has this issue been resolved in recent yarn versions? I'm experiencing this problem using yarn 1.6.0 and wondering if I can solve this by upgrading.

[bantic]

I've observed this behavior as well, with @rondale-sc, using both yarn 1.15.2 and yarn 1.13.0. In our case it appears that the way the dependency is specified mattered. I.e., specifying it as user/repo#SHA does not update as expected (more details below), but specifying as https://github.com/user/repo.git#SHA does work as expected.

The situation:

• package.json specifies a dependency using user/repo#SHA Github URL form

  "devDependencies": {
    "my-dep": "user/repo#SHA"

• Someone else makes a commit that updates the dep SHA in package.json and the resolved sha and url in yarn.lock, but the version of the dependency doesn't change.
• I do git pull and pick up that commit
• I do yarn install, and it does not update the package on disk to the one specified by the sha.

I was able to fix this in 3 different ways:

1. rm yarn.lock && yarn install
2. rm -rf node_modules && yarn cache clean && yarn install
3. Change the dependency specification to more explicit URL form https://github.com/user/repo.git#SHA before making the commit as described above.

Observed with node 8.15 and macOS Mojave.

Updated to point out that we were specifying deps using the short user/repo#SHA Github URL form, but that it works when using the more explicit URL form https://github.com/user/repo.git#SHA.