"yarn upgrade" of tagged dependency fails with "expected manifest"

[tomasAlabes]

Do you want to request a feature or report a bug?

Bug
What is the current behavior?
An error running yarn upgrade myDep where myDep needs "next" version.

error An unexpected error occurred: "expected manifest".
info If you think this is a bug, please open a bug report with the information provided in "/Users/talabes/repositories2/dynamic-process-editor/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/upgrade for documentation about this command.

If the current behavior is a bug, please provide the steps to reproduce.

With a package.json with:

dependencies: {
   "myDep": "next"
}

Running yarn upgrade myDep when there's a new next version of myDep generates this error.

error An unexpected error occurred: "expected manifest".
info If you think this is a bug, please open a bug report with the information provided in "/Users/talabes/repositories2/dynamic-process-editor/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/upgrade for documentation about this command.

What is the expected behavior?
Not throw and error, and either

• Modify the yarn.lock, and not the package.json.
• Or modify both, changing the version in the package.json to something like 1.0.0-next.7 (whatever the "next" tag points to).

I personally need the first option, but I don't know what the yarn developers think. I created this issue some time ago: #4632, and there's another that seems similar: #4382, #2788, #2648, #1690, #2154

Also, should yarn upgrade --next work? Or is there a way to upgrade all dependencies tagged with "next" (or any other tag)?

Please mention your node.js, yarn and operating system version.
yarn 1.2.1
node 8.7.0
macos 10.12.6

[tomasAlabes]

Having

• package.json -> "myDep": "next"
• yarn.lock -> myDep@1.0.0-next.1

I found that yarn is looking for the manifest of myDep@next when the pattern is 1.0.0-next.1 (the version in the yarn.lock). Shouldn't yarn have the above pattern?

[tomasAlabes]

Hi @kaylieEB, there's seems to be many opportunities for improvement in yarn upgrade, I've listed some of the issues raised before besides mine. Do you think you can help?
Thanks!

[juanallo]

I can confirm that this happens in yarn 1.2.1 but it doesn't in 1.1.0

[kaylie-alexa]

@tomasAlabes thanks for filing the issue! I think you're right dist-tags can be better handled since we currently only really support latest. How would you like me to help? If you want to submit a PR, this is a good place to start looking https://github.com/yarnpkg/yarn/blob/3d6a03935e17db88891bb9673b245edd99a23bc5/src/package-request.js

[rally25rs]

This is an interesting issue.The behavior for tag names in package.json seems... not well defined. NPM5 actually leaves the tag in place during install, but replaces them with the actual semver version when you upgrade:

$ npm -v
5.5.1

$ cat package.json
{
  ...,
  "dependencies": {
    "grunt": "rc1"
  }
}

$ npm i
npm WARN yarn-test@1.0.0 No description
npm WARN yarn-test@1.0.0 No repository field.

added 85 packages in 1.579s

$ cat package.json
{
  ...,
  "dependencies": {
    "grunt": "rc1"
  }
}

$ npm upgrade grunt
npm WARN yarn-test@1.0.0 No description
npm WARN yarn-test@1.0.0 No repository field.

+ grunt@1.0.0-rc1
added 85 packages in 1.234s

$ cat package.json
{
  ...,
  "dependencies": {
    "grunt": "^1.0.0-rc1"
  }
}

Attempting to reproduce it with Yarn, I actually get the same behavior with 1.2.1:

$ cat package.json
{
  ...,
  "dependencies": {
    "grunt": "rc1"
  }
}

$ yarn
yarn install v1.2.1
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
success Saved lockfile.
✨  Done in 1.09s.

$ cat package.json
{
  ...,
  "dependencies": {
    "grunt": "rc1"
  }
}

$ yarn upgrade grunt
yarn upgrade v1.2.1
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Rebuilding all packages...
success Saved lockfile.
success Saved 80 new dependencies.
├─ abbrev@1.1.1
├─ argparse@1.0.9
├─ array-find-index@1.0.2
├─ async@1.5.2
├─ balanced-match@1.0.0
├─ brace-expansion@1.1.8
├─ builtin-modules@1.1.1
├─ camelcase-keys@2.1.0
├─ camelcase@2.1.1
├─ coffee-script@1.10.0
├─ colors@1.1.2
├─ concat-map@0.0.1
├─ currently-unhandled@0.4.1
├─ dateformat@1.0.12
├─ decamelize@1.2.0
├─ error-ex@1.3.1
├─ esprima@2.7.3
├─ eventemitter2@0.4.14
├─ exit@0.1.2
├─ find-up@1.1.2
├─ findup-sync@0.3.0
├─ get-stdin@4.0.1
├─ getobject@0.1.0
├─ glob@6.0.4
├─ graceful-fs@4.1.11
├─ grunt-cli@1.0.0-rc1
├─ grunt-legacy-log-utils@0.1.1
├─ grunt-legacy-log@1.0.0-rc1
├─ grunt-legacy-util@1.0.0-rc1
├─ grunt@1.0.0-rc1
├─ hooker@0.2.3
├─ hosted-git-info@2.5.0
├─ iconv-lite@0.4.19
├─ indent-string@2.1.0
├─ inflight@1.0.6
├─ inherits@2.0.3
├─ is-arrayish@0.2.1
├─ is-builtin-module@1.0.0
├─ is-finite@1.0.2
├─ is-utf8@0.2.1
├─ isexe@2.0.0
├─ js-yaml@3.5.5
├─ load-json-file@1.1.0
├─ lodash@3.10.1
├─ loud-rejection@1.6.0
├─ map-obj@1.0.1
├─ meow@3.7.0
├─ minimatch@3.0.4
├─ minimist@1.2.0
├─ nopt@3.0.6
├─ normalize-package-data@2.4.0
├─ number-is-nan@1.0.1
├─ object-assign@4.1.1
├─ once@1.4.0
├─ parse-json@2.2.0
├─ path-exists@2.1.0
├─ path-is-absolute@1.0.1
├─ path-type@1.1.0
├─ pify@2.3.0
├─ pinkie-promise@2.0.1
├─ pinkie@2.0.4
├─ read-pkg-up@1.0.1
├─ read-pkg@1.1.0
├─ redent@1.0.0
├─ repeating@2.0.1
├─ resolve@1.1.7
├─ rimraf@2.2.8
├─ semver@5.4.1
├─ signal-exit@3.0.2
├─ spdx-correct@1.0.2
├─ spdx-expression-parse@1.0.4
├─ spdx-license-ids@1.2.2
├─ sprintf-js@1.0.3
├─ strip-bom@2.0.0
├─ strip-indent@1.0.1
├─ trim-newlines@1.0.0
├─ underscore.string@3.2.3
├─ validate-npm-package-license@3.0.1
├─ which@1.2.14
└─ wrappy@1.0.2
✨  Done in 6.04s.

$ cat package.json
{
  ...,
  "dependencies": {
    "grunt": "^1.0.0-rc1"
  }
}

[michalstocki]

@rally25rs I can confirm this behavior in yarn 1.2.1, however I run into a problem described in the issue in the following situation:

$ cat package.json
{
  ...,
  "dependencies": {
    "@my-org/my-dep": "master"
  }
}

$ yarn -v
1.2.1

$ yarn add @my-org/my-dep@myTag
yarn add v1.2.1
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
error An unexpected error occurred: "expected manifest".

$ yarn upgrade @my-org/my-dep@myTag
yarn upgrade v1.2.1
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
error An unexpected error occurred: "expected manifest".

$ npm dist-tags ls @my-org/my-dep
myTag: 0.2.0-myTag.282
latest: 0.2.0-master.500
master: 0.2.0-master.264

I have the same behavior with the yarn installed with --nightly flag (v1.2.1-20171026.1140)

[tomasAlabes]

As far as I could debug the issue, I found that the patterns used by the https://github.com/yarnpkg/yarn/blob/master/src/package-resolver.js#L379 don't contain the manifest of my-dep@myTag, declared in the package.json. Instead, it contains the pattern matching the actual version in the yarn.lock, something like my-dep@1.0.0-myTag.1.

For example, patterns map:

• grunt@1.0.1 -> { manifest }
• my-dep@1.0.0-myTag.1 -> { manifest }

And the getStrictResolvedPattern() in https://github.com/yarnpkg/yarn/blob/master/src/package-resolver.js#L387 is looking for my-dep@myTag, so it fails.

[kirill-konshin]

The issue still exists 2 years after the inception, current Yarn version is 1.19.1.

[kirill-konshin]

yarn outdated by the way gives correct result.

[zhenwenc]

Any update on this issue?

[alexgorbatchev]

also seeing the same issue on yarn 1.22.18