[BUG] non-previosly seen peer-dependency errors popping up in 8.6.0

[azatoth]

Is there an existing issue for this?

• I have searched the existing issues

This issue exists in the latest npm version

• I am using the latest npm

Current Behavior

After upgrading from 8.5.5 to 8.6.0 I'm starting to get peer dependency issues that are not present when running under 8.5.5.

I don't see anything apparent in https://github.com/npm/cli/blob/latest/CHANGELOG.md#v860-2022-03-31 to account for this change. While it's possible these errors were present before, but hidden; the lack of changelog entry to account for this speaks against that.

$ npm i -g npm@8.5.5

changed 14 packages, and audited 201 packages in 1s

10 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
$ npm i

up to date, audited 1180 packages in 2s

130 packages are looking for funding
  run `npm fund` for details

4 vulnerabilities (2 moderate, 2 high)

To address all issues, run:
  npm audit fix

Run `npm audit` for details.
$ npm i -g npm@8.6.0

changed 30 packages, and audited 201 packages in 1s

10 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
$ npm i
npm ERR! code ERESOLVE
npm ERR! ERESOLVE could not resolve
npm ERR! 
npm ERR! While resolving: @shiftcoders/dynamo-easy@7.1.0
npm ERR! Found: tslib@2.3.0
npm ERR! node_modules/tslib
npm ERR!   tslib@"^2.1.0" from the root project
npm ERR!   tslib@"^2.0.0" from @aws-sdk/abort-controller@3.20.0
npm ERR!   node_modules/@aws-sdk/abort-controller
npm ERR!     @aws-sdk/abort-controller@"3.20.0" from @aws-sdk/node-http-handler@3.21.0
npm ERR!     node_modules/@aws-sdk/node-http-handler
npm ERR!       @aws-sdk/node-http-handler@"3.21.0" from @aws-sdk/client-sso@3.21.0
npm ERR!       node_modules/@aws-sdk/client-sso
npm ERR!         @aws-sdk/client-sso@"3.21.0" from @aws-sdk/credential-provider-sso@3.21.0
npm ERR!         node_modules/@aws-sdk/credential-provider-sso
npm ERR!       1 more (@aws-sdk/client-sts)
npm ERR!   46 more (@aws-sdk/client-sso, @aws-sdk/client-sts, ...)
npm ERR! 
npm ERR! Could not resolve dependency:
npm ERR! peer tslib@"^1.10.0" from @shiftcoders/dynamo-easy@7.1.0
npm ERR! node_modules/@shiftcoders/dynamo-easy
npm ERR!   dev @shiftcoders/dynamo-easy@"^7.1.0" from the root project
npm ERR! 
npm ERR! Conflicting peer dependency: tslib@1.14.1
npm ERR! node_modules/tslib
npm ERR!   peer tslib@"^1.10.0" from @shiftcoders/dynamo-easy@7.1.0
npm ERR!   node_modules/@shiftcoders/dynamo-easy
npm ERR!     dev @shiftcoders/dynamo-easy@"^7.1.0" from the root project
npm ERR! 
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR! 
npm ERR! See /home/azatoth/.npm/eresolve-report.txt for a full report.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/azatoth/.npm/_logs/2022-04-01T10_02_06_637Z-debug-0.log

Expected Behavior

I expect 8.6.0 to not error out on these peer-dependency issues.

Steps To Reproduce

No response

Environment

• npm: 8.6.0
• Node.js: v14.19.0
• OS Name: Ubuntu 21.10
• System Model Name:
• npm config:

; "user" config from /home/azatoth/.npmrc

//npm.pkg.github.com/:_authToken = (protected) 
//registry.npmjs.org/:_authToken = (protected) 

; "project" config from /home/azatoth/Project/(protected)/.npmrc

@(protected):registry = "https://npm.pkg.github.com/" 

; node bin location = /home/azatoth/.nvm/versions/node/v14.19.0/bin/node
; cwd = /home/azatoth/Project/(protected)
; HOME = /home/azatoth
; Run `npm config ls -l` to show all defaults.

[jeroennoten]

Same problem here. No problems on 8.5.5, errors regarding peer dependencies on 8.6.0.

[piccit]

I'm seeing this problem too but it's when running npm ci (after having installed some specific packages using --legacy-peer-deps)

[aldeed]

To be clear about how big this bug is, everybody with a CI step that does npm i -g npm && npm i (or npm ci), that step is now suddenly erring (assuming they have ever previously ignored peer dep errors, which most projects have). The workaround is to pin to npm@8.5.5 but to prevent opening the floodgates of confusion you may want to fix or revert ASAP.

[johanholmerin]

Git bisect shows it to be caused by bd96ae4. Reproduction at johanholmerin/npm-4664

[neogibson]

Yup we are seeing the same error in our CI with 8.6.0, reverting to 8.5.5 works for now though.

[ruyadorno]

hi @johanholmerin thanks so much for taking the time to bisect and sending a reproduction, super handy and we super appreciate it! ❤️

It looks like what most folks here are seeing here is a legitimate ERESOLVE error (which began throwing as of v7) that got stored to the lockfile via using either npm i --force or npm i --legacy-peer-deps to generate that file. This should have always been an error and the fix included in v8.6.0 is actually just surfacing these problems in the context of npm ci.

I apologize for the extra churn this fix seems to be inflicting in some users, we're taking the extra steps of specifically documenting that in the npm ci help page, ref: #4666

How to fix it:

In your project folder, run:

npm config set legacy-peer-deps=true --location=project

Then npm ci should be able to install your project again without errors.

[wSedlacek]

This should have always been an error and the fix included in v8.6.0 is actually just surfacing these problems in the context of npm ci.

So just to confirm it is not expected behavior to have installs done with --force to continue working on future installs without --force?

[piccit]

a legitimate ERESOLVE error (which began throwing as of v7) that got stored to the lockfile via using either npm i --force or npm i --legacy-peer-deps to generate that file.

I understand what you're saying here, but why is a ci doing that validation and not just using what is in the lockfile?

[jakeonfire]

@ruyadorno running npm config set legacy-peer-deps=true --location=project does indeed fix npm install but does not fix bin/webpack (which results in a stream of errors about not being able to resolve aliased packages like react and react-dom). any ideas? we are using preact and aliasify if that is relevant...

[nlf]

So just to confirm it is not expected behavior to have installs done with --force to continue working on future installs without --force?

correct.

I understand what you're saying here, but why is a ci doing that validation and not just using what is in the lockfile?

the root cause of this is that the package tree as described in your package-lock.json is invalid. previously, we did no validation and simply installed what your package-lock.json told us to install. doing this, however, was very bad as it produced broken or otherwise incorrect trees (i.e. overrides not being applied, changes from the package.json not being detected, etc).

we added a step that validates that your package-lock.json is even correct before we will install it to disk, this helps us detect these problems before they cause problems for our users. we do not plan to restore the behavior of installing whatever was in the package-lock.json without validation, as it causes more harm than good.

@ruyadorno running npm config set legacy-peer-deps=true --location=project does indeed fix npm install but does not fix bin/webpack (which results in a stream of errors about not being able to resolve aliased packages like react and react-dom). any ideas? we are using preact and aliasify if that is relevant...

can you open a new issue for us with reproduction steps? we can take a look and see if this is a problem npm is creating for you there.

[MoltenCoffee]

In our case this broke our production builds.
I think a transition with a warning in the CLI, possibly referring to the overrides option would have been more developer friendly, followed by a change in a major release, as it's not backwards compatible.

we added a step that validates that your package-lock.json is even correct before we will install it to disk, this helps us detect these problems before they cause problems for our users. we do not plan to restore the behavior of installing whatever was in the package-lock.json without validation, as it causes more harm than good.

I'd say this defeats the purpose of lockfiles. The only viable way to generate lockfiles is by using a package manager, which we can expect to make valid choices or actions based on user input, such as the legacy peer deps.
npm ci adding the validation would in my opinion be great for corrupted lockfiles, not for lockfiles based on decisions the package manager doesn't agree with.

I genuinely applaud the effort towards making NPM (and thus the ecosystem) more reliable, predictable and stable, I just think this change was a bit abrupt. Just my two cents.

[JeffJacobson]

In the issue title "previosly" is misspelled and should be "previously".

[wjaspers]

Also seeing this albeit my output shows identical version numbers are supposedly mismatched. This still seems off.

example:

npm ERR! Found: @angular/core@13.3.0
npm ERR! node_modules/@angular/forms
npm ERR!   @angular/core@"~13.3.0" from the root project

[lukekarrys]

@wjaspers that looks very similar to #3666. Does that issue look the same as yours? If not, opening a new issue with reproductions steps would be the best way for us to help.

[lukekarrys]

If anyone else is having related issues, please open a new issue so we can triage as I think we've gotten to the bottom of this one.

[AndreasA]

@lukekarrys #3666 is an issue since 7.x I think. This is new with 8.6.0 as mentioned above, the user was able to install everything fine with 8.5.x but since 8.6.0 this error occurs.

not really sure how that could be related to #3666 as that ticket is way older?

I just ran into a similar issue with Shopware 6.4.10.0 and their build-administration script but that might be an issue on their end but it was working with 8.5 as well.

[domeq]

@lukekarrys This is a new bug as @AndreasA is saying above and IMHO it's a serious one.

Simplest case to reproduce it:

1. Start in an empty directory with NPM 8.5.0 installed.

2. Do npm i react react-apollo-hooks@0.5.0.
   react-apollo-hooks has "react": "^16.8.0" in its peer deps so react 16 gets installed even though there's now react 18 available.

3. Now try updating react: npm i react@17.
   This prints nice warnings, but the update happens and the version gets bumped in package.json.

   npm WARN ERESOLVE overriding peer dependency
   npm WARN While resolving: undefined@undefined
   npm WARN Found: react@16.14.0
   npm WARN node_modules/react
   npm WARN   peer react@"^16.8.0" from react-apollo-hooks@0.5.0
   npm WARN   node_modules/react-apollo-hooks
   npm WARN     react-apollo-hooks@"^0.5.0" from the root project
   npm WARN   1 more (the root project)
   npm WARN
   npm WARN Could not resolve dependency:
   npm WARN peer react@"^16.8.0" from react-apollo-hooks@0.5.0
   npm WARN node_modules/react-apollo-hooks
   npm WARN   react-apollo-hooks@"^0.5.0" from the root project
   
   removed 3 packages, changed 1 package, and audited 19 packages in 689ms
   

4. Now during build/deploy you obviously run npm install or npm ci:

• NPM 8.5.0: works fine, no errors, warnings - nothing ✅
• NPM 8.6.0: installation fails with an error:

  npm ERR! code ERESOLVE
  npm ERR! ERESOLVE could not resolve
  npm ERR!
  npm ERR! While resolving: react-apollo-hooks@0.5.0
  npm ERR! Found: react@17.0.2
  npm ERR! node_modules/react
  npm ERR!   react@"^17.0.2" from the root project
  npm ERR!
  npm ERR! Could not resolve dependency:
  npm ERR! peer react@"^16.8.0" from react-apollo-hooks@0.5.0
  npm ERR! node_modules/react-apollo-hooks
  npm ERR!   react-apollo-hooks@"^0.5.0" from the root project
  npm ERR!
  npm ERR! Conflicting peer dependency: react@16.14.0
  npm ERR! node_modules/react
  npm ERR!   peer react@"^16.8.0" from react-apollo-hooks@0.5.0
  npm ERR!   node_modules/react-apollo-hooks
  npm ERR!     react-apollo-hooks@"^0.5.0" from the root project
  npm ERR!
  npm ERR! Fix the upstream dependency conflict, or retry
  npm ERR! this command with --force, or --legacy-peer-deps
  npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
  

As mentioned above:
it would make sense to show warnings about peer conflicts in package.json during npm install same way NPM does during package update, but suddenly starting to fail entire npm install is a very controversial change for a minor release.

[AndreasA]

There was a similar issue with 8.4.0 as the behavior regarding npm ci changed. It suddenly failed ifpackage-lock.json and package.json were not perfectly in synch.

Which I think is good to do and I actually had expected to be the case for quite some time. Hhowever, as it changed the behavior (potential failings CIs) it should not have happened in a minor version.

Any changes that are potentially breaking which this one is and the one from 8.4.0 should only occur in a major release because then people expect things to potentially break.

Also if I understand it correctly, this issue happens because peer dependencies are not updated, if a dependency is updated?

If so, isn't this also an issue with NPM because it should already fail the update in such a scenario. And maybe add an option to update the dependency and its peer dependencies. Similar to composer with --update-with-dependencies and --update-with-all-dependencies. and it fails to update a dependency if the peer dependencies would need to be updated but aren't (either by specifying them or by using the corresponding option)

[avegancafe]

Sorry to rehash this issue still, but what's the actual suggested solution for this? Always use npm install --force if someone that didn't know the implications of that happened to be upgrading or installing a package and saw that as a suggested recommended solution to their problem in the CLI? @domeq 's example seems pretty common, and doesn't even involve using --force

[MoltenCoffee]

Sorry to rehash this issue still, but what's the actual suggested solution for this?

I think most can be resolved using the overrides option in your package file

[fritzy]

I have created a pinned issue summarizing this discussion as well as #4998.

See the please refer to the pinned issue, #5113