[BUG] Overrides are not updating after running npm install

[Exelord]

Is there an existing issue for this?

• I have searched the existing issues

This issue exists in the latest npm version

• I am using the latest npm

Current Behavior

After updating the overrides, npm install is not updating the package.lock or node_modules.

To make it work I had to remove both package.lock and node_modules, then the overrides have been applied.

Expected Behavior

Change in overrides should update package.lock and node_modules after running npm install

Steps To Reproduce

1. Add a dependency
2. npm install
3. Add override
4. npm install

Environment

• npm: >=8.3.0
• Node.js: 16.13.2
• OS Name: macOS 12
• System Model Name:
• npm config:

; copy and paste output from `npm config ls` here

[rushilsrivastava]

Also facing it on 8.3.2

[lautapercuspain]

It's reproducible on NPM v 8.1.2 as well.

• Node.js: 16.13.2
• MacOS Monterrey 12.0.1

[zzdjk6]

Same issue here.

OS: MacOS 11.3.1
Node: 16.13.2
NPM: 8.3.2

[shdb2]

Experiencing the same issue.
Node: 16.13.2
npm: 8.4.0

[koflin]

Same here:

• NPM: 8.4.1
• Node: 16.13.2
• OS: Windows 10

[dbjorge]

Wanted to clarify that for some variations of this scenario, the results are even worse than "npm install doesn't update package-lock.json". Below is a repro case where the results are "npm install does update package-lock.json, such that a reviewer looking quickily over a diff would think that the override has taken effect, but actually it has only partially updated package-lock.json and the version of the package in node_modules does not reflect the override".

# Baseline setup, all as expected
> node --version
v16.14.0
> npm --version
8.4.1
> mkdir npm-cli-4232-repro && cd npm-cli-4232-repro && git init && npm init -y
> npm install is-odd
> rg '\"version\"' ./node_modules
./node_modules\is-number\package.json
4:  "version": "6.0.0",

./node_modules\is-odd\package.json
4:  "version": "3.0.1",
> git add package*.json && git commit -m 'before override'

# Now edit package.json adding "overrides": { "is-number": "5.0.0" }

# Expected behavior: npm install should update the lockfile and installed versions such that the override to v5.0.0 is satisfied
> npm install
# Actual behavior: the installed version of is-number remains 6.0.0, not 5.0.0
> rg '\"version\"' ./node_modules
./node_modules\is-number\package.json
4:  "version": "6.0.0",

./node_modules\is-odd\package.json
4:  "version": "3.0.1",
# Actual behavior: package-lock.json is only partially updated.
# A reviewer skimming the most obvious version number change will see that the is-odd requires section correctly indicates 5.0.0
# However, the is-number entry in dependencies is incorrectly still referencing 6.0.0
> git diff package-lock.json
diff --git a/package-lock.json b/package-lock.json
index 894fae9..2b7850b 100644
--- a/package-lock.json
+++ b/package-lock.json
@@ -34,8 +34,7 @@
   },
   "dependencies": {
     "is-number": {
-      "version": "6.0.0",
-      "resolved": "https://registry.npmjs.org/is-number/-/is-number-6.0.0.tgz",
+      "version": "https://registry.npmjs.org/is-number/-/is-number-6.0.0.tgz",
       "integrity": "sha512-Wu1VHeILBK8KAWJUAiSZQX94GmOE45Rg6/538fKwiloUu21KncEkYGPqob2oSZ5mUT73vLGrHQjKw3KMPwfDzg=="
     },
     "is-odd": {
@@ -43,7 +42,7 @@
       "resolved": "https://registry.npmjs.org/is-odd/-/is-odd-3.0.1.tgz",
       "integrity": "sha512-CQpnWPrDwmP1+SMHXZhtLtJv90yiyVfluGsX5iNCVkrhQtU3TQHsUWPG9wkdk9Lgd5yNpAg9jQEo90CBaXgWMA==",
       "requires": {
-        "is-number": "^6.0.0"
+        "is-number": "5.0.0"
       }
     }
   }

Once the user is in this scenario, both package-lock.json and /node_modules/ must be deleted in order to get the correct behavior from npm install; if you delete either one but not the other, npm install will return you to the unexpected state.

(Node 16.14.0, npm 8.4.1, Windows 11 21H2 22000.493)

[cristianoccazinsp]

Overrides not working at all with npm 8.4.1, e.g.:

  "overrides": {
    "react-native": {
      "hermes-engine": "~0.11.0"
    }
  },

[fexxdev]

npm@8.5.2, i can confirm it still doesn't work with this version

[grandsilence]

npm@8.1.2, node v16.13.2, Windows 10 21H2, same issue.

[AlanSl]

This issue is attracting some general "overrides don't work for me at all" comments in addition to the specific original reported issue, so it might be worth mentioning a couple of things to check for that can cause overrides to not work at all:

1. There's an undocumented limitation of overrides in monorepos with workspaces: overrides are only read in the project root package.json; overrides in workspace package.json files are ignored by design.
2. Overrides weren't released until NPM version 8.3.0 - @grandsilence overrides aren't working for you because you're on npm 8.1.2

---

So if it seems overrides don't work at all, and you think this issue might be related, do npm install -g npm@latest and check the overrides aren't in a workspace (and hoist them if they are). Then try the steps described above (e.g. deleting package-lock and node_modules, then looking at exactly what's being written to package-lock).

[niiick1]

I was also expecting it to work on npm 8.1.2. If it only works on 8.3.0 and above, the documentation should state that. Based on this, I assumed it works on any npm 8:

[ljharb]

8.x is a range that resolves to the latest version only; iow, to 8.5.4 as of this writing.

[niiick1]

If that's the case, it should be written more clearly it's latest version only IMO

I was not the only one here that tested on 8.1.2. I think it would be beneficial having a since 8.x.y written for each property in the documentation. It would be helpful folks like me that can't always use latest.

There's no need to fish npm's changelog to check that :)

[ljharb]

That sounds like it’d be a great improvement; node’s docs tell you which versions each thing landed in and it’s very helpful.

[sirbarrence]

In case it helps anyone, running npm update instead of npm install will update the overrides in the lockfile without needing to delete the lockfile or node_modules. Unfortunately it also tries to update all other packages and takes forever, which you may not want.

Running npm update <name-of-override-package> just installs the latest default version that the other dependencies want rather than the override version. So that's no help.

[amerikan]

I'm on npm 8.4.0.

I added:

"overrides": {
+    "google-calendar": {
+      "needle": "2.9.1"
+    }
+  },

Ran npm install and then the following lines were changed in package-lock.json.

[dios-david]

Same as @amerikan using npm 8.3.1 or latest (8.5.4) on osx arm64

[ljharb]

Can all of you try latest npm, 8.5.4 i think?

[craigjbass]

I am seeing this issue on npm 8.5.4

[jpveooys]

Can confirm I also seem to be seeing this and #4322 with npm 8.5.4 while investigating migrating from yarn v1 to npm v8.

My working branch is here if it helps: https://github.com/defencedigital/mod-uk-design-system/compare/spike/npm-v8-2

The project uses workspaces and the overrides are in the root package.json. (Note there is one peer dependency error you will need to bypass.)

In that example, trim stays on 0.0.1 after npm install (which npm update fixes), while axe-core ends up on mixed versions no matter what I do.

[sbonami]

In case it helps, my specific scenario correctly sees the override applied to the first-level package. Where it fails is in the much-deeper instance. I'm not familiar with the internals of npm's tree generation, but this coincides with a working override for the dependency in the root of node_modules but not for the nested version.

OS: MacOS 12.3.1
Node: 16.14.2
NPM: 8.5.0

$ cat package.json

{
  ...
  "overrides": {
    "glob-parent": "^6.0.2"
  }
}

$ npm ls glob-parent

package-name@#.#.# /path/to/package
├─┬ @typescript-eslint/parser@5.27.1
│ └─┬ @typescript-eslint/typescript-estree@5.27.1
│   └─┬ globby@11.1.0
│     └─┬ fast-glob@3.2.11
│       └── glob-parent@5.1.2
└─┬ eslint@8.17.0
  └── glob-parent@6.0.2
  
$ npm ls fast-glob

package-name@#.#.# /path/to/package
├─┬ @typescript-eslint/parser@5.27.1
│ └─┬ @typescript-eslint/typescript-estree@5.27.1
│   └─┬ globby@11.1.0
│     └── fast-glob@3.2.11

/path/to/package/node_modules
├─┬ fast-glob/
│ └─┬ node_modules/
│   └── glob-parent@5.1.2
└── glob-parent@6.0.2

[patoncrispy]

If you're in a workspace, move the override to the root package.json. That works for me.

[Especially]

Note that the INITIAL install will abide by the override rules set, and the subsequent installs (e.g., run npm install twice) will ignore overrides.

[julian91]

This problem still exists for workspaces. We spent 2 weeks migrating a monorepo with ~30 microservices and ~50 modules to npm workspaces X turborepo.
Now having to exclude 1 of our frontend projects from the workspace and having an entire CI/CD flow for one service seperately from the rest of the monorepo because it cant deal with vue3 (yet) is pretty frustrating.

Are there at least any updates IF or WHEN this will be fixed?

[surbhirverma]

Same, I was able to reproduce this issue

[ForbiddenEra]

I'm seeing this issue on 8.5.5 currently; removing node_modules and package-lock.json and doing npm install does install the overridden dependencies, however, I'm also seeing a different, potentially related issue here.

When doing npm ls --depth=10, I get errors:
(any lines with ... have been removed for brevity)

...
├── cookie@0.5.0
...
├─┬ socket.io@4.5.2
...
│ ├─┬ engine.io@6.2.0
...
│ │ ├── cookie@0.5.0 deduped invalid: "~0.4.1" from node_modules/engine.io
...
│ │ └── ws@8.8.1 deduped invalid: "~8.2.3" from node_modules/engine.io
...
├─┬ ws@8.8.1
│ ├── UNMET OPTIONAL DEPENDENCY bufferutil@^4.0.1
│ └── UNMET OPTIONAL DEPENDENCY utf-8-validate@^5.0.2

npm ERR! code ELSPROBLEMS
npm ERR! invalid: cookie@0.5.0 ~/example/node_modules/cookie
npm ERR! invalid: ws@8.8.1 ~/example/node_modules/ws

My package.json contains:

        "dependencies": {
...
                "cookie": "^0.5.0",
                "socket.io": "^4.5.2",
...
                "ws": "^8.8.1",
        },
        "overrides": {
                "ws": "$ws",
                "cookie": "$cookie"
        },
...

I've also tried:

        "overrides": {
                "socket.io": {
                       "ws": "$ws",
                       "cookie": "$cookie"
                }
        }

and,

        "overrides": {
                "socket.io": {
                        "engine.io": {
                                "ws": "$ws",
                                "cookie": "$cookie"
                        }
                },
        },

and even,

        "overrides": {
                "socket.io": {
                        "engine.io": {
                                "ws": "$ws",
                                "cookie": "$cookie"
                        }
                },
                "ws": "$ws",
                "cookie": "$cookie"
        },

with no difference.

Another oddity I noticed - if I edit node_modules/engine.io/package.json and make NO changes (so, basically just touching the file / updating it's last modified) the error disappears without running anything between that and npm ls --depth=10. However, if I subsequently run npm install and then do npm ls --depth=10, the error comes back.

If I change node_modules/engine.io/package.json to use ^ instead of ~ for the packages and remove package-lock.json and run install, nothing changes and I still get the error - setting ^ should allow the replacement versions (8.8.1 for ws and 0.5.0 for cookie) to be allowed for that package. I suppose if I also removed the node_modules along with package-lock.json and then did npm install, that it might work, however then the changes made to that file would obviously be gone.

Potentially even worse than this (eg. other than the error on npm ls and the pain of having to force it to install the overrides by removing node_modules and package-lock.json) and possibly more frustrating/damaging is that running a npm dedupe actually replaces the overridden versions with the versions requested by the package instead! Basically npm dedupe in this situation is un-deduping! (But then, of course, the npm ls --depth=10 error is gone!)

# npm dedupe

added 2 packages, ...
# npm ls --depth=10
├─┬ cookie-parser@1.4.6
...
│ └── cookie@0.5.0 deduped
├── cookie@0.5.0
...
├─┬ socket.io@4.5.2
...
│ ├─┬ engine.io@6.2.0
...
│ │ ├── cookie@0.4.2
...
│ │ └─┬ ws@8.2.3
...
├─┬ ws@8.8.1
...

I may try updating (however this particular dev machine is on Ubuntu 18.04.6 LTS which has npm 8.5.5 as it's latest packaged version) or attempting to reproduce on another machine with a later npm version but it's not a show-stopper right now so not a huge priority (unless someone can confirm at least the dedupe and npm ls --depth=10 issues are fixed in a particular version, the not updating/installing the overridden version after adding an override in package.json is annoying but doesn't show errors or replace with the wrong version like the other issues so it's less of a concern).

[HomyeeKing]

so is there any solution here?
I think the npm docs should clarify what should do to make overrides rules work

[samuliasmala]

Note that the INITIAL install will abide by the override rules set, and the subsequent installs (e.g., run npm install twice) will ignore overrides.

I can confirm this is the behavior in the latest npm@8.19.2. This can be reproduced easily with the following package.json:

{
  "name": "test",
  "version": "1.0.0",
  "engines": {
    "npm": ">=8.3.0"
  },
  "dependencies": {
    "json-server": "^0.17.0"
  },
  "overrides": {
    "json-server": {
      "package-json": "7.0.0"
    }
  }
}

1. npm install in the folder containing only the above package.json --> 0 vulnerabilities
2. Subsequent npm install right after the previous (so node_modules and package-lock.json exists) --> 5 vulnerabilities
3. npm update --> 0 vulnerabilities
4. rm -rf node_modules/ && npm install --> 5 vulnerabilities
5. rm package-lock.json && npm install --> 5 vulnerabilities
6. rm -rf node_modules/ && rm package-lock.json && npm install --> 0 vulnerabilities

From the above it can be concluded that the overrides property is only honored when running npm install first time (i.e. without package-lock.json and node_modules present) and when running npm update.

[lukekarrys]

I have not gone through all the reports in this thread since it was closed. But I did check #4232 (comment) from @samuliasmala and I have confirmed that issue, so I opened a new issue for it here #5850.

If anyone else has a different issue than that, it's most helpful if you can open a new issue with your reproduction.

[rene-leanix]

From the above it can be concluded that the overrides property is only honored when running npm install first time (i.e. without package-lock.json and node_modules present) and when running npm update.

Uninstalling and reinstalling just the package for which the override was defined seems to be sufficient:

npm uninstall json-server && npm install json-server

(With --save-dev for dev dependencies, of course.)

[KuSh]

Doesn't work with latest npm version for firebase-tools at all :

❯ npm --version
9.2.0
❯ jq .overrides package.json
{
  "express": "^4.17.3",
  "got": "^11.8.5"
}
❯ npm uninstall firebase-tools

removed 687 packages, and audited 872 packages in 6s

104 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
❯ npm install -D firebase-tools
npm WARN deprecated har-validator@5.1.3: this library is no longer supported
npm WARN deprecated debug@4.1.1: Debug versions >=3.2.0 <3.2.7 || >=4 <4.3.1 have a low-severity ReDos regression when used in a Node.js environment. It is recommended you upgrade to 3.2.7 or 4.3.1. (https://github.com/visionmedia/debug/issues/797)
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142

added 687 packages, and audited 1559 packages in 9s

130 packages are looking for funding
  run `npm fund` for details

10 vulnerabilities (6 moderate, 4 high)

To address all issues, run:
  npm audit fix

Run `npm audit` for details.

I've tested many combinations without much luck :

{
  "overrides": {
    "firebase-tools": {
      "express": "^4.17.3",
      "got": "^11.8.5"
    }
  }
}

{
  "overrides": {
    "firebase-tools": {
      ".": "$firebase-tools",
      "express": "^4.17.3",
      "got": "^11.8.5"
    }
  }
}

{
  "overrides": {
    "firebase-tools": {
      ".": "$firebase-tools",
      "express": "^4.17.3",
      "package-json": {
        "got": "^11.8.5"
      }
    }
  }
}

[amanda-eero]

From the above it can be concluded that the overrides property is only honored when running npm install first time (i.e. without package-lock.json and node_modules present) and when running npm update.

Uninstalling and reinstalling just the package for which the override was defined seems to be sufficient:

npm uninstall json-server && npm install json-server

(With --save-dev for dev dependencies, of course.)

This works for me, thank you very much for sharing!!! In this case I didn't need to remove the node_modules neither the package-lock.json to override correctly.

[KorayAydemir]

It never works for me. Trying to override powerglitch dependency of react-powerglitch:
package.json in project root:

"overrides":{
    "react-powerglitch":{"powerglitch":"2.3.0"}
},

In /node_modules/react-powerglitch/package.json, after trying everything above, like deleting both node_modules and package-lock.json

{
...
  "dependencies": {
    "powerglitch": "^2.2.0"
  },
...
}

[maximkoretskiy]

I spent a day on the issue and it feels like the bug only reveals when a dependency has its own package-lock.json or npm-shrinkwrap.json.