[BUG] npm install on Mac - Out of Memory

[sathishsoundharajan]

Is there an existing issue for this?

• I have searched the existing issues

This issue exists in the latest npm version

• I am using the latest npm

Current Behavior

npm install stuck very long period of time. Eventually Mac Ran out of Memory started ask me to quit applications, even after quitting the applications npm stuck for very long period of time.

Expected Behavior

npm install should work fine

Steps To Reproduce

I dunno how to reproduce this issue outside my internal service.

Error screenshots

As you can see npm i is the only command which i ran, other npm install in activity manager created by npm i believe.
But process eventually shows this error

Environment

• npm: 8.10.0
• Node.js: v16.15.0
• OS Name: Mac OS Monterey
• System Model Name: Mac Book Pro 15, 2016 Model
• Config

cat ~/.npmrc
python=/Library/Frameworks/Python.framework/Versions/3.9/bin/python3

npm config ls -l
; "default" config from default values

_auth = (protected)
access = null
all = false
allow-same-version = false
also = null
audit = true
audit-level = null
auth-type = "legacy"
before = null
bin-links = true
browser = null
ca = null
cache = "/Users/sathish.sa/.npm"
cache-max = null
cache-min = 0
cafile = null
call = ""
cert = null
ci-name = null
cidr = null
color = true
commit-hooks = true
depth = null
description = true
dev = false
diff = []
diff-dst-prefix = "b/"
diff-ignore-all-space = false
diff-name-only = false
diff-no-prefix = false
diff-src-prefix = "a/"
diff-text = false
diff-unified = 3
dry-run = false
editor = "vi"
engine-strict = false
fetch-retries = 2
fetch-retry-factor = 10
fetch-retry-maxtimeout = 60000
fetch-retry-mintimeout = 10000
fetch-timeout = 300000
force = false
foreground-scripts = false
format-package-lock = true
fund = true
git = "git"
git-tag-version = true
global = false
global-style = false
globalconfig = "/usr/local/etc/npmrc"
heading = "npm"
https-proxy = null
if-present = false
ignore-scripts = false
include = []
include-staged = false
include-workspace-root = false
init-author-email = ""
init-author-name = ""
init-author-url = ""
init-license = "ISC"
init-module = "/Users/sathish.sa/.npm-init.js"
init-version = "1.0.0"
init.author.email = ""
init.author.name = ""
init.author.url = ""
init.license = "ISC"
init.module = "/Users/sathish.sa/.npm-init.js"
init.version = "1.0.0"
install-links = false
json = false
key = null
legacy-bundling = false
legacy-peer-deps = false
link = false
local-address = null
location = "user"
lockfile-version = null
loglevel = "notice"
logs-dir = null
logs-max = 10
; long = false ; overridden by cli
maxsockets = 15
message = "%s"
metrics-registry = "http://registry.npmjs.org/"
node-options = null
node-version = "v16.15.0"
noproxy = [""]
npm-version = "8.10.0"
offline = false
omit = []
omit-lockfile-registry-resolved = false
only = null
optional = null
otp = null
pack-destination = "."
package = []
package-lock = true
package-lock-only = false
parseable = false
prefer-offline = false
prefer-online = false
prefix = "/usr/local"
preid = ""
production = null
progress = true
proxy = null
read-only = false
rebuild-bundle = true
; registry = "https://registry.npmjs.org/" ; overridden by project
save = true
save-bundle = false
save-dev = false
save-exact = false
save-optional = false
save-peer = false
save-prefix = "^"
save-prod = false
scope = ""
script-shell = null
searchexclude = ""
searchlimit = 20
searchopts = ""
searchstaleness = 900
shell = "/bin/zsh"
shrinkwrap = true
sign-git-commit = false
sign-git-tag = false
sso-poll-frequency = 500
sso-type = "oauth"
strict-peer-deps = false
strict-ssl = true
tag = "latest"
tag-version-prefix = "v"
timing = false
tmp = "/var/folders/ht/gb72mbnj03d62yl8x0070vm80000gq/T"
umask = 0
unicode = true
update-notifier = true
usage = false
user-agent = "npm/{npm-version} node/{node-version} {platform} {arch} workspaces/{workspaces} {ci}"
userconfig = "/Users/sathish.sa/.npmrc"
version = false
versions = false
viewer = "man"
which = null
workspace = []
workspaces = null
workspaces-update = true
yes = null

; "user" config from /Users/sathish.sa/.npmrc

python = "/Library/Frameworks/Python.framework/Versions/3.9/bin/python3"

; "project" config from /Users/sathish.sa/Desktop/workspace/node-v16/service-v16/.npmrc

registry = "http://registry.npmjs.org/"

; "cli" config from command line options

long = true

[ljharb]

How many git deps are in your package.json?

[sathishsoundharajan]

35 @ljharb but each git dependency may have other git dependency.. Is that causing the issue ? But it was not an issue until we were using node 12.16.1 and npm 6.13.4

[ljharb]

Typically the number of git deps you want is zero, so 35 seems like a pretty high number and the likely culprit, but I'll let the npm team weigh in.

[sathishsoundharajan]

Oh. But my company kind of uses git dependency for private lib's.. Can i add any more info for the core team.. I can help anyway i can. @ljharb

[ljharb]

@sathishsoundharajan unrelated to this issue, i'd suggest your company use a private registry. git is for development, not consumption.

[sathishsoundharajan]

Sure Thanks for the suggestion @ljharb .. Will try my best to convince them.

[sathishsoundharajan]

For everyone interested in this thread, i have tried out these possibilities before raising this issue

1. Tried different internal repositories on different Macbook Pro's all of them reported the same issue.
2. Tried building the service via docker using node-alpine image, docker exited with code 137 ( out-of-memory ) after a while building the image.
3. Tried reducing the npm version to 6 but keeping node version 16 seems to stop this issue from happening but other native modules like "libxmljs" are failing to compile with this combination.

[sathishsoundharajan]

@dlockhart @ljharb In my case, we are using gitlab.. but i believe underlying issue is npm not able to handle git based dependencies.

[sathishsoundharajan]

Is there config, which i can tweak to reduce the no of npm install process OR git-remote-https process limit ?

It seems to be happening only after migrating node version 16 and npm version to 8+ latest.
This happens consistently.

[isaacs]

It looks like you have a git deps depending on one another in a cycle in order to perform their prepare scripts. npm checks out git deps into a temporary folder to construct artifacts that are needed in order to install them properly.

So, if you have git@github.com:org/a that requires git@github.com:org/b and b also depends on a, and they both have prepare scripts, then nothing can install until everything is installed, and npm can't know which one it should load first. Of course, fork bombing your machine is not a good failure mode, at all 😅

If you run with --ignore-scripts does it complete successfully? (Although, perhaps with broken deps, since it won't properly build them.)

Also: it should have created a npm-debug.log file somewhere and printed the path to it for you in the crash message. Can you look for a big log file in ~/.npm/_logs/ and share that (perhaps privately with the team, if it has internal repos that you don't want to leak the name/url -- it should be smart enough to filter out auth info you were foolish enough to hard code tokens in your dependency URLs, but I'd check to make sure).

(caveat: I no longer work on npm, though it's very likely that whatever bug is causing this is one I put in there at some point in the past)

[sathishsoundharajan]

Thank you @isaacs for your response.

Here is what i have tried after your comment,

1. Tried npm i --legacy-peer-deps --ignore-scripts --verbose the installation of packages stuck at one point and never moved to next step for almost 30mins ( so had to manually cancel the operation ). Even while running this --ignore-scripts command i can see bunch of npm install process created and stuck without exiting.
2. The example you have mentioned above for fork bomb. I believe it could be one of the reasons we do have internal lib which has the same kind of dependencies ( but the weird part is all of these issues didn't occur when we were using node.js 12.16.1 and with npm associated with it. )
3. The internal lib which we are maintaining does not have have pre or post (prepare scripts ), but those lib's dependencies may have prepare scripts.

I can share privately the logs through slack ( or anyone from npm core team ). I have cleared all the existing logs from ~.npm/_logs/ folder and then ran npm i --legacy-peer-deps --ignore-scripts --verbose. After manual cancellation of this installation process it has 11+ log files.

[sathishsoundharajan]

Reran the installation once again with npm i --legacy-peer-deps --verbose. Process ran for 1h 30mins got stuck and Macbook automatically logged out the user by killing all the process running internally.

This time also cleared the old logs from the folder and this process generated multiple logs.

[darcyclarke]

@sathishsoundharajan this seems like a duplicate of npm/pacote#127. The temporary workaround there looked like setting _PACOTE_NO_PREPARE_=true in your .npmrc file (but I wouldn't rely on that long-term).

I was able to repro this by just creating a circular linked dep with a prepare script (which I'm guessing is similar, if not the same to the git dep circular fork bombing @isaacs noted & the dupe/linked issue - if they're orthogonal then we should try to fix both)

We can look into how to help folks prevent this but the easiest route will always be limiting or eliminating your use of git deps (as @ljharb mentions).

Notably, prepare scripts are not effected by --ignore-scripts at the moment -- ref.

cli/workspaces/arborist/lib/arborist/rebuild.js

Lines 179 to 182 in 8a49e3a

	// links should run prepare scripts and only link bins after that
	if (type === 'links') {
	await this[_runScripts]('prepare')
	}

Which is why using it likely doesn't change anything. This is a separate issue/concern & something we'll likely change in the next major.

Let me know if setting _PACOTE_NO_PREPARE_=true resolves your problem though &/or if there's a specific version of npm which was/is working (from my research, it seems even npm@6 was/is susceptible to this).

[sathishsoundharajan]

Thank you so much @darcyclarke for looking into this.

My finding with setting _PACOTE_NO_PREPARE_=true

1. Tried creating .npmrc file under project root directory and tried running npm i --legacy-peer-deps --verbose. Didn't see any improvement.

npm config ls -l
; "default" config from default values

_auth = (protected)
access = null
all = false
allow-same-version = false
also = null
audit = true
audit-level = null
auth-type = "legacy"
before = null
bin-links = true
browser = null
ca = null
cache = "/Users/ecom-sathish.sa/.npm"
cache-max = null
cache-min = 0
cafile = null
call = ""
cert = null
ci-name = null
cidr = null
color = true
commit-hooks = true
depth = null
description = true
dev = false
diff = []
diff-dst-prefix = "b/"
diff-ignore-all-space = false
diff-name-only = false
diff-no-prefix = false
diff-src-prefix = "a/"
diff-text = false
diff-unified = 3
dry-run = false
editor = "vi"
engine-strict = false
fetch-retries = 2
fetch-retry-factor = 10
fetch-retry-maxtimeout = 60000
fetch-retry-mintimeout = 10000
fetch-timeout = 300000
force = false
foreground-scripts = false
format-package-lock = true
fund = true
git = "git"
git-tag-version = true
global = false
global-style = false
globalconfig = "/usr/local/etc/npmrc"
heading = "npm"
https-proxy = null
if-present = false
ignore-scripts = false
include = []
include-staged = false
include-workspace-root = false
init-author-email = ""
init-author-name = ""
init-author-url = ""
init-license = "ISC"
init-module = "/Users/ecom-sathish.sa/.npm-init.js"
init-version = "1.0.0"
init.author.email = ""
init.author.name = ""
init.author.url = ""
init.license = "ISC"
init.module = "/Users/ecom-sathish.sa/.npm-init.js"
init.version = "1.0.0"
json = false
key = null
legacy-bundling = false
legacy-peer-deps = false
link = false
local-address = null
location = "user"
lockfile-version = null
loglevel = "notice"
logs-max = 10
; long = false ; overridden by cli
maxsockets = 15
message = "%s"
metrics-registry = "https://registry.npmjs.org/"
node-options = null
node-version = "v16.15.0"
noproxy = [""]
npm-version = "8.5.5"
offline = false
omit = []
only = null
optional = null
otp = null
pack-destination = "."
package = []
package-lock = true
package-lock-only = false
parseable = false
prefer-offline = false
prefer-online = false
prefix = "/usr/local"
preid = ""
production = null
progress = true
proxy = null
read-only = false
rebuild-bundle = true
registry = "https://registry.npmjs.org/"
save = true
save-bundle = false
save-dev = false
save-exact = false
save-optional = false
save-peer = false
save-prefix = "^"
save-prod = false
scope = ""
script-shell = null
searchexclude = ""
searchlimit = 20
searchopts = ""
searchstaleness = 900
shell = "/bin/zsh"
shrinkwrap = true
sign-git-commit = false
sign-git-tag = false
sso-poll-frequency = 500
sso-type = "oauth"
strict-peer-deps = false
strict-ssl = true
tag = "latest"
tag-version-prefix = "v"
timing = false
tmp = "/var/folders/ht/gb72mbnj03d62yl8x0070vm80000gq/T"
umask = 0
unicode = true
update-notifier = true
usage = false
user-agent = "npm/{npm-version} node/{node-version} {platform} {arch} workspaces/{workspaces} {ci}"
userconfig = "/Users/ecom-sathish.sa/.npmrc"
version = false
versions = false
viewer = "man"
which = null
workspace = []
workspaces = null
yes = null

; "user" config from /Users/ecom-sathish.sa/.npmrc

python = "/Library/Frameworks/Python.framework/Versions/3.9/bin/python3"

; "project" config from /Users/ecom-sathish.sa/Desktop/workspace/node-v16/com-v16/.npmrc

_PACOTE_NO_PREPARE_ = (protected)

; "cli" config from command line options

long = true

2. When i went through this bug [BUG] out of memory on npm install: fork bomb preparing from git repos if they have scripts pacote#127 i was certain it should have solved the problem, so thought of debugging the npm cli by running node --inspect-brk /usr/local/n/versions/node/16.15.0/bin/npm i --legacy-peer-deps --verbose seems somehow process.env._PACOTE_NO_PREPARE_ is undefined. Even though npm config ls -l clearly shows the value is set.

3. So instead of relying on .npmrc i have manually set the variable _PACOTE_NO_PREPARE_=true node --inspect-brk /usr/local/n/versions/node/16.15.0/bin/npm i --legacy-peer-deps --verbose and tried debugging once again. This time value is set to true as expected. But even then npm install processes keep hogging the memory so i had to manually stop the debugging process.

My understanding from this code https://github.com/npm/pacote/blob/main/lib/git.js#L181 the forked process will have already resolved urls so it will skip creating new spawn by checking .includes. But i dunno how to debug the spawn processes..

[sathishsoundharajan]

Hey @darcyclarke & npm team,

Do you have any workaround for this issue :). Converting very large code base to private registry is going to take lot of time. So until we plan to that migration, is there way i can bypass this fork-bombs ?

Please let me know if any help i can do to contribute

[btmills]

I believe I have a repro of this in a public repository, eslint/eslint-plugin-markdown, that demonstrates a behavior change from npm 8.9 to 8.10. With npm<=8.9.0, CI installs successfully, but in npm>=8.10.0, the install step gets killed after several minutes.

If you wish to try it yourself, docker run -it --rm node:16 bash -c 'npm install -g npm@8.10 && git clone https://github.com/eslint/eslint-plugin-markdown.git && cd eslint-plugin-markdown/examples/react && npm install --verbose' will be killed after running out of memory with thousands of pids. Change npm@8.10 to npm@8.9, and everything succeeds. Adding --env _PACOTE_NO_PREPARE_=true after --rm has no effect.

Was this an intentional behavior change in npm 8.10? I only noticed because our CI started failing on unrelated changes, and it turns out main would now be broken.

[sathishsoundharajan]

@btmills Thanks for posting a public repro.. Hope this helps the team.

[sathishsoundharajan]

I believe my issue is similar to #4028

[sathishsoundharajan]

@darcyclarke Any update on fixing this issue ? This still happens with Node 18 LTS & its npm version

[darcyclarke]

@sathishsoundharajan hey! I'm no longer working at npm/GitHub but maybe @nlf / @saquibkhan can give you an idea of where this is.

[saquibkhan]

@darcyclarke Any update on fixing this issue ? This still happens with Node 18 LTS & its npm version

@sathishsoundharajan are you noticing this issue with npm 9. can you check the npm version you using?

This is currently in backlog and will check with team if we want to prioritize it.

also can you run this commend to fetch all packages using prepare npm query ":attr(scripts, [prepare])" | jq 'map(.name)|join("\n")' -r | xargs -I {}

[sathishsoundharajan]

Hey @darcyclarke Thank you for pointing me to right people. Wish you all the best for future endeavours

@saquibkhan
Yes we are still seeing this error in npm 9. If you need more info, please let me know.

Note:

1. node_modules installed using Node.js v12.16.1 & npm@6.13.4
2. npm query ":attr(scripts, [prepare])" | jq 'map(.name)|join("\n")' -r | xargs -I {} executing this command with npm@6.13.4 got error parse error: Invalid numeric literal at line 2, column 6
3. so just changed npm@9.5.0 and executed npm query ":attr(scripts, [prepare])" | jq 'map(.name)|join("\n")' -r | xargs -I {}

yargs
@google-cloud/recaptcha-enterprise
@grpc/grpc-js
@grpc/proto-loader
cliui
y18n
yargs
yargs-parser
@grpc/proto-loader
cliui
y18n
yargs
yargs-parser
acorn
uuid
amqplib
assert
rotating-file-stream
is-plain-object
uuid
uuid
z-schema
rotating-file-stream
z-schema
catharsis
is-plain-object
uuid
z-schema
dom-serializer
domhandler
domelementtype
domhandler
domutils
domhandler
amqplib
domelementtype
entities
uuid
domelementtype
entities
uuid
uuid
elliptic
emittery
uuid
z-schema
entities
uuid
error-stack-parser
acorn
uuid
z-schema
dom-serializer
domhandler
domutils
entities
htmlparser2
is-plain-object
uuid
z-schema
sonic-boom
uuid
rotating-file-stream
z-schema
uuid
rotating-file-stream
z-schema
uuid
rotating-file-stream
z-schema
soap
uuid
soap
uuid
soap
uuid
soap
uuid
soap
uuid
soap
uuid
soap
uuid
soap
uuid
soap
uuid
rotating-file-stream
soap
uuid
uuid
soap
uuid
z-schema
uuid
rotating-file-stream
z-schema
uuid
rotating-file-stream
z-schema
uuid
uuid
z-schema
rotating-file-stream
rotating-file-stream
uuid
z-schema
amqplib
assert
rotating-file-stream
domhandler
hapi-swagger
htmlparser2
soap
uuid
rotating-file-stream
amqplib
assert
domhandler
rotating-file-stream
hapi-swagger
htmlparser2
rotating-file-stream
soap
uuid
amqplib
assert
domhandler
hapi-swagger
htmlparser2
soap
uuid
domhandler
rotating-file-stream
soap
uuid
domhandler
htmlparser2
htmlparser2
is-plain-object
soap
uuid
amqplib
assert
domhandler
hapi-swagger
htmlparser2
soap
uuid
amqplib
assert
domhandler
hapi-swagger
htmlparser2
soap
uuid
amqplib
assert
rotating-file-stream
domhandler
hapi-swagger
htmlparser2
soap
uuid
rotating-file-stream
uuid
z-schema
uuid
rotating-file-stream
z-schema
uuid
rotating-file-stream
z-schema
rotating-file-stream
uuid
z-schema
uuid
z-schema
gaxios
gcp-metadata
get-caller-file
cliui
y18n
yargs
yargs-parser
uuid
z-schema
google-auth-library
google-gax
google-p12-pem
got
@sindresorhus/is
@szmarczak/http-timer
defer-to-connect
gtoken
gulp-jsdoc3
hapi-good-winston
hapi-swagger
htmlparser2
http-status
uuid
z-schema
istanbul-reports
uuid
z-schema
uuid
loupe
markdown-it-anchor
entities
yargs-parser
mime
uuid
cliui
y18n
yargs
yargs-parser
cliui
y18n
yargs
yargs-parser
node-fetch
uuid
nunjucks
yargs
dom-serializer
domhandler
domutils
entities
htmlparser2
is-plain-object
amqplib
assert
uuid
rotating-file-stream
soap
uuid
uuid
z-schema
rotating-file-stream
soap
uuid
amqplib
assert
rotating-file-stream
soap
uuid
amqplib
assert
rotating-file-stream
soap
uuid
amqplib
assert
soap
uuid
soap
uuid
soap
uuid
rotating-file-stream
uuid
soap
uuid
amqplib
assert
rotating-file-stream
soap
uuid
rotating-file-stream
soap
uuid
amqplib
assert
rotating-file-stream
soap
uuid
amqplib
assert
rotating-file-stream
soap
uuid
amqplib
assert
rotating-file-stream
soap
uuid
uuid
z-schema
dom-serializer
domhandler
domutils
entities
htmlparser2
is-plain-object
uuid
z-schema
pino-abstract-transport
uuid
z-schema
proto3-json-serializer
uuid
z-schema
uuid
uuid
z-schema
rfc6902
cliui
uuid
yargs
y18n
z-schema
uuid
rotating-file-stream
z-schema
uuid
z-schema
rotating-file-stream
z-schema
rotating-file-stream
soap
uuid
uuid
rotating-file-stream
soap
uuid
z-schema
soap
uuid
sonic-boom
uuid
z-schema
stackframe
yargs
uuid
z-schema
tail
dom-serializer
domhandler
domhandler
domutils
domhandler
entities
htmlparser2
dom-serializer
domhandler
domhandler
domutils
domhandler
entities
htmlparser2
dom-serializer
domhandler
domutils
entities
rotating-file-stream
htmlparser2
is-plain-object
rotating-file-stream
dom-serializer
domhandler
domhandler
domutils
domhandler
entities
htmlparser2
z-schema
uuid
z-schema
thread-stream
uuid
z-schema
type-detect
uuid
z-schema

[cscleison]

@saquibkhan this happens in npm9 as well. my team pinned npm to 6 and we've been waiting for a fix for a couple of years now 😅
there's another more popular issue duplicating this one #4028

the reason i assume this doesn't get traction is because it's inconsistent to repro. some people change OS, change laptop, rename a repo, change terminal and mitigate it. i'm pretty sure in our case is due to dependencies of our direct dependencies (material ui, core js etc)... so you will only face this issue if you have a bad combo of dependencies.

someone seems to have identified the root cause and opened an issue npm/pacote#127

[sathishsoundharajan]

Hey @fritzy @saquibkhan Are you able to identify the issue ?