-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] npm cache verify EMFILE #4783
Comments
We've also been experiencing the same |
This happens in our K8s based Jenkins runs as well. Any idea how to remedy this problem? |
I've started seeing this error consistantly with npm version 9.1.1. PS C:\Users\username> nvs use 18
PATH -= C:\Program Files\nodejs
PATH += C:\Users\username\scoop\apps\nvs\current\nodejs\node\18.12.1\x64
PS C:\Users\username> npm -v
8.19.2
PS C:\Users\username> npm cache verify
Cache verified and compressed (~\AppData\Local\npm-cache\_cacache)
Content verified: 9299 (1723927212 bytes)
Content garbage-collected: 26 (46328224 bytes)
Index entries: 9312
Finished in 105.62s But with npm v9.1.1 I get PS C:\Users\username> npm -v
9.1.1
PS C:\Users\username> npm cache verify
npm ERR! code EMFILE
npm ERR! syscall open
npm ERR! path C:\Users\username\AppData\Local\npm-cache\_cacache\index-v5\e1\3b\ae2dd6c9c3c3772cfeb540619f7ef408d59bf74e601601105094cb752b82
npm ERR! errno -4066
npm ERR! EMFILE: too many open files, open 'C:\Users\username\AppData\Local\npm-cache\_cacache\index-v5\e1\3b\ae2dd6c9c3c3772cfeb540619f7ef408d59bf74e601601105094cb752b82'
npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\username\AppData\Local\npm-cache\_logs\2022-11-13T15_00_57_816Z-debug-0.log The logfile contains:
|
FYI: this same error EMFILE: too many open files is happening in mid-January 2023 using npm 9.3.0. The last open was for a file at: |
same error, using 9.2.0 |
The only cure was the following sequence of steps:
I've learned the hard way to always |
npm -v: My error from log file: I tried deleting this file and it came up with "next" Also I tried rename to just "a" but still: thus the problem is not in the long name (in Windows there is a problem with this, sometimes), but in the number of open/cached files 🙂
|
Same issue here after updating to 9.6.0. |
still an issue for 9.6.1 |
I love frontend development |
Running into this too when running
|
Same hare for |
Same issue present on Jenkins with node v20.9.0 and npm v10.2.1
|
Same error for us. Is there anybody on the development team who care about this very critical bug? It seems a very basic problem and no one really care about it? |
Very annoying issue, forced to stay with npm@8.5.5 |
I'm experiencing the same bug with npm @10.2.3. The error appeared in building a docker container. Locally the build worked perfectly, but on GitHub Actions I get the same error. I my case I could solve this by setting the
Leaving the comment here, hoping to save someone a frustrating day of googling as I did. References: |
hey @robinvanderknaap , is that I am running into If I could flag to lastly here is a related issue I'm tracking on the AWS codebuild side: aws/aws-codebuild-docker-images#686 (comment) |
@skilbjo It's a flag in a GitHub Actions build file. But the flag is specifically set for the |
Builds keep failing with EMFILE errors, so follow the advice from a similar issue work around: npm/cli#4783 (comment) Signed-off-by: Scott Dickerson <sdickers@redhat.com>
Not sure if obvious but as a workaround you can of course use NPMv8 which doesn't suffer from this issue. Maybe okay enough for use in your CICD pipeline for now. npx -p npm@8 npm ci |
Builds keep failing with EMFILE errors, so follow the advice from a similar issue work around: npm/cli#4783 (comment) --------- Signed-off-by: Scott Dickerson <sdickers@redhat.com>
We were seeing this issue after migrating from Yarn v1 to npm v10. In our case, we'd been setting Updating this to |
TL;DR for those new to this issue, I believe this is the bug, which is present in NPM 9 but not in NPM 8:
For example on AWS Lambda, which can e.g. be used as a CI backend by AWS CodeBuild. AWS Lambda has a hard limit on open file descriptors of 1024 (see here). Since this also affects |
@ottokruse you perfectly highlight the bug I am having with npm + AWS Lambda backend for AWS CodeBuild (and all others who want to use npm + AWS CodeBuild) do you have a workaround you've found I can use in the buildspec when using npm + AWS Lambda backend for AWS Codebuild? mind pasting the full builspec if you have found a workaround? |
@skilbjo the only work around I'm aware of is to use NPM v8. So wherever you do |
It would be grateful if someone tried the change in #7631 to see if it solves the problem. Thank you in advance. |
This change solves npm#4783 <!-- What / Why --> <!-- Describe the request in detail. What it does and why it's being changed. --> During 'npm cache verify', currently all the cache files are open at the same time, which will bring EMFILE error in an environment that limit max open files. This change limits the concurrent open files in garbageCollect() with p-map module to avoid this problem. ## References Fixes npm#4783
<!-- What / Why --> <!-- Describe the request in detail. What it does and why it's being changed. --> During 'npm cache verify', currently all the cache files are open at the same time, which will bring EMFILE error in an environment that limit max open files. This change limits the concurrent open files in garbageCollect() with p-map module to avoid this problem. I first sent this pull request to npm/cli and it was merged, but realized that the original was in this repository. ## References npm/cli#7631 npm/cli#4783
|
Try npm v10.8.2 or later. |
Is there an existing issue for this?
This issue exists in the latest npm version
Current Behavior
Currently observing EMFILE errors when running
npm cache verify
using npm >=8.6.0 on our Jenkins agents (EC2 instances), despite increasingulimit -n
from 1024 to 8192. Cannot reproduce in the same exact instance when using npm 8.5.5.Please let me know what detail might be useful to help debug this! I've collected strace output and silly-level logging. If any of that is useful, I can attempt to sanitize it to share!
In the meantime, I'll likely move forward with removing the
npm cache verify
step from our Jenkins pipelines, and see if the other commands can succeed, although that will leave me with fewer warm fuzzies.Expected Behavior
Steps To Reproduce
npm cache verify
Environment
The text was updated successfully, but these errors were encountered: