[BUG] npm ci fails with EPERM/unlink error on windows #568
Comments
|
Seeing the same thing: C:\ws\sol>npm ci
npm WARN prepare removing existing node_modules/ before installation
npm ERR! path C:\ws\sol\node_modules\tape
npm ERR! code EPERM
npm ERR! errno -4048
npm ERR! syscall unlink
npm ERR! Error: EPERM: operation not permitted, unlink 'C:\ws\sol\node_modules\tape'
npm ERR! { [Error: EPERM: operation not permitted, unlink 'C:\ws\sol\node_modules\tape']
npm ERR! cause:
npm ERR! { Error: EPERM: operation not permitted, unlink 'C:\ws\sol\node_modules\tape'
npm ERR! errno: -4048,
npm ERR! code: 'EPERM',
npm ERR! syscall: 'unlink',
npm ERR! path: 'C:\\ws\\sol\\node_modules\\tape' },
npm ERR! stack:
npm ERR! 'Error: EPERM: operation not permitted, unlink \'C:\\ws\\sol\\node_modules\\tape\'',
npm ERR! errno: -4048,
npm ERR! code: 'EPERM',
npm ERR! syscall: 'unlink',
npm ERR! path: 'C:\\ws\\sol\\node_modules\\tape' }
npm ERR!
npm ERR! The operation was rejected by your operating system.
npm ERR! It's possible that the file was already in use (by a text editor or antivirus),
npm ERR! or that you lack permissions to access it.
npm ERR!
npm ERR! If you believe this might be a permissions issue, please double-check the
npm ERR! permissions of the file and its containing directories, or try running
npm ERR! the command again as root/Administrator (though this is not recommended).
npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\stijn\AppData\Roaming\npm-cache\_logs\2019-12-16T12_03_09_625Z-debug.log |
|
If I run |
Hi @Download, |
darcyclarke
added
platform:windows
darcyclarke
added
the
Release 6.x
|
@momesana I am guessing since you started working around it with yarn you never got to the bottom of this? I am seeing this happen once in a while in our CI builds, always with the same package. I am at a loss as to why it is happening since we are killing hanging node processes. |
|
@momesana Yes when I encounter it again I will do this. |
|
My team actually put in a retry loop on npm ci because of this issue, it only gives up after 5 retries. Made our problems go away, but it does mean it takes extra time when it fails. |
|
For me the problem was solved running terminal/vscode as admin when I try to run |
|
npm If your bug is preproducible on If your issue was a feature request, please consider opening a new RRFC or RFC. If your issue was a question or other idea that was not CLI-specific, consider opening a discussion on our feedback repo Closing: This is an automated message. |
1 similar comment
|
npm If your bug is preproducible on If your issue was a feature request, please consider opening a new RRFC or RFC. If your issue was a question or other idea that was not CLI-specific, consider opening a discussion on our feedback repo Closing: This is an automated message. |
yep, we never had issues like that with yarn though |
We have a moderately large web-application and installing the dependencies sometimes leads to EPERM error messages that npm can't recover from. This happens on multiple systems but can't reliably be reproduced. If the problem occurs it seems to persist until the cache is cleared via
npm cache clean --forceand the next run may or may not yield the desired result (that is: a successful run ofnpm install). The problem only happens on Windows, neither Linux nor Mac-OS are affected and it only appears to happen with large dependencies (in our case antd, monaco-editor and react-icons).I couldn't reproduce the problem when fetching the files from npmjs directly. It is noteworthy that we use a private artifactory repository (to shield us from issues like left-pad suddenly being pulled from npmjs). There is a proxy that hides artifactory and it seems like the connection to the proxy may be a bit flaky at times (that's merely an educated guess though) so it might be the case that the connection/download is reset while the fetch is ongoing. Let me show the error message that npm spits out when things go south:
or on another colleagues system
or during another run
and again in another run about 4 hours and a gazillion attempts later:
Surprisingly the sha1 hash for antd mentioned in the latest instance and flagged as corrupt is in fact correct as can be calculated via sha1sum on the downloaded package from artifactory and from npmjs:
Now in a node interactive REPL, I will get the exact same value (i. e. KEy/g89atsJt/7YGZXbyAupgr1A=) if I convert the above checksum to base64 representation:
The problem has never occurred on our CI system (Jenkins) which is run on the same server rack as artifactory. Nor does it occur if we bypass artifactory altogether and access npmjs directly (as I've already mentioned above). We have a bunch of developers working on linux/macos and about 6-7 developers running on windows. Almost every windows user in the team has been affected at least once in the course of the last 2 months (after we introduced npm into the build pipeline). It's a showstopper par excellence to say the least.
Searching the internet yields many bug-reports that describe the same problem but the proposed solutions at best amount to voodoo programming.
We took a closer look at the EPERM issue and it turns out that the Node process leaks the file-handle (pay attention to the entry with the label DELETE_PENDING):
The EPERM error presumably results from npm failing to close the file-handle in time. The file-handle is only closed once Node.exe itself terminates. In other words: the process blocking the file and the process trying to delete it (more preceisely unlink it) are identical (as evident by the fact that they share the same PID).
Our theory so far is the following:
For some reason fetching large files occasionally fails. If that happens on linux, there is a file-handle leak but that doesn't prevent npm from trying again and finally successfully downloading the file. The file-handle leak on windows however prevents npm from deleting/overwriting the file in question in its further attempts. This ultimately causes
npm ci (or install)to fail.Maybe there is a missing
fs.closeSync(fd)in the respective npm code for fetching packages in case of an exception.We tried this with different versions of node/npm btw. We had successfull runs and failures with all these versions. We cleared the cache using
npm cache clean --forcebefore each run. In particular we tested the following versions of node (and the respective npm version packaged along with it):Anyways, we had to get things working so we switched over to yarn and it works like a charm. Unfortunately that also means one further dependency to manage so I personally would prefer a less brittle npm. Maybe that helps you guys investigate and work out the issue.
The text was updated successfully, but these errors were encountered: