-
-
Notifications
You must be signed in to change notification settings - Fork 252
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Doesn't work #25
Comments
Can you provide more details? There's something in there that it's failing to delete. What files are in there? How are you creating this dir? |
Seems that it was creating files while rimraf was trying to delete at the same time |
@isaacs: could you please check out the log pasted in the phantomjs ticket and tell me if you need more info in order to investigate this issue ? |
If you're creating files while trying to rimraf the dir, then yes, that will definitely not work. That will also confuse the builtin |
@mlarcher What's happening in the phantomjs ticket's npm log looks like the phantomjs install script fails (exits non-zero), but is still doing stuff (perhaps because it shelled out to windows unzip, which does not stop writing files immediately when the process exit event happens.) Do not fetch stuff at install time. Fetch it when you need it. The correct behavior in this case is for rimraf to surface the error. Rimraf is not, and cannot ever be, an atomic FS operation. |
While I agree that rimraf should not work in this case maybe a more meaningful error would be beneficial. |
In case other people are having this issue its most likely a virus scanner scanning the folder where the file operations are happening. This will make the files in use during the deletion. In this specific case it was caused by ESET Anti-Virus. SOLUTION: Ignore the following paths Make sure to replace the username with yours obviously. I have a feeling this is why the issue is so intermittent for most people and unconfirmed. This should also help fix Medium/phantomjs#108 |
What error code gets raised with the anti-virus program thwarts rimraf? There's some special handling for EBUSY for exactly that purpose. |
This is the error code that gets thrown while the anti-virus is scanning the files in the directory. I assume what is happening here is that the phantomjs install is trying to remove the directory while the Anti-Virus still has file descriptors open in the folder. Not sure how you would formulate a workaround for that? Maybe some sort of blocking?
|
ENOTEMPTY would happen when rmdir'ing the directory. That would indicate that either:
In the first case, it would fail with the unlink error, so it's not that. If the directory was locked (because, as you suggest, a file descriptor was open) then we'd get an If it IS an If there is some other process writing to the dir while we're trying to delete it, then that is a valid error, and rimraf is doing the correct thing by failing in this way. |
Actually I just grabbed that trace off the other thread. I can try to get this error reproducible we just got around it by doing this last night. So I figured it might help some of the other people that have been having the issue intermittently. I was going to note that this feels a lot like a race condition which may be why we are not seeing the right error codes. However I have not had a look at any of the underlying code so I am unsure how the process works. I should mention that when I got the error I am on the latest version of node 0.10.21, I just didnt have a stack trace handy to show you. |
It seems like this error only happens in connection to PhantomJS, is that correct? I'd look through their build script, and see if it's trying to write to directors it shouldn't, or continuing to do things after exiting with a non-zero status. |
I dont believe so I have followed a couple other threads into this one not all related to phantom. I am currently spinning up a VM to make this reproduce-able. The references above for say grunt-contrib-clean seems to be running into it as well with the same intermittent symptoms so I believe they're related. |
@isaacs Okay, got this in a nice reproducible environment. Setup
Instructions
Output
This is in a fresh Windows 7 VM and I followed this guide for setting up my environment: https://gist.github.com/nullivex/7115612 If you would like access to the VM message me off list. |
@nullivex Take a look at that stack trace. The problem is in the karma/phantomjs install script. I'll take a look at this when I get a chance, but my best guess is that it's trying to delete a dir that's currently being written to. |
Yeah that's what we figured as well. Again I'm not sure where the fix would I believe what is happening is that when phantomjs or any other package I'd be happy to try to work out a pull request against phantomjs if we can However since AV programs are really common for windows users it is worth On Wednesday, November 13, 2013, Isaac Z. Schlueter wrote:
eSited LLC |
There's a similar issue here: To reproduce:
...................................................................................rm: could not remove directory (code ENOTEMPTY): Z:\cordo rm: could not remove directory (code ENOTEMPTY): Z:\cordova-cli-master\temp/.cordova/hooks/before_build rm: could not remove directory (code ENOTEMPTY): Z:\cordova-cli-master\temp/.cordova/hooks/before_build rm: could not remove directory (code ENOTEMPTY): Z:\cordova-cli-master\temp/.cordova/hooks/before_build .{ [Error: EPERM, operation not permitted 'Z:\cordova-cli-master\temp.cordova\hooks\before_build\fail.bat'] The tests work fine on linux, I traced it down to code that tries to recursively remove a directory in 'shelljs'. What happens is: a) File is deleted (before_build\fail.bat) b) Removal of directory fails with ENOTEMPTY At the last failure: fs.readdirSync(dir) reports the file as existing! so fs.lstatSync() throws [Error: EPERM, operation not permitted This points to node on Windows either: I think it's more (b) since I tried adding sleep(5) and doesn't change the fact the directory entry still thinks the file exists. |
I give up, best next guess is a bug in libuv: I'm not sure what FindFirstFileW and FindNextFileW are doing, from what I can tell, the win32 api is FindFirstFile, FindNextFile? |
@jbondc Do you have any AV installed? Or any other program that actively monitors certain file types like executables (.bat, .exe, etc). I have only seen this occur when someone has an AV that scans executable's. |
On Windows 8.1, if I disable:
I get the same result. Here's what the order of operations looks like: |
Strange everything for me seems to work fine on windows 8 as long as I dont have an AV enabled. And the fact that its happening with a .bat leads me to believe that maybe that bat file hadnt finished executing or something is scanning it not sure. But overall @isaacs I guess we really need some way to block until all the file handles have free'd up for things to work as expected? Just brainstorming. |
I get the same result in Windows 7, really odd. Maybe some answers here: But using a FAT filesystem, I get the same result. unlink on UNIX: unlink for windows: The unlink for windows is more complex, maybe a timing/thread issue there. I'm just guessing. |
Maybe relevant for 'FindFileFirst': http://msdn.microsoft.com/en-us/library/windows/desktop/aa364952%28v=vs.85%29.aspx |
None of the above. ENOTEMPTY is in fact a condition similar to EBUSY that can happen on directories. When a file is deleted while another handle or another process still has the file open, the file just gets 'marked for deletion'. The directory entry remains until the file is actually deleted, which happens when the last handle to it is closed. This is unlike unix where the inode has the same semantics of a file on windows (it isn't actually deleted until all FDs are closed), but the directory entry can be removed immediately. Note that this case is also distinct from the EBUSY case - that happens when deletion itself is prevented by another process that did not set the FILE_SHARE_DELETE flag when opening the file. But it's likely that virus scanners set the flag when opening files to be scanned, so antivirus software might indeed trigger this. (and yes, it is dumb) |
But which handle or process still has the file open? It appears that only node is accessing those files. Should CloseHandle() be before SET_REQ_SUCCESS() ? https://github.com/joyent/libuv/blob/master/src/win/fs.c#L693 |
The virus scanner, that starts scanning the file in response to it being created. It probably won't show up so near to the node action. But I can't look through that file and have only your screen shot to go by.
no. throwing shit at the wall doesn't really help. |
Thanks for the feedback, @piscisaureus.
So, for rimraf's purposes, much as it pains me, should we just keep trying for a while like we do with EBUSY? Of course, I wouldn't ever want to do such a thing on Unix, but if it would work around a windows-specific issue, then that's probably fine. |
Right on. Unfortunately there is no good way to tell this case apart of a real ENOTEMPTY condition, and it's hard to tell what is an appropriate time to retry. Not anything that isn't very complex atleast. So yeah, just keep trying for a while...
|
Fair enough, but I'm pretty sure this case has nothing to do with the virus scanner: I put up the log files here: Can also export in csv or xml. |
@jbondc It seems that in this case node has the file open when it gets deleted. So you get the same effect. |
* Delete folder that works on windows (isaacs/rimraf#25) * . Co-authored-by: Håkansson Lars IT-konsulter <lars.hakansson@guest.skogsstyrelsen.se>
The text was updated successfully, but these errors were encountered: