Dune clean fails on USB stick

[yannl35133]

When I try to run dune clean on my project which is located on a USB stick, the call to rmdir is refused as Directory not empty.
It works perfectly fine if I copy it on the computer.

How to reproduce:
In an empty folder, on the command line
dune build
dune clean
Error: rmdir: _build: Directory not empty

I'm on Kubuntu 19.10,
dune 2.0.0 from opam (BTW, is it normal that dune --version echoes n/a ?)

[ghost]

Wait, building on a usb stick???

More seriously, I can't think of what in Dune would cause this behaviour. dune clean is barely more than a glorified rm -rf _build. When the call to rmdir fails, is the directory really empty?

[ghost]

BTW, is it normal that dune --verision echoes n/a ?

No, that's fixed in master.

[nojb]

I've seen issues with dune clean on Windows complaining that _build/log could not be unlinked. May be related. Don't know how to reproduce it yet. I think @ejgallego also saw this problem.

[ghost]

What are the file systems in each cases? It could also be an issue with dune holding on to an open file inside _build such as _build/log, though I believe we don't open the log file for dune clean.

[nojb]

In my case, it is ntfs.

[yannl35133]

NTFS also (for the key).
As for dune holding on to an open file, why would it do it on a stick but not on my computer ?
Regarding the USB stick, I'm still a student and I have not learned how to use git, I code my (relatively small) project both on my computer and at school, and I cannot control much there.

[nojb]

What are the file systems in each cases? It could also be an issue with dune holding on to an open file inside _build such as _build/log, though I believe we don't open the log file for dune clean.

I did a quick experiment and the log is indeed opened when doing dune clean (sorry, don't have time to investigate further at the moment). Callstack:

$ ./dune.exe clean
init
Raised by primitive operation at file "src/stdune\\log.ml", line 23, characters 42-69
Called from file "bin\\common.ml", line 73, characters 2-13
Called from file "bin\\common.ml" (inlined), line 97, characters 2-29
Called from file "bin\\main.ml", line 92, characters 4-40
Called from file "vendor/cmdliner/src\\cmdliner_term.ml", line 25, characters 19-24
Called from file "vendor/cmdliner/src\\cmdliner.ml", line 146, characters 9-16
Called from file "vendor/cmdliner/src\\cmdliner.ml", line 176, characters 18-36
Called from file "vendor/cmdliner/src\\cmdliner.ml", line 312, characters 20-46
Called from file "bin\\main.ml", line 193, characters 10-51
Error: unlink: _build\log: Permission denied

[ejgallego]

Seems a duplicate of #2921

[yannl35133]

I just checked, the problem only happens on my NTFS key. Should I change title to NTFS filesystem?

[ghost]

@yannl35133 yeah, that seems to be NTFS related.

Regarding the file being kept open, deleting an open file is fine most of the time, but maybe it's not on NTFS.

[nojb]

In order to be able to delete an open file under Windows, you need to open it using the FILE_SHARE_DELETE flag (O_SHARE_DELETE flag in Unix).

[ghost]

Yeah, we use that flag in a few cases. Maybe we should use it more systematically. In any case, it doesn't seem useful to open the log file in dune clean so we might as well not do it.

[yannl35133]

Thanks for this. When can we expect this to be included in opam's version?

[ghost]

We started the release of Dune 2.1.0, so it should be in opam soon.