Stat error on symbolic link during scanning

[bobot]

Expected Behavior

Not failing when a symlink pointing to a non existent file exists in the source directories

Actual Behavior

Fails during scanning

Reproduction

In the dune repository:

• ln -s non-existent a_file.txt

• make dev

  ocaml bootstrap.ml
  ocamlc -custom -w -24 -g -o .duneboot.exe -I boot unix.cma boot/libs.ml boot/duneboot.ml
  ./.duneboot.exe
  ./dune.exe build @install
  Error: stat: a_file.txt: No such file or directory
  make: *** [Makefile:49 : dev] Erreur 1
  

Specifications

• Version of dune: 01bcdbd
• Version of ocaml (output of ocamlc --version): 4.09.1
• Operating system (distribution and version): ubuntu 20.4

[bobot]

Another issue which should touch the same part of the code: on cygwin we have problem with one test file which uses a " in its name (for testing this edge case in frama-c) tests/syntax/foo".c

EDIT: cygwin with mingw

[aalekseyev]

@bobot, can you please paste an error message about the file with a ", if you have it?
(maybe it points us directly to the bug without having to experiment)

[bobot]

Unfortunately I just have Error: stat: tests/syntax/foo".c: No such file or directory . The build target was not depending on this file.

[nojb]

How did you create a file with an embedded double quote under Windows? According to https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file#naming-conventions it shouldn't be possible.

[aalekseyev]

About the broken symlink issue, it looks like the crash was introduced in #3570 by accident, by detaching the stat call from its error handling code.
It's now being fixed by #3675 "by accident" because of a not-entirely-correct merge. Hah!

[bobot]

How did you create a file with an embedded double quote under Windows?

Using VirtualBox, shared directory. I think it is seen by windows as a shared directory on the network.

In cygwin:

• ls lists it but I don't find how to access it.
• Windows explorer also list it, but can't open it
• Ocaml: Sys.readdir lists it, but accessing it return an Invalid argument instead of No such file or directory

# let c = open_in "z:\\frama-c\\tests\\syntax\\foo\".c";;
Exception: Sys_error "z:\\frama-c\\tests\\syntax\\foo\".c: Invalid argument".
#  let c = open_in "z:\\frama-c\\tests\\syntax\\bts0323.h";;
val c : in_channel = <abstr>
#  let c = open_in "z:\\frama-c\\tests\\syntax\\foo";;
Exception:
Sys_error "z:\\frama-c\\tests\\syntax\\foo: No such file or directory".

Perhaps the fix for the error handling code will fix that, and we can delay the error later if the file is needed. And so I will just disable the test on windows.

[aalekseyev]

@bobot, indeed I suspect the change in #3675 (just merged) may be enough to improve the behavior.
Can you try rebasing to see if it helps?

[aalekseyev]

We said during the meeting that we should probably make broken symlinks and special files have uniform behavior, but I'm not sure anymore.

• The historical behavior for broken symlinks seems fine, and we've just reverted the regression.
• The current behavior for special files was justified in Ignore special files (BLK, CHR, FIFO, SOCKET) #3570 and I don't have a good reason to disagree.

With that, I think we should just close the issue. @bobot, agreed?

[bobot]

The current behavior for special files was justified in #3570 and I don't have a good reason to disagree.

Can you precise or point me to the justification?

We can close this issue since the failure is fixed (in windows case also). However during the meeting I think we discussed to replace the current behavior, ignoring the file, by an error delayed for when a rule use the file is executed, no? It can go to a postponed issue.

[aalekseyev]

Can you precise or point me to the justification?

Sorry, that was not really in the PR, but in the issues: #3546 #3124. I can only see voices for ignoring these special files.

I think we discussed to replace the current behavior, ignoring the file, by an error delayed

Indeed we did. I don't know if I still agree, though.
For a broken symlink one can make an argument that the user probably meant to refer to the link destination, so we want to complain about it missing.
For a socket or a fifo it's hard to imagine user actually wanting to use it in the build rules, so it's plausible to me that keeping the two behaviors different is useful.