Change delayed delete mechanism to prevent cross-drive mv failure for in-use DLL

[Liozou]

On Windows, during precompilation of package A, a DLL file is generated and may replace the existing one. If A is already loaded in the julia session however, the corresponding soon-to-be-obsolete DLL cannot be removed while julia is running. Currently, this problem is solved by moving the old DLL in a special "julia_delayed_deletes" folder, which will be cleaned in a later julia session by Pkg.gc().
However, moving an in-use DLL is only permitted on the same drive, and the "julia_delayed_deletes" is located in the tempdir, a.k.a. on a fixed drive, say C:. Thus, if the DEPOT_PATH points to a ".julia" in another drive, say D:, any precompilation occuring on an already loaded package will fail. This is what happens in #59589, actually resulting in an infinite loop that bloats up both memory and disk. @staticfloat had actually predicted that such an issue could occur in #53456 (comment).

This PR fixes #59589 by changing the "delayed deleting" mechanism: instead of moving the old DLLs to a fixed folder, they are renamed in their initial folder and recorded in a list stored at a fixed location. Upon Pkg.gc(), the listed obsolete files will be deleted (JuliaLang/Pkg.jl#4392).

It also prevents any similar infinite loops by cutting the rm -> mv -> rm cycle into rm -> rename. I also removed the private argument allow_delayed_delete from rm, which is only called in by Pkg but does not appear to be useful.

EDIT: current state is #59635 (comment)

[Liozou]

The CI failures come from the loading.jl test that attempt to remove the depot at the end of the test. This fails, because the DLL is still present (albeit renamed) within... So I'm reverting the design of this PR to something closer to the current state. In the current version of the PR:

• When an in-use DLL is being rm-ed, rename it to a unique name in the same folder.
• Move it to a special directory is either joinpath(tempdir(), "julia_delayed_deletes") if it is on the same drive as the DLL, else joinpath(drive, "julia_delayed_deletes") with the correct drive. This way, the depot can be removed during the same julia session. (removed it 8badff9)
• The path to the now obsolete DLL is written in a file stored in joinpath(tempdir(), "julia_delayed_deletes_ref"). This file has a unique name generated by mktemp, so that multiple julia processes can attempt to rm the same in-use DLL without corruption.
• Upon Pkg.gc(), the files in the "julia_delayed_deletes_ref" folder are read, each containing a single line with the path to an obsolete DLL to remove. If all DLLs in a "julia_delayed_deletes" folder have been removed, the folder itself is removed; if all files referenced in the "julia_delayed_deletes_ref" folder are removed, that folder is also removed.
• Like currently, the rm called by Pkg.gc() in this particular setting is the only one that unsets the allow_delayed_delete kwarg. By default, rm attempts this strategy upon trying to delete an in-use DLL.

[vtjnash]

Thanks for working on this!

[Liozou]

Thanks again for the reviews! In the current solution, I partially reverted the last changes so that the renamed obsolete DLLs remain in their folder. This required modifying the loadings.jl tests: instead of making the test depots with mktempdir(), which inevitably fails at cleanup because it encounters in-use DLLs, they are now made with mktempdir(; cleanup=false). Then, in an atexit hook, a special julia process is spawned, whose sole job is to wait for the test process to die, and then to cleanup these depots. This only affects Windows, nothing is changed for the other systems.

[vtjnash]

The special julia process to be spawned is very clever. Nice job. We might even want to move this code into temp_cleanup_purge_all if the TEMP_CLEANUP set is non-empty after attempting in-process cleanup?

[Liozou]

Thanks a lot for the review again, the eof(stdin) strategy is smart! I'm not familiar with the lower-level Pipe code and the solution you proposed errors (since Base.dup expects a RawFD) so I patched something which I believe does what is expected, but an external check is most welcome!
I also moved this logic into a dedicated function of Base.Filesystem to be called exactly once atexit (to avoid spawning multiple such processes).