Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Symlinks on the mounted Windows directories are not compatible with native. #353

Closed
tinysun212 opened this issue May 13, 2016 · 71 comments
Closed

Comments

@tinysun212
Copy link

tinysun212 commented May 13, 2016

On build 14341,

I created a symbolic link to a file on a mounted Windows directory in BASH.

tinysun@TEST-PC:/mnt/c/Temp$ ln -s origfile.txt symlink.txt
tinysun@TEST-PC:/mnt/c/Temp$ ls -l
total 0
-rwxrwxrwx 1 root root 39 May  4 01:48 origfile.txt
lrwxrwxrwx 1 root root 12 May 13 02:58 symlink.txt -> origfile.txt
tinysun@TEST-PC:/mnt/c/Temp$

In the command prompt CMD.exe, the file appeared as a JUNCTION and can not be accessed.

C:\Temp>dir
 Volume in drive C has no label.
 Volume Serial Number is EAE0-0FD9

 Directory of C:\Temp

05/13/2016  11:58 AM    <DIR>          .
05/13/2016  11:58 AM    <DIR>          ..
05/04/2016  10:48 AM                39 origfile.txt
05/13/2016  11:58 AM    <JUNCTION>     symlink.txt [...]
               2 File(s)             51 bytes
               2 Dir(s)  194,170,494,976 bytes free

C:\Temp>type symlink.txt
The file cannot be accessed by the system.

C:\Temp>

It should be displayed as follows.

05/13/2016  11:58 AM    <SYMLINK>     symlink.txt [origfile.txt]

I think the 'JUNCTION' can be used when the link references a directory not a file.

@vyv03354
Copy link

Looks like cmd.exe does not (yet?) know WSL symlinks and it treats every unknown reparse point as a junction. But actually this is not a junction. It is a new type of reparse point whose reparse tag is 0xA000001D. Hopefully it will be documented in the future Windows 10 Anniversary SDK.

Unlike Windows symlinks/junctions, no filesystem/filter drivers resolve WSL symlinks automatically. I guess this is an intentional decision to prevent Windows apps from bypassing SeCreateSymbolicLinkPrivilege by using WSL symlinks.

@russalex
Copy link
Contributor

I do owe some documentation on this one. Should get that up in the next few days.

@vyv03354 is correct that the primary reason we are not adding symlinks to Windows is because of the privilege required. As of now, symlinks created in WSL are only usable within WSL. This is something we will be looking at in later releases.

@ned14
Copy link

ned14 commented Jul 14, 2016

@russalex Can it not be the case that if the user running WSL has the SeCreateSymbolicLinkPrivilege that WSL creates NTFS format symbolic links instead of a custom link type?

@russalex
Copy link
Contributor

Sven covers this question a little bit in his video on the WSL File system Blog post. There have been some discussions on doing this in the next release, but there are some questions and behavior that we would need to work out first.

I know there is a user voice page on this very request. Right now it only has 3 votes.

@rickdoesdev
Copy link

rickdoesdev commented Aug 3, 2016

Personally, coming from Linux and OSX the entire SeCreateSymbolicLinkPrivilege situation is one that has repeatedly caused me headache and grief. I was hoping against hope that BoW would finally fix the issue but, alas, it seems instead of making symlinks work properly in user space, MS has decided to just make a new standard? I can't follow any of the symlinks created in the linux subsystem even in windows exporer, much less in my IDE or other windows host apps. This makes me sad. :(

@ned14
Copy link

ned14 commented Aug 3, 2016

@rcuddy It is very straightforward to award any user group the SeCreateSymbolicLinkPrivilege and then you can create symbolic links to your heart's content! Note that any user with the ability to escalate to administrator still cannot create links, so you'll need to make your user an ordinary user, not administrator capable user for this to work.

@rickdoesdev
Copy link

@ned14 Sure, I can add that privilege, but as you've pointed out, I then lose the ability for that user to escalate privileges. That doesn't seem a fair trade. ;)

@CherryDT
Copy link

CherryDT commented Apr 5, 2017

Is there any workaround to this issue? I now ran into a problem there when I tried to do a backup of a folder using robocopy. It's a development folder and npm (under WSL) created a few symlinks from node_modules/.bin/xxx to something, and now robocopy chokes on these "files" because it perceives it ias "access denied" and keeps retrying - which is something I would like to have for files actually looked or so, but not for "broken" symlinks, so I would prefer not to have to disable the error handling in robocopy...

@JFLarvoire
Copy link

Repost of the suggestion I just posted on Issue #1475:

lxss symlinks also break my win32 backup tools.
+1 for the urgent need to unify lxss symlinks and win32 symlinks.

The tools on both sides of the programming world should be able to view and copy directory trees created on the other side, with all links in the copy usable by both sides as well.
=> The win32 and Linux subsystems must hide to user-mode programs what target path is really written on disk, and instead return to them converted targets they can understand.
For relative links this is pretty straightforward: Simply change all slahes to backslashes for the Win32 SS, or forward slahes for the Linux SS.
For absolute links, the problem is that the targets don't refer to the same root. Yet that root can be identified by the kind of slash written by the link creator:

  • If the link contains back slashes, it was created by a Win32 process relative to a Windows volume.
  • If it contains forward slashes, it was created by a Linux process, relative to the Linux FS root.

Then the subsystems can convert the link targets to absolute paths formatted as their respective user-mode programs expect.

@noinkling
Copy link

noinkling commented Apr 13, 2017

Would also like to see this. I've tried it both ways (link and target both on Windows file system):

  1. Creating a symlink via ln -s from Bash/WSL.
  2. Creating a symlink via mklink in Windows cmd.

With approach 1 I observe the behaviour mentioned here. Also the link will simply not appear in my editor on the Windows side (VS Code). In Windows Explorer it will appear but give an error upon trying to open it. However it works with everything on the WSL side.

With approach 2 everything works on the Windows side (as you would expect), but the symlink isn't visible and doesn't work on the WSL side. However, subsequently trying approach 1 results in this error: ln: failed to create symbolic link 'link.txt': File exists, and doing an rm will actually remove it, so it does exist at some level in WSL.

So to sum things up, a symlink created on either side cannot be effectively utilised by the other side, kinda breaking the file system interoperability promise. I would expect DriveFS to correctly handle symlinks created with mklink at least.

Thanks for your consideration.

@chx
Copy link

chx commented May 3, 2017

Yes, this a big mess especially when git gets involved....

@martixy
Copy link

martixy commented Jul 3, 2017

Bash also doesn't see Windows symlinks(or at least ls refuses to list them).
Though you can technically cd inside(it doesn't say No such file or directory as it would with an actual non-existing file/dir). None of the contents are there though.

And yea... it gets worse with git.

@rhyek
Copy link

rhyek commented Jul 3, 2017

I have a mess of utilities installed redundantly on Windows and Ubuntu due to this issue. Hopefully it will get resolved at some point.

@nick4fake nick4fake mentioned this issue Jul 20, 2017
@CyrusNajmabadi
Copy link

CyrusNajmabadi commented Sep 13, 2017

I would also like this. Currently on the Bash side we're using Yarn for dependency management. We're sharing files between Windows and Bash by using /mnt/c/... . Yarn ends up using symlinks and works well if you're entirely in the Bash side. However, these symlinks are not seen on the windows side properly, which breaks development tools over there.

Note: if i manually make a junction or symlink on the Windows side (i use the Link Shell extension here: http://schinagl.priv.at/nt/hardlinkshellext/linkshellextension.html) then the Bash side sees things fine. So there does appear to be interoperable symlinks, it's just that "ln -s" doesn't make them inside Bash. Fixing this would be a godsend. Thanks!

Note: I'm on 16251.

@jordwalke
Copy link

I was sad to hear that you could not create/edit files in the wsl system from Windows. The workaround was said to be that we should create files (for example, from git/npm/yarn) on the Windows side of things, and then access those files from Windows and WSL. Well when doing what is suggested, you run into this issue with symlinks very quickly. The first thing I tried was log into bash, cd into my windows home directory, do a git checkout, then set up some symlinks (just like with what yarn/npm) would do - so that I could edit these files from windows or WSL. But when trying to access the files from Windows, nothing worked because symlinks weren't respected on Windows' side.

So accessing WSL files from Windows doesn't work (okay, I was sad but got over it), but then I discovered that in practice, the encouraged workflow (access Windows files from both Windows and WSL) doesn't work in practice either because many of the kinds of files/repos/projects you want to setup on the Windows mount, require symlinks to work correctly.

Do the BashOnWindows developers agree that this is a major issue?

Thank you very much for your investment in WSL - I'm very excited about the long term potential of this project, and I consider this to be the final blocker to actually using WSL instead of Cygwin/MSys for day to day project development.

@ned14
Copy link

ned14 commented Sep 22, 2017

@jordwalke The way I work around that is to have separate build directories for each source project, one for WSL, one for Windows.

@jordwalke
Copy link

@ned14 yeah, I imagine something like that would be possible, but at that point, I would just rather consider rsyncing two directories between the two systems which pretty much treats WSL like a VM but faster. I just think WSL could be so much more than that if Windows would just support Linux created symlinks from the windows drive to the windows drive. (I'm not even asking for the ability for Windows to create/touch Linux files - it should be fine as long as we can create files on the Windows drive using the Linux command line (including symlinks) which Windows will respect).

@sunilmut
Copy link
Member

@jordwalke - @SvenGroot is working on unifying the symlinks story between WSL and Windows so that WSL will create symlinks in a way that Windows can understand them.

@chx
Copy link

chx commented Sep 22, 2017

So glad to hear that. Are you addressing the absymally slow file performance as well :) ?

@ned14
Copy link

ned14 commented Sep 22, 2017

Are you addressing the absymally slow file performance as well :) ?

There's only so much they can do about that. On native Windows, opening a file is approx 15x slower than Linux. 4Kb i/o is approx 3x slower, and so on. Those figures are for warm cached filesystem.

Now, WSL is many times slower again, but ultimately there is only so much they can do. You will have to accept that filesystem heavy Linux programs will always run under WSL at a fraction of Linux.

@SvenGroot
Copy link
Member

File system performance is always a priority for us, and we are working on several ways to improve this. We've already made significant strides in RS3 (the upcoming Fall Creator's Update), and are continuing to work to improve this further.

There are limits to what we can do due to requirements of application and file system filter compatibility, but we're very much aware that file system performance is a pain point for WSL (and Windows as a whole) and are working to improve the situation.

@JFLarvoire
Copy link

@SvenGroot - Thanks for addressing this issue. If you're looking for a beta tester, I volunteer.
Or you might try it with one of my directory management tools, such as dirc.exe (A directory comparison tool), available in SysTools.zip there: https://github.com/JFLarvoire/SysToolsLib/releases
Like cmd.exe itself, the latest version of dirc.exe detects LXSS links, but can't display their target.

@jordwalke
Copy link

I'm very much looking forward to this upcoming feature. When does the next Creators update get released?

@noinkling
Copy link

noinkling commented Oct 19, 2017

Update for anyone following this issue: I installed the Fall Creator's Update, and it looks like symlinks created from the Windows side (e.g. using mklink) now work properly in WSL.

Links created on the WSL side (e.g. using ln -s) still aren't properly recognized by Windows.

If you have stuff that relies on symlinks, it may be worth creating them or "initializing" things (e.g. git cloning) on the Windows side if possible (if you want interoperability).

@ckapilla
Copy link

ckapilla commented May 28, 2018

Lucky me, I've been waiting to really use WSL until today -- thanks @tinysun212 for getting this ball rolling two years ago!

@mrsaraiva
Copy link

I'm running version 1803 (build 17134.191), but symlinks to files created on WSL are still being treated as directory junctions, making it inaccessible.

@FWest98
Copy link

FWest98 commented Nov 21, 2018

Thanks for the work you put in the symlinks, @SvenGroot . I do actually have a question about the requirements you posted, in order for WSL to make a Windows symlink:

The target is reachable from Windows (which means the link is relative and does not cross any mount points).
The target exists (so we can determine if it's a file or a directory).
The user has sufficient privileges to create symlinks (which currently requires elevation unless Developer Mode is turned on).

Today I noticed a strange effect with VirtualBox and its shared folders. I can allow VB to create symlinks as well, for example by running it elevated (or enabling developer mode). Then VB does allow me to create any symlink I want inside the Linux guest, also to non-Windows locations. For example:
VB: ln -s /proc test
Then in VB it shows up as a normal symlink (lrwxrwxrwx 1 vagrant vagrant 0 Nov 21 16:26 test -> /proc), and it even maps to Windows without problems (21-11-2018 17:26 <SYMLINK> test [\proc]), to my great surprise. Of course I cannot open the symlink or anything like that, but it is apparently stored in Windows. Consequently, this symlink works without any problems in WSL where it actually maps to /proc.

I think in practice this behaviour (making symlinks even though the requirements are not met) has the same effect as making the symlink windows-incompatible as is done in WSL now: Windows cannot do anything with it, but VB/WSL can. On the other hand, if I were to create this symlink in WSL currently, I cannot access it in VirtualBox, which has actually broken some things for me (the links were incorrectly read by VB).

So my question then is: Is VirtualBox doing something tricky here? I would assume the Windows API would also enforce these requirements, but apparently not? Is this behaviour desirable or potentially harmful? Would it be an idea to adopt this behaviour in WSL as well, to provide compatibility between VB and WSL? (or is it better/easier to fix VB to support the reparse tag used by WSL?) Or for that matter, not just VB but I can also think of NFS shares and many other things.

@gabefair
Copy link

Right, it looks like the fix for WSL to read directory junctions and symbolic links created in windows is no longer present in Windows 1809.
image

@pgl
Copy link

pgl commented Mar 28, 2019

Can this issue be reopened?

@ghost
Copy link

ghost commented Mar 28, 2019

I know I'm in unsupported territory with the following:
I have no problems running the XFCE4 Desktop (with VcXsrv as my Windows-based X server), if I take my home directory on a VolFs partition. However, I have been playing with a home directory on a DrvFs partition (so I can share it between different WSL Distros, and have it backed up: my new home directory is actually on my OneDrive).
Everything I do works smoothly, which is good. Until I try to start my XFCE4 Desktop: That locks up when I try to change configurations and preferences...

Looking to find the culprit, I found this difference between a xfce4 session on /home/USER, and /mnt/h/USER:
Failed to symlink /mnt/h/USER/.config/pulse/2f0e63ef2f494c5b8971513575f340c0-runtime to /tmp/pulse-FebW397VI5uG: Operation not permitted

It looks like the symlink from DrvFs to VolFs is not permitted?

I am using Windows 10 Version 1809, and this is with a "fresh" copy of Ubuntu 18.04LTS....

I'm not sure if this relates to the topic in this threat (where the stated problem is that the Windows symlink is not compatible with the WSL symlink)... But it feels to me it is all part of the same underlying problem...

Since this issue 353 is closed, must I assume my problem is actually supposed to be resolved?

@therealkenc
Copy link
Collaborator

It looks like the symlink from DrvFs to VolFs is not permitted?

Should be, if you've got Windows symlinks enabled. But you've invited OneDrive to the party, that adds a variable. Try the same scenario with a DrvFS mount that is NTFS not OneDrive, and see if it still fails. You'll need WSL metadata enabled, and I take as a given WSL metadata isn't supported by OneDrive (or, say, FAT32). I am not sure whether Windows symlinks are supported by OneDrive or not (never looked). If they are not that would be a OneDrive ask not a WSL ask. But given the base problem (EPERM) it looks like metadata (*NIX file mode) not symlinks.

@ghost
Copy link

ghost commented Mar 29, 2019

Thank you ! I took OneDrive out of the equation (although I think it just plain NTFS monitored by sync)...
Below is my simple experiment, that still gives me an error trying to create a Windows Simlink (from within WLS):

This is my /etc/wsl.conf:

[automount]
root       = /mnt/
mountFsTab = true
enabled    = true
options    = "metadata,uid=1000,gid=1000,umask=022,fmask=011"

This is my /etc/fstab:

LABEL=cloudimg-rootfs   /        ext4   defaults        0 0
\\MACHINE\C$\Temp /mnt/h drvfs  metadata,uid=1000,gid=1000,umask=022,fmask=011
  • On Windows 10, I created C:\Temp\windows.txt
  • On WSL, I created /tmp/wsl.txt

The result of WSL 'mount -l':

\\MACHINE\C$\Temp on /mnt/h type drvfs (rw,relatime,uid=1000,gid=1000,umask=22,fmask=11,case=off)
C: on /mnt/c type drvfs (rw,noatime,uid=1000,gid=1000,umask=22,fmask=11,metadata,case=off)
D: on /mnt/d type drvfs (rw,noatime,uid=1000,gid=1000,umask=22,fmask=11,metadata,case=off)
S: on /mnt/s type drvfs (rw,noatime,uid=1000,gid=1000,umask=22,fmask=11,metadata,case=off)

Some observations:

  • /mnt/h does NOT show metadata, although it was requested in /etc/fstab
  • Yet the files on the DrvFs are marked with the correct user and mode
  • I am mounting the Windows directory with UNC, not a network drive

Then my experiments:

$ cd /tmp
$ ln -s /mnt/h/windows.txt .
$ cd /mnt/h
$ ln -s /tmp/wsl.txt .
ln: failed to create symbolic link './wsl.txt': Operation not permitted

Note that Symbolic links do work on my Windows 10 machine (I have had the Developer Mode setting turned on since the early days of WSL):

C:\temp>mklink new.txt windows.txt
symbolic link created for new.txt <<===>> windows.txt

Am I missing a setting? I'm worried about 'mount -l' not showing the metadata option...

Confirming:

OS Name Microsoft Windows 10 Enterprise
Version 10.0.17763 Build 17763

Using Ubuntu 18.04 LTS

@SvenGroot
Copy link
Member

SvenGroot commented Mar 29, 2019

If you mount using a UNC path, DrvFs goes through SMB rather than directly to NTFS. This is both a lot slower and lacks quite a bit of functionality, including metadata, case sensitivity, and indeed symlinks. It is strongly not recommended to mount DrvFs this way for local filesystems.

To achieve the effect you want, use bind mounts instead. Put the following in your /etc/fstab:

C:              /mnt/c  drvfs   rw,noatime,metadata,uid=1000,gid=1000,umask=022,fmask=011
/mnt/c/Windows  /mnt/h  none    bind

Note you have to explicitly mount C: before creating the bind mount because we process /etc/fstab before automounting DrvFs volumes.

@ghost
Copy link

ghost commented Mar 29, 2019

Thank you, that does the trick indeed...
I am trying to count on the main documentation for WSL (https://docs.microsoft.com/en-us/windows/wsl/about), but that one falls far short on essential information, especially on new features.

I have found the blogs (https://devblogs.microsoft.com/commandline/) to be very instructive, but I probably missed the advise you provided above: don't quite know which blogs I should read. Maybe I should just prepare a pot of coffee, and read them all...

I think the key for an even more successful WSL would be a comprehensive documentation site that organizes everything there is to know?

Anyway, my apologies for not using a bind mount, that is one pearl of great information!!!

Oh: Once I switch to a bind mount, there is also no issue with OneDrive: I can put symbolic links in there to my heart's content! My WSL Home Directory is now fully managed in OneDrive, and I can even run XFCE4 from that home directory!!!

@github-cygwin
Copy link

github-cygwin commented Apr 1, 2019

I really wonder why there isn't an easy way out: Instead of the current Windows symlink reparse point, invent a new symlink reparse point which is type agnostic (not file or dir, but "just a symlink"). Create symlinks using this new type on both sides of the fence (Windows/WSL). Support the old symlink reparse point for backward compat only.

This new symlink reparse point could be defined to contain a Windows path as well as a POSIX path, so it could simply work for files and dirs both, and it would finally be possible to create working symlinks with no target on the Windows side as well.

What is so complicated about this that this hasn't already been done?

Corinna

@martixy
Copy link

martixy commented Apr 1, 2019

Corinna, symlinks/reparse points are a feature of the File System. New file systems don't come around that often. And this is not a problem that could force the whole windows ecosystem to switch to a new file system.

@github-cygwin
Copy link

github-cygwin commented Apr 1, 2019

Creating and evaluating a reparse point content is an OS thing, and since
reparse points can be simply read by the application as well, even applications can
decide how to treat a reparse point.

Having said that, new reparse points can be easily defined by Microsoft, as you can
see by the addition of IO_REPARSE_TAG_LX_SYMLINK for WSL. And I'm only talking
about NTFS here anyway.

The idea is that the Windows side refrains from requiring the file type. This can
be done by either not requiring the file type in apps like Explorer or CMD. This
is possible, just have a look into Cygwin. Native apps, or the OS symlink
evaluation routine, or the NTFS driver just have to do it. In theory this wouldn't
even require to create a new symlink type, IO_REPARSE_TAG_SYMLINK would be
sufficient if the type check just goes away. But for backward compat, that symlink
type could stay as is and a new symlink reparse point type could take over the job.

Corinna

@therealkenc
Copy link
Collaborator

Instead of the current Windows symlink reparse point, invent a new symlink reparse point which is type agnostic (not file or dir, but "just a symlink").

So... Cygwin classic symlinks. Shortcuts. Registry shell objects (aka #2779). WSL reparse point symlinks. Windows native symlinks. Windows NFS symlinks.

And a new kind.

New file systems don't come around that often.

Sure they do.

The irony is all the WSL devs are doing their kernel development on (MSFT's most unfortunately named) GVFS. It caches metadata (because of course it does) to work-around the underlying problems with stat on NTFS. That was released in 2017.

MSFT's Plan 9 filesystem implementation was released just last month.

[DrvFS with SMB on WSL] is both a lot slower and lacks quite a bit of functionality, including metadata, case sensitivity, and indeed symlinks.

this.

@SvenGroot
Copy link
Member

Removing the "matching file type" restriction on existing symlinks, or introducing a new symlink type that doesn't have that restriction, is definitely something we looked at.

Unfortunately, the assumption that the symlink has the same type as what you'll eventually open is baked into the system at a bunch of places, and done in such a way that it would be hard to change without likely breaking something. Backwards compatibility can be a pain sometimes. While it wouldn't be impossible to pull this off, it would definitely not be trivial.

@github-cygwin
Copy link

I see. When Windows introduced symlinks back in Vista, we@Cygwin were very happy at first. Unfortunately the target-type flag broke hope pretty quickly. Which is why Cygwin still doesn't use native symlinks by default and doesn't allow to create dangling native symlinks so as not to break any assumptions outside its own realm.

But either way, I'm pretty sure that only symlinks working transparently on both sides will fix all issues with symlinks.

Corinna

@ned14
Copy link

ned14 commented Apr 2, 2019

Firstly, Corinna I'd like to thank you so much for all the material on the internal workings of the NT kernel API which you have built up over many years on various mailing lists. I found them invaluable during the development of LLFIO, the reference library implementation for https://wg21.link/P1031 Low level file i/o library. Thank you so much for that.

I'd like to raise the point that fixing Windows symlinks to work well between WSL and native would help solve a long standing bug when using git on Windows, or git within WSL on a mounted Windows drive, where somebody has committed a symbolic link to the repo. Right now it doesn't work most of the time, and moreover, it doesn't work in a particularly non-obvious and hard to detect way.

If Windows symlinks understood both Windows and POSIX semantics, then git-created symlinks - whether created by a git inside WSL or outside - would "just work". This ought to be not hard to fix - git-created symlinks are always internal to the repo, referencing one part of the git repo from another part of the git repo. They should "just work".

@tinysun212
Copy link
Author

I posted that I tested and it was solved in version 1803.

Now, my PC is updated to 1809 (OS Build 17763.379). After testing again, I found that the result of the test which I wrote in the first post is reappeared. The OS is regressed at this issue.

04/02/2019  07:56 PM    <JUNCTION>     symlink.txt [...]

This issue should be reopened. Does anyone know how to re-open this? @therealkenc

@github-cygwin
Copy link

Firstly, Corinna I'd like to thank you so much for all the material on the internal workings of the NT kernel API which you have built up over many years on various mailing lists. I found them invaluable during the development of LLFIO, the reference library implementation for https://wg21.link/P1031 Low level file i/o library. Thank you so much for that.

Thanks!

I'd like to raise the point that fixing Windows symlinks to work well between WSL and native would help solve a long standing bug when using git on Windows, or git within WSL on a mounted Windows drive, where somebody has committed a symbolic link to the repo. [...]

Do you have a pointer to a description what exactly goes wrong?

If Windows symlinks understood both Windows and POSIX semantics, then git-created symlinks - whether created by a git inside WSL or outside - would "just work". [...]

The fact that the type check is "baked into the system at a bunch of places" as @SvenGroot outlines is really one of the major problems. Oh well.

Corinna

@ned14
Copy link

ned14 commented Apr 2, 2019

Do you have a pointer to a description what exactly goes wrong?

It was some time ago I last wrestled with this issue, so my memory may be faulty. The principle cause are repos which use libtool, and they ship in the repo prebuilt binaries linked by libtool. That said, I've occasionally bumped into repos which are Unix-only, and thus they think it's fine to commit symlinks.

There are two separate problems:

  1. drvfs appears to git that it doesn't support symlinks, when in fact WSL can create WSL-only symlinks just fine. Rather than git reporting this, or warning about it, instead it silently creates filesystem entities at those locations which don't work, but in a highly non-obvious way. I've no idea if the cause is git, or drvfs, or something else here.

  2. I can't see why relative symlinks from "the other side" can't work. I mean, if you get a Windows symlink which says "..\foo\bar", that clearly maps into "../foo/bar" in WSL. Or vice versa.

Now I appreciate that relative vs absolute symlinks on Windows is a bit fickle. Plenty of absolute symlinks ought to be relative. But it would be a start if Windows symlinks with the SYMLINK_FLAG_RELATIVE flag set would parse within WSL.

Outside WSL, it's much harder. Symlinks have their own reparse type. Me personally, I would have WSL create Windows symlinks all the time if the symlink is a relative path, but with the PrintName being the POSIX side of things, and SubstituteName being the Windows side of things. That dispenses with the problem entirely for many use cases.

Obviously absolute path symlinks are much harder to solve. But relative path symlinks I think are very solvable, and without needing to invent new reparse point types, or break anything existing.

@github-cygwin
Copy link

github-cygwin commented Apr 2, 2019

Do you have a pointer to a description what exactly goes wrong?

It was some time ago I last wrestled with this issue, so my memory may be faulty. The principle cause are repos which use libtool, and they ship in the repo prebuilt binaries linked by libtool. That said, I've occasionally bumped into repos which are Unix-only, and thus they think it's fine to commit symlinks.
[...]
Me personally, I would have WSL create Windows symlinks all the time if the symlink is a relative path, but with the PrintName being the POSIX side of things, and SubstituteName being the Windows side of things. That dispenses with the problem entirely for many use cases.

I think this should be the general idea, but it's a bit tricky if you don't bump the
reparse point tag at the same time. Unprepared systems accessing these symlinks
from remote might misbehave if the content of IO_REPARSE_TAG_SYMLINK symlinks
is changed. So, yeah, defining a new symlink type which is filled with Windows and
POSIX path would be the way to go.

Obviously absolute path symlinks are much harder to solve.

I don't think so. The content of the POSIX path inside this symlink could be absolute
in terms of the WSL view on drives, i.e. /mnt/<drive char>/... (hope I got that right).
Converting paths inside its own root can be easily fixed inside WSL. That would have
the additional advantage that these symlinks work nicely on Cygwin, too 😇

Corinna

@ned14
Copy link

ned14 commented Apr 2, 2019

I think this should be the general idea, but it's a bit tricky if you don't bump the
reparse point tag at the same time. Unprepared systems accessing these symlinks
from remote might misbehave if the content of IO_REPARSE_TAG_SYMLINK symlinks
is changed. So, yeah, defining a new symlink type which is filled with Windows and
POSIX path would be the way to go.

As far as I am aware, PrintName in the Windows symlink reparse structure is not used for anything except displaying extra information about the symlink.

Therefore using it to store the POSIX symlink ought to work absolutely fine. Nothing breaks. All WSL needs to do is decide if it's a valid POSIX path, and that's trivially easy.

@github-cygwin
Copy link

I only saw this used in cmd's dir output, too, so far. I hope this is a safe bet.

Corinna

@ghost
Copy link

ghost commented Oct 22, 2019

ln -s somedir someotherdir now generates a file again instead of a symlink. That file is not accessible in explorer (or in VSCode, for that matter).
I'm running Windows 10 Home, version 1903, build 19002.1002, and WSL v1.

@loliconer
Copy link

loliconer commented May 30, 2020

Nowadays, windows 10 (2004) still can't open symlinks created in WSL2. windows treat them as general files.

@therealkenc
Copy link
Collaborator

Nowadays, windows 10 (2004) still can't open symlinks

Landing zone for that over at #4868.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests