git bash should error on 'ln -s' if it is not enabled

[ernstklrb]

• I was not able to find an open or closed issue matching what I'm seeing

Setup

• Which version of Git for Windows are you using? 2.28.0
• Is it 32-bit or 64-bit? 64

$ git --version --build-options
git version 2.28.0.windows.1
cpu: x86_64
built from commit: 77982caf269b7ee713a76da2bcf260c34d3bf7a7
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?
  Windows 10 64bit

$ cmd.exe /c ver

Microsoft Windows [Version 10.0.18363.1379]

• What options did you set as part of the installation? Or did you choose the
  defaults? I do not know - it is installed by Admin.

# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"
> type "C:\Program Files (x86)\Git\etc\install-options.txt"
> type "%USERPROFILE%\AppData\Local\Programs\Git\etc\install-options.txt"
$ cat /etc/install-options.txt

** insert your machine's response here **

• Any other interesting things about your environment that might be related
  to the issue you're seeing?
  none

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

This report is about 'git bash' itself, not about git.

• What I see

Apparently, when creating symlinks is not enabled at install time, git bash will resort to copying the source to the target instead., if a ln -s source target command is issued. This is what I see from issuing stat on the source and the result.

• What I would like to see

In case symlinking has not been enabled, ln -s should stop with an error (instead of resorting to a semantically incompatible alternative behavior).

[dscho]

@ernstklrb you have a good point. MSYS2's Bash (which we use in Git for Windows) does not create symlinks by default.

However, there is a "knob" you can turn: setting the environment variable MSYS=winsymlinks:nativestrict (see Cygwin's documentation, MSYS is the MSYS2 equivalent of CYGWIN).

Now, we already have prior art for modifying the MSYS variable when launching Git Bash: git-for-windows/build-extra@b73b4e3

Essentially, you can add the line MSYS=winsymlinks:nativestrict to /etc/git-bash.config (or modify an existing line setting MSYS, multiple options can be specified, separated by blanks).

Could you try that (as administrator, because /etc/git-bash.config is actually C:\Program Files\Git\etc\git-bash.config) and see whether that works as intended?

If so, we can add that setting automatically when support for symbolic links is enabled in Git for Windows' installer.

[ernstklrb]

I have tested with a fresh installation of git for windows, with "symbolic links" enabled in the installer.
I get the following behavior:

• before adding the MSYS=... line:
  'ln -s' silently creates a copy instead
• after the line added (actually, the file git-bash.config had to be freshly created)
  ln: failed to create symbolic link 'c.txt': Operation not permitted

Maybe a better message would have been "Operation not possible", but I guess that is outside of your scope

[ernstklrb]

Now I did a second test, again with a fresh installation (after uninstallation and completely removing c:\Program Files\Git).

This time without enabling symbolic links.

I get the same behavior as in the "with symlinks" case.

So I think you either need to add the MSYS-line in both cases (with and without symlinks enabled), or maybe just in this second case without symlinks enabled.

But it is not sufficient to do it just in the "enabled" case you referred to in your post.

[dscho]

ln: failed to create symbolic link 'c.txt': Operation not permitted

Can you verify that mklink can create symlinks in your setup (this should be the case if you enabled Developer Mode)?

[dscho]

I get the same behavior as in the "with symlinks" case.

Wait, I am confused. What was the "with symlinks" case? Or better: what was the behavior?

So I think you either need to add the MSYS-line in both cases (with and without symlinks enabled), or maybe just in this second case without symlinks enabled.

I think you're right on that. Unless we can guarantee that git-bash.config is removed (and potentially recreated) when reinstalling.

But that's not what I am trying to work out right now. I am trying to work out how to get Git Bash to expose the ln -s behavior that you want.

[rimrul]

Wait, I am confused. What was the "with symlinks" case?

If I undertand it correctly they're just saying ln -s behaves the same, wether you enable symlinks in the installer or not, it only depends on the MSYS environment variable.

[dscho]

Wait, I am confused. What was the "with symlinks" case?

If I undertand it correctly they're just saying ln -s behaves the same, wether you enable symlinks in the installer or not, it only depends on the MSYS environment variable.

Yes, that would make sense because Bash does not care about Git's config, nor does the MSYS2 runtime.

[ernstklrb]

ln: failed to create symbolic link 'c.txt': Operation not permitted

Can you verify that mklink can create symlinks in your setup (this should be the case if you enabled Developer Mode)?

So when I tried "MSYS=winsymlinks:nativestrict" first, I did not have permissions for doing mklink. This resulted in 'ln -s' erroring out, like I would expect.

Now, with permissions for mklink enabled as per your wiki entry, the behavior has changed: 'ln -s' in git bash will create a symlink.

[dscho]

Now, with permissions for mklink enabled as per your wiki entry, the behavior has changed: 'ln -s' in git bash will create a symlink.

Ouch. We should have edited https://github.com/git-for-windows/git/wiki/Symbolic-Links#allowing-non-administrators-to-create-symbolic-links a long time ago, I think, to prominently encourage Developer Mode, and only as a fall-back to do the permission dance. Unfortunately, I lack the time to edit this page...

[ernstklrb]

Improving the documentation reg. symbolic links might be good.
Regardless, I think it is worth improving the default behavior in the non-developer-mode case to not silently copy instead of symlinking. Because

• this is about the initial behavior of git bash after installation without any "tweaking"
• in a corporate environment, e.g., it might be difficult to get mklink permissions enabled

[dscho]

Improving the documentation reg. symbolic links might be good.

Would you be so kind?

[dscho]

I think it is worth improving the default behavior in the non-developer-mode case to not silently copy instead of symlinking.

That's a decision for the MSYS2 project, though...

[ernstklrb]

I think it is worth improving the default behavior in the non-developer-mode case to not silently copy instead of symlinking.

That's a decision for the MSYS2 project, though...

For git-bash, I guess, the git-for-windows project can decide how to handle the symlink issue.

I suggest you add MSYS=winsymlinks:nativestrict to /etc/git-bash.config on installation. (At least when /etc/git-bash.config does not yet exist.) Rather provoke a clear error message and avoid bad surprises later with "symlinks" that happen to be copies.

[ernstklrb]

Improving the documentation reg. symbolic links might be good.

Would you be so kind?

Maybe. I do not even understand the first paragraph: What is "Developer Mode", what is "symlink emulation", is "core.symlinks=true" a git config item? I only ever use symlinks in the shell "by hand". I do not track symlinks with git (which is a much more complex case in the context of git-for-windows I suspect).

[dscho]

I suggest you add MSYS=winsymlinks:nativestrict to /etc/git-bash.config on installation. (At least when /etc/git-bash.config does not yet exist.) Rather provoke a clear error message and avoid bad surprises later with "symlinks" that happen to be copies.

I'd rather only do that when the "Enable symbolic link support" checkbox is checked. And then only once I am sure that this works as intended.

I do not even understand the first paragraph: What is "Developer Mode",

You can switch on Developer Mode in Windows 10 (which is also a prerequisite to using WSL, if I remember correctly). Here is a link: https://www.howtogeek.com/292914/WHAT-IS-DEVELOPER-MODE-IN-WINDOWS-10/. Maybe you can find a more authoritative one, i.e. something official from Microsoft? It would be good to have a link in that wiki page.

what is "symlink emulation",

That is probably a left-over from olden times. Cygwin has a way to emulate symbolic links, but only Cygwin can interpret those.

is "core.symlinks=true" a git config item?

Yes.

I only ever use symlinks in the shell "by hand". I do not track symlinks with git (which is a much more complex case in the context of git-for-windows I suspect).

Git's source code itself has a symbolic link it tracks: RelNotes: https://github.com/git/git/blob/v2.31.0/RelNotes

But yeah, I agree, it is not a common thing on Windows.

[ernstklrb]

I suggest you add MSYS=winsymlinks:nativestrict to /etc/git-bash.config on installation. (At least when /etc/git-bash.config does not yet exist.) Rather provoke a clear error message and avoid bad surprises later with "symlinks" that happen to be copies.

I'd rather only do that when the "Enable symbolic link support" checkbox is checked. And then only once I am sure that this works as intended.

sure this should only be done after you are sure that this works as intended (on the prevalent versions of windows).

Still my main concern in this issue is that "unexpected" / "surprising" behavior of git bash should be avoided.

To me as a mostly linux user, silently getting a copy when issuing a ln -s command is surprising. And it is prone to lead to subtle problems if not noticed by the user immediately.

Apparently this only happens when neither in "Developer Mode", nor the user has the SeCreateSymbolicLinkPrivilege privilege.
Yet this is a relevant case, in particular to the user not educated about these things.
And to the user with no admin priviledge on their machine.

This is why I argue for "MSYS=winsymlinks:nativestrict" as a general default: it will avoid the surprise.

[ernstklrb]

After reading a bit about symlinks on windows (as a linux user), and looking at the current wiki text reg. symlinks, I have the impression that the interaction between
(a) permissions of the windows user,
(b) installation option "Enable symbolic link support", and
(c) the git config item "core.symlinks"
makes the behavior of git-for-windows quite complex. Some of the complexity is possibly due to historic reasons (from a time when there were no (more or less) proper symlinks available on windows).

One specific problem here is that checking (b) actually does not enable symlink support, as long as (a) is not given.

How about removing (b) altogether, and instead resorting to notifying the user at runtime in case an attempt to create a symlink fails?

[dscho]

Still my main concern in this issue is that "unexpected" / "surprising" behavior of git bash should be avoided.

I understand that. And you can make a case for that at the project whose authority it is to change that behavior: MSYS2.

This is why I argue for "MSYS=winsymlinks:nativestrict" as a general default: it will avoid the surprise.

For you. For existing users who got used to the current behavior, you are causing surprise. And with over 3m downloads, I would be surprised if the number of those users were negligible.

the interaction [...] makes the behavior of git-for-windows quite complex.

I fully agree.

How about removing (b) altogether, and instead resorting to notifying the user at runtime in case an attempt to create a symlink fails?

I think that would be too late. At least in the installer, users have a chance to notice that their symlink support is disabled, and they can then go and enable Developer Mode before cloning and ending up with plain files (because Git does not actually copy the symlinked files, instead, when core.symlinks = false, it checks out files with the target path as contents.

[Klaim]

For you. For existing users who got used to the current behavior, you are causing surprise. And with over 3m downloads, I would be surprised if the number of those users were negligible.

I think a new option in the installer would help clarify that you can enable this at install time, and previous users will be informed and chose to enable it or not.

Note that I'm digging up this issue because as a build2 user and in particular when packaging projects using other buildsystems/package-managers I have to work on Windows with a ton of cloned git projects which use a lot of symlinks.
Through the years I had a lot of trouble with the manipulations of symlinks on Windows mainly because git-on-windows was "hiding" the issue. Now I have some notes somewhere to check for when I have these issues, which basically are summarized here, and in build2's doc + the windows Developer mode to activate + the MSYs option to enable.

I also realized that even when you setup all this, cloning a repository which was created by people on linux and cloning on windows will not lead to symlink to folders to properly work, while recreating them manually locally will. This one is weird and I didn't find the exact reason yet so I'll report it another time.

All that to point that symlink working correctly is quite important, at least for package managers which work on windows.

[dscho]

I think a new option in the installer would help clarify that you can enable this at install time, and previous users will be informed and chose to enable it or not.

Sure, that makes sense. You could piggy-back on top of the work that went into the pseudo console support code and ensure that winsymlinks:nativestrict is written into /etc/git-bash.config.

However, that would only affect Git Bash users. Everybody else would call git.exe directly, and the shell script parts of Git would then not have that option turned on.

It is a bit tricky to get this right because we cannot simply set environment variables in Git's system config.

[jcrben]

@dscho

You can switch on Developer Mode in Windows 10 (which is also a prerequisite to using WSL, if I remember correctly)

You haven't needed it since 2017.

Security departments are probably still leery of enabling Developer Mode.

[dscho]

@jcrben could you back up that claim by a link to authoritative documentation as to which Windows version should no longer require Developer Mode?

[jcrben]

@dscho yep here's the blog post from Microsoft in 2017: https://devblogs.microsoft.com/commandline/developer-mode-no-longer-required-for-windows-subsystem-for-linux/

[rimrul]

@jcrben that post is about developer mode no longer being required for WSL, though and has no mention of Symlinks.

[jcrben]

@rimrul yep, you need Developer Mode for symlinks. I was responding to the comment that Developer Mode "is also a prerequisite to using WSL".

My point is that you shouldn't assume Developer Mode and therefore you shouldn't assume symlinks. Developer Mode is probably not a common thing on Windows workstations even for developers inside highly-secured corporate environments.

It's perhaps a bit easier to access to WSL than symlinks natively inside Windows.