Cannot lock the ref -> detached HEAD

[lchiocca]

When rebasing a sym-linked repository, I'm getting a detached head in 2.4.2.1, which didn't happen in 2.4.0.1. The setup is as following:

• Referencing git.exe through the path, but the same error occurs with git-cmd.exe and git-bash.exe
• clone a repo
• create a "git-new-workdir" repo. The script can be found in the official git repo under "contrib/workdir/git-new-workdir". But I guess you know this. I have converted this to a batch file in case you need it
• git rebase the symlinked repo -> issue

Here is the console output (first with 2.4.0.1 then with 2.4.2.1):

c:\workspace\git\MyCompany\5.90\CompanyProduct>mklink /d c:\git c:\PortableGit-2.4.0.1
symbolic link created for c:\git <<===>> c:\PortableGit-2.4.0.1

c:\workspace\git\MyCompany\5.90\CompanyProduct>git status
On branch 5.90
Your branch is up-to-date with 'origin/5.90'.
nothing to commit, working directory clean

c:\workspace\git\MyCompany\5.90\CompanyProduct>git rebase
First, rewinding head to replay your work on top of it...
Fast-forwarded 5.90 to refs/remotes/origin/5.90.

c:\workspace\git\MyCompany\5.90\CompanyProduct>git status
On branch 5.90
Your branch is up-to-date with 'origin/5.90'.
nothing to commit, working directory clean

c:\workspace\git\MyCompany\5.90\CompanyProduct>rd c:\git

c:\workspace\git\MyCompany\5.90\CompanyProduct>mklink /d c:\git c:\PortableGit-2.4.2.1
symbolic link created for c:\git <<===>> c:\PortableGit-2.4.2.1

c:\workspace\git\MyCompany\5.90\CompanyProduct>git status
On branch 5.90
Your branch is up-to-date with 'origin/5.90'.
nothing to commit, working directory clean

c:\workspace\git\MyCompany\5.90\CompanyProduct>git rebase
First, rewinding head to replay your work on top of it...
Fast-forwarded 5.90 to refs/remotes/origin/5.90.
error: unable to create directory for .git/refs/heads/5.90
fatal: update_ref failed for ref 'refs/heads/5.90': Cannot lock the ref 'refs/heads/5.90'.
Could not move back to refs/heads/5.90

c:\workspace\git\MyCompany\5.90\CompanyProduct>git status
HEAD detached at refs/heads/5.90
nothing to commit, working directory clean

c:\workspace\git\MyCompany\5.90\CompanyProduct>

[lchiocca]

If you want to check the git-new-workdir.bat that I ported to work on any windows:
http://pastebin.com/GHLEiX7k

[dscho]

Let's just paste the script here so that everybody involved can be certain that they have the revision that you are talking about:

@echo off
goto :start

:usage
echo usage: %0 <repository> <new_workdir> [<branch>]
goto :eof

:start

set argC=0
for %%x in (%*) do Set /A argC+=1

if %argC% LSS 2 goto usage
if %argC% GTR 3 goto usage

set orig_git=%1
set new_workdir=%2
set branch=%3

rem want to make sure that what is pointed to has a .git directory ...
pushd "%orig_git%"
IF ERRORLEVEL 1 (
    echo cannot change to directory: "%orig_git%"
    goto eof
)
set git_dir=
for /F %%l in ('git rev-parse --git-dir') do set git_dir=%%l
IF ERRORLEVEL 1 (
    echo Not a git repository: "%orig_git%"
    goto eof
)
popd
if %git_dir%==.git set git_dir=%orig_git%\.git
if %git_dir%==. set git_dir=%orig_git%

rem don't link to a configured bare repository
for /F %%l in ('git --git-dir="%git_dir%" config --bool --get core.bare') do set isbare=%%l
if ztrue==z%isbare% (
    echo "%git_dir%" has core.bare set to true, remove from "%git_dir%/config" to use %0
    goto eof
)

rem don't link to a workdir
for %%l in ("%git_dir%\config") do set config_attribs=%%~al
if "%config_attribs:~-1%" == "l" (
    echo "%orig_git%" is a working directory only, please specify a complete repository.
    goto eof
)

rem don't recreate a workdir over an existing repository
if exist %new_workdir% (
    echo destination directory '%new_workdir%' already exists.
    goto eof
)

rem make sure the links use full paths
pushd "%git_dir%"
for /F %%l in ('cd') do set git_dir=%%l
popd

rem create the workdir
md "%new_workdir%\.git"
IF ERRORLEVEL 1 (
    echo unable to create "%new_workdir%"!
    goto eof
)

rem create the links to the original repo.  explicitly exclude index, HEAD and
rem logs/HEAD from the list since they are purely related to the current working
rem directory, and should not be shared.
md "%new_workdir%\.git\logs"
for %%l in (config refs logs/refs objects info hooks packed-refs remotes rr-cache svn) do (
    if exist "%git_dir%\%%l\" (
        mklink /D "%new_workdir%\.git\%%l" "%git_dir%\%%l"
    ) else (
        mklink "%new_workdir%\.git\%%l" "%git_dir%\%%l"
    )
)

rem now setup the workdir
pushd "%new_workdir%"
rem copy the HEAD from the original repository as a default branch
copy "%git_dir%\HEAD" .git\HEAD
rem Need to remove the git_dir variable, otherwise git checkout would pick it up
set git_dir=
rem checkout the branch (either the same as HEAD from the original repository, or
rem the one that was asked for)
git checkout -f %branch%
popd

:eof

In addition, I would really like you to look into creating a Minimal, Complete & Verifiable Example, i.e. a script that demonstrates the failure (which is much more painless a way to recreate the problem in other developers' setups than any other method).

[dscho]

Also, please note that I just marked this as a "question" rather than a bug because the problems occur due to the use of scripts and methods that are not officially supported by Git for Windows.

That means that I, personally, as maintainer of Git for Windows will have to pay more attention to other tickets that are about officially supported features. It also means that I appreciate whatever improvements this ticket brings to Git for Windows.

[lchiocca]

Hi dscho, I'm putting together an exact and minimal test case to follow. I'm not sure about the question, though. It used to work with 2.4.0.1, but no longer works, but I know what you mean.

[lchiocca]

Rats, you can only get into the situation of a deatched head if you used a mix of 2.4.0.1 and 2.4.2.1. But the lock can be easily reproducable:

git clone https://github.com/lchiocca/GitWindowsIssue186.git
git-new-workdir.bat GitWindowsIssue186 GitWindowsIssue186SymLink
cd GitWindowsIssue186SymLink
git checkout old

Output for 2.4.0.1:

c:\workspace\git>git clone https://github.com/lchiocca/GitWindowsIssue186.git
Cloning into 'GitWindowsIssue186'...
Checking connectivity... done.

c:\workspace\git>git-new-workdir.bat GitWindowsIssue186 GitWindowsIssue186SymLink
symbolic link created for GitWindowsIssue186SymLink\.git\config <<===>> c:\workspace\git\GitWindowsIssue186\.git\config
symbolic link created for GitWindowsIssue186SymLink\.git\refs <<===>> c:\workspace\git\GitWindowsIssue186\.git\refs
symbolic link created for GitWindowsIssue186SymLink\.git\logs/refs <<===>> c:\workspace\git\GitWindowsIssue186\.git\logs/refs
symbolic link created for GitWindowsIssue186SymLink\.git\objects <<===>> c:\workspace\git\GitWindowsIssue186\.git\objects
symbolic link created for GitWindowsIssue186SymLink\.git\info <<===>> c:\workspace\git\GitWindowsIssue186\.git\info
symbolic link created for GitWindowsIssue186SymLink\.git\hooks <<===>> c:\workspace\git\GitWindowsIssue186\.git\hooks
symbolic link created for GitWindowsIssue186SymLink\.git\packed-refs <<===>> c:\workspace\git\GitWindowsIssue186\.git\packed-refs
symbolic link created for GitWindowsIssue186SymLink\.git\remotes <<===>> c:\workspace\git\GitWindowsIssue186\.git\remotes
symbolic link created for GitWindowsIssue186SymLink\.git\rr-cache <<===>> c:\workspace\git\GitWindowsIssue186\.git\rr-cache
symbolic link created for GitWindowsIssue186SymLink\.git\svn <<===>> c:\workspace\git\GitWindowsIssue186\.git\svn
        1 file(s) copied.
Your branch is up-to-date with 'origin/master'.
c:\workspace\git>cd GitWindowsIssue186SymLink

c:\workspace\git\GitWindowsIssue186SymLink>git checkout old
Branch old set up to track remote branch old from origin by rebasing.
Switched to a new branch 'old'

c:\workspace\git\GitWindowsIssue186SymLink>

For 2.4.2.1:

c:\workspace\git>git-new-workdir.bat GitWindowsIssue186 GitWindowsIssue186SymLink
symbolic link created for GitWindowsIssue186SymLink\.git\config <<===>> c:\workspace\git\GitWindowsIssue186\.git\config
symbolic link created for GitWindowsIssue186SymLink\.git\refs <<===>> c:\workspace\git\GitWindowsIssue186\.git\refs
symbolic link created for GitWindowsIssue186SymLink\.git\logs/refs <<===>> c:\workspace\git\GitWindowsIssue186\.git\logs/refs
symbolic link created for GitWindowsIssue186SymLink\.git\objects <<===>> c:\workspace\git\GitWindowsIssue186\.git\objects
symbolic link created for GitWindowsIssue186SymLink\.git\info <<===>> c:\workspace\git\GitWindowsIssue186\.git\info
symbolic link created for GitWindowsIssue186SymLink\.git\hooks <<===>> c:\workspace\git\GitWindowsIssue186\.git\hooks
symbolic link created for GitWindowsIssue186SymLink\.git\packed-refs <<===>> c:\workspace\git\GitWindowsIssue186\.git\packed-refs
symbolic link created for GitWindowsIssue186SymLink\.git\remotes <<===>> c:\workspace\git\GitWindowsIssue186\.git\remotes
symbolic link created for GitWindowsIssue186SymLink\.git\rr-cache <<===>> c:\workspace\git\GitWindowsIssue186\.git\rr-cache
symbolic link created for GitWindowsIssue186SymLink\.git\svn <<===>> c:\workspace\git\GitWindowsIssue186\.git\svn
        1 file(s) copied.
Your branch is up-to-date with 'origin/master'.
c:\workspace\git>cd GitWindowsIssue186SymLink

c:\workspace\git\GitWindowsIssue186SymLink>git checkout old
error: unable to create directory for .git/refs/heads/old
fatal: Cannot lock the ref 'refs/heads/old'.

c:\workspace\git\GitWindowsIssue186SymLink>

[lchiocca]

Let me also explain my use-case, just you know why I (some the others in our company) use this feature.

It really all started when we changed from hosting git internally to an external party. They have a really bad link to switzerland. Cloning our main repo takes two hours, which is just above 1GB in size. You can argue that you only need this once, but that's not quite true when you use eclipse as an IDE. Eclipse is really, really bad when it comes to switching branches and projects that were added/removed when switching. Hence we were "forced" to have for each branch a separate git working directory. To be able to still see changes in future branches without having to pull each of the branches, we opted to use the git-new-workdir, which works pretty well. The only known drawback up til now was, you could only run git-gc on the un-symlinked repo.

Anyhow, that the reason why we use git-new-workdir. Would love to see it working again in 2.4.2 :)

Thanks for all the hard work you're doing!
Loris

[lchiocca]

I nailed it down and it's not in git-for-windows at all! After a bit of debugging, i landed in sha1_file.c/safe_create_leading_directories(".git/refs/heads/old"). The stat(".git", *) says "Yeah... path exists and in the next iteration stat(".git/refs", *) returns 0, so it tries to mkdir(".git/refs") which returns EEXIST, but then there's the check that's calling stat again, which will fail everything.

So in summary: not an issue in git-for-windows. But it's an issue in the stat function. Could you tell me where I could file this?

[lchiocca]

Sorry... it's actually both stat() and S_ISDIR() that are buggy with symlinks under windows.

[lchiocca]

Also for the records: In the git-new-workdir.bat, I've used a directory symbolic link (mklink /D). If I use a directory junction (mklink /J) it works perfectly.

Closing this issue. But would still like to file the stat() and S_ISDIR bug somewhere.

[dscho]

Awesome job @lchiocca !

it's actually both stat() and S_ISDIR() that are buggy with symlinks under windows.

Actually we override the stat() function in Git for Windows, so it is a Git for Windows bug after all :-(

You can find the definition here and I think that S_ISDIR() is actually correct, but mingw_stat() needs to set the flags correctly.