Unlink of file XXX failed. Should I try again?

[gpakosz]

Hello,

Sometimes, after a e.g. git fetch, Git decides to auto GC a repository. In the process, it fails to remove .pack and .idx files. Likely, those files are locked.

Git can ask the question for more than 50 times depending on the number of unreachable commits there are following branch deletion and remotes pruning. When Git starts to complain, either you answer "no" repeatedly until the auto GC ends, or you have to CTRL-C. Ultimately, invoking git gc manually removes most of the pack files

[dscho]

@gpakosz hmm. Do you think you can come up with a reproducible example? Something like a shell script that creates a small repository, clones it, then sets gc thresholds ridiculously low, adds more commits and then fetches them?

[kgybels]

@dscho Something like this?

KGybels@PC-KG MINGW64 /c/git/bug
$ git --version
git version 2.6.1.windows.1

KGybels@PC-KG MINGW64 /c/git/bug
$ ls
steps.sh*

KGybels@PC-KG MINGW64 /c/git/bug
$ cat steps.sh
#!/bin/bash

git init origin
cd origin
touch test1.txt
git add test1.txt
git commit -m v1
originpath=`pwd`
cd ..
git clone file://$originpath clone
cd origin
touch test2.txt
git add test2.txt
git commit -m v2
cd ../clone
git config gc.autoPackLimit 1
git fetch

KGybels@PC-KG MINGW64 /c/git/bug
$ ./steps.sh
Initialized empty Git repository in C:/git/bug/origin/.git/
[master (root-commit) 775d4c6] v1
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 test1.txt
Cloning into 'clone'...
remote: Counting objects: 3, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (3/3), done.
Checking connectivity... done.
[master 25c50c7] v2
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 test2.txt
remote: Counting objects: 2, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 2 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (2/2), done.
From file:///c/git/bug/origin
   775d4c6..25c50c7  master     -> origin/master
Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
Counting objects: 5, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (5/5), done.
Total 5 (delta 0), reused 3 (delta 0)
Unlink of file '.git/objects/pack/pack-97e500b3cd7317acb865c29bb44d66886116c51a.pack' failed. Should I try again? (y/n) n
Unlink of file '.git/objects/pack/pack-97e500b3cd7317acb865c29bb44d66886116c51a.idx' failed. Should I try again? (y/n) n

KGybels@PC-KG MINGW64 /c/git/bug
$

[dscho]

@kgybels very nice!

[gpakosz]

Oh nice @kgybels, I added the task in my TODO but I'm thankful you took over. 👍

[cs96and]

I sometimes have this problem and I believe it is related to TortoiseGit's TGitCache.exe having those files open.

[gpakosz]

It's definitely not related to TortoiseGit. As you can see above, @kgybels provided a script that reproduces the problem. It's totally unrelated to any Git frontend

[dscho]

@gpakosz sorry for the long delay! In the meantime, I had taken care of #446 which made this fix substantially quicker, though.

[cs96and]

@gpakosz But is TortoiseGit installed? It can have those files locked even though you are currently using git on the command line.

[dscho]

@cs96and if TortoiseGit, or for that matter, any software locks pack files, the "Unlink of file XXX failed." is actually the proper response and there is no bug in Git.

The bug at hand is caused by Git itself keeping locks on the files that it wants to garbage-collect. And that bug is what I just fixed.

[linquize]

Not just TortoiseGit, Atom text editor also sometimes locks the packfile too.

[dscho]

Well, Atom text editor is included in "any software", no?

Apart from that, what business does Atom text editor have to lock pack files? Seriously.

[kostix]

@dscho, because it's more of an IDE (written in HTML5 on top of node.js and craploads of other bits of what is considered "hot stuff" these days) than a text editor. IDEs strive to provide VCS integration. Supposedly they're using libgit2 or something. Since it's done by github folks I'd not be surprised to actually find pieces of GfW inside (as it's AFAIK true for that piece of software named "Github for Windows").

All in all, failing to lock a locked file is for sure the only sensible way to go IMO.

[gpakosz]

@dscho no worry, thanks for taking care of fixing it, much appreciated

[dscho]

This hit upstream's master, so it will be fixed in the next release of Git for Windows.

[gpakosz]

Thank you @kgybels, @dscho and @gitster. This one has been nagging me literally for years.

[dscho]

@gpakosz note that in Open Source, you can always work on your own itches... 😃

[gpakosz]

@dscho That's true. I confess I never really tried to climb the learning curve of being able to compile and debug Git For Windows. And the first years it was all too easy to blame NTFS ;)

[dscho]

It's always easy to blame NTFS. And for the record: when I started working on Git, I also had no idea how to compile and debug Git for Windows...

[kgybels]

There is a regression from 2.17.1.windows.2 to 2.18.0.windows.1. I already send a patch to the Git mailing list.

[dscho]

I already send a patch to the Git mailing list.

Thanks! Can you let me know when it gets into next so I can cherry-pick? (I would like to avoid cherry-picking a commit that I later have to revert in order to cherry-pick a newer iteration... maintenance... or do you want me to pick it as a quick fix already?)

[kgybels]

@dscho Can you let me know when it gets into next so I can cherry-pick?

Change is merged to next on 2018-07-24 at 71c05d2. It will also be merged to master.

[dscho]

I opened a PR. Thank you so much!