fatal: mmap: could not determine filesize

[LiveFreeAndRoam]

• I was not able to find an open or closed issue matching what I'm seeing

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

64-bit.

$ git --version --build-options
git version 2.24.1.windows.2
cpu: x86_64
built from commit: 992f0773022527b1b0cb1e0c13aec97dd5248053
sizeof-long: 4
sizeof-size_t: 8

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

$ cmd.exe /c ver
Microsoft Windows [Version 10.0.16299.1508]
(c) 2017 Microsoft Corporation. All rights reserved.

• What options did you set as part of the installation? Or did you choose the
  defaults?

>type "C:\Program Files\Git\etc\install-options.txt"
Editor Option: VisualStudioCode
Custom Editor Path:
Path Option: CmdTools
Plink Path: C:\Program Files\PuTTY\plink.exe
SSH Option: Plink
Tortoise Option: false
CURL Option: WinSSL
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Performance Tweaks FSCache: Enabled
Use Credential Manager: Enabled
Enable Symlinks: Disabled
Enable Builtin Interactive Add: Disabled

• Any other interesting things about your environment that might be related
  to the issue you're seeing?

This error only occurs with the "git status" command and only for one of my repositories. It is a 6.1GB (not a typo) repo.

NOTE: cygwin's git v2.17.0 successfully returns "git status" results. E.g.

cygwin$ git --version
git version 2.17.0
cygwin$  git status
On branch mybranch
Your branch is ahead of 'origin/mybranch' by 2 commits.
  (use "git push" to publish your local commits)

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   gui.js
        modified:   sidebar.js

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        testing/

no changes added to commit (use "git add" and/or "git commit -a")

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

I have run git from Bash, CMD, PowerShell, Git bash, and from within VSCode. All behave the same way.

• What commands did you run to trigger this issue? If you can provide a
  Minimal, Complete, and Verifiable example
  this will help us understand the issue.

PS D:\myrepo> & 'C:\Program Files\Git\mingw64\bin\git.exe' status
Refresh index: 11% (1065/9674)
fatal: mmap: could not determine filesize

• What did you expect to occur after running these commands?

The output from "git status".

• What actually happened instead?

The following error is displayed:

fatal: mmap: could not determine filesize

• If the problem was occurring with a specific repository, can you provide the
  URL to that repository to help us with testing?

I'm afraid it's company confidential.

[dscho]

* If the problem was occurring with a specific repository, can you provide the
  URL to that repository to help us with testing?

I'm afraid it's company confidential.

And I am afraid that the provided information is insufficient to allow others to help.

You could at least try to replicate the problem with an open source repository. And if that does not reproduce the problem, you could try to use git fast-export --anonymize and see whether the problem reproduces in a freshly git init-ed repository where that history was git fast-imported, and if it reproduces, you could investigate further from there, or obtain permission to publish the anonymized repository.

But in the current form, this ticket is too scarce on details to help. If this is all the information we will ever see publicly, I will be forced to close this ticket eventually, without any resolution.

[LiveFreeAndRoam]

Thanks Johannes.

I found it! I tried your suggestion, but after anonymizing, the problem no longer occurred. I continued searching.

After much more investigation, I eventually stumbled on it. I discovered that the repo (which comes from Linux server) has filename con.c and con.h. The con filename is Windows reserved. I was able to rename these files using cygwin, and now git is working again!

This leads me to ask: is it possible for git-for-windows to detect these Windows reserved filenames and display a more descriptive error message?

[dscho]

is it possible for git-for-windows to detect these Windows reserved filenames and display a more descriptive error message?

I think that is precisely what #2440 is about. This PR made it into the latest snapshot and also the latest -rc version. Would you please test whether it still happens with such a bad error message when using https://github.com/git-for-windows/git/releases/tag/v2.25.0-rc1.windows.1 (you can test with the Portable Git if you do not want to overwrite your current Git for Windows installation)?

[LiveFreeAndRoam]

I think that is precisely what #2440 is about.

Amazing! It is indeed the same issue.

I am now testing with v2.25. Starting with a tree containing con.[ch] that was cloned by v2.24, I then installed v2.25. Some preliminary findings...

1. The v2.25 error message is a little cryptic, not really explaining why there is a problem. Is there any chance of surfacing a more precise error message, e.g. one that identifies the filename as a "Windows reserved filename"?

2. Git output goes on to incorrectly conclude that there are untracked working tree files that would be overwritten by checkout, when the code should probably exit before displaying the untracked list. I'm still verifying this. It takes a long time because the download speed is a terrible 60KiB/s! (I should have another update tomorrow).

3. It seems the con.[ch] files are no longer visible in the git working tree, yet they seem to still cause problems... perhaps this is because the .git tree still contains references to them? (I'm guessing). E.g. git checkout knows they exist in the repo, but the files are never checked out.

$ git checkout mybranch
error: invalid path 'con.c'
error: invalid path 'con.h'
error: The following untracked working tree files would be overwritten by checkout:
    <a long list of files that are, in fact, tracked - can you exit before displaying this list?>
Aborting

[dscho]

1. The v2.25 error message is a little cryptic, not really explaining why there is a problem. Is there any chance of surfacing a more precise error message, e.g. one that identifies the filename as a "Windows reserved filename"?

Unfortunately, this would not exactly be easy in the current code base :-(

2. Git output goes on to incorrectly conclude that there are untracked working tree files that would be overwritten by checkout, when the code should probably exit before displaying the untracked list. I'm still verifying this. It takes a long time because the download speed is a terrible 60KiB/s! (I should have another update tomorrow).

Wow, that's bad! What it should do is to simply stop after invalid path. Hrm.

[LiveFreeAndRoam]

From my results below, the patch in v2.25 needs a little more work. Let me know if there is something else you would like me to try.

v2.25 Cloned but Checkout Failed

Using v2.25, I cloned a tree that contains the con.[ch] files. The clone was successful, but the checkout failed, as shown below. Since no files were checked out, the working tree is unusable.

$ git clone git@github.mycorp.com:myhub/myrepo.git repo
Cloning into 'repo'...
remote: Enumerating objects: 35, done.
remote: Counting objects: 100% (35/35), done.
remote: Compressing objects: 100% (30/30), done.
remote: Total 191722 (delta 3), reused 12 (delta 2), pack-reused 191687
Receiving objects: 100% (191722/191722), 1.56 GiB | 129.00 KiB/s, done.
Resolving deltas: 100% (131470/131470), done.
error: invalid path 'con.c'
fatal: unable to checkout working tree
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'

Working tree is empty

$ ls

How do I populate the working tree?

I tried moving the offending file, but it fails too.

$ git mv con.c cons.c
fatal: bad source, source=con.c, destination=cons.c

Downgrade to v2.24 for the checkout

With v2.24 installed, the initial checkout succeeded, with errors for the con.[ch] files.

$ git checkout mybranch
error: unable to stat just-written file con.c: No such file or directory
error: unable to stat just-written file con.h: No such file or directory
Updating files: 100% (9679/9679), done.
Switched to a new branch 'mybranch'
D       con.c
D      con.h
Branch 'mybranch' set up to track remote branch 'mybranch' from 'origin'.

Switching to other branches succeeds, without the error message above.

$ git checkout feature1
Switched to a new branch 'feature1'
D       con.c
D       con.h
Branch 'feature1' set up to track remote branch 'feature1' from 'origin'.

$ git status
On branch feature1
Your branch is up to date with 'origin/feature1.

Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        deleted:    con.c
        deleted:    con.h

no changes added to commit (use "git add" and/or "git commit -a")

At this point, I have a useable working tree.

[dscho]

Using v2.25, I cloned a tree that contains the con.[ch] files. The clone was successful, but the checkout failed, as shown below.

This is actually as intended. You don't want the clone to fail because the user would not have a way to fix things otherwise (a failed clone removes the worktree altogether).

How do I populate the working tree?

I tried moving the offending file, but it fails too.

$ git mv con.c cons.c
fatal: bad source, source=con.c, destination=cons.c

This fails because there is no such file. You can always call git mv --cached con.c cons.c, followed by git restore --source=HEAD -- cons.c.

[dscho]

So I guess this can be closed now.