LFS incompatible to cURL+winssl

[hsattler]

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

$ git --version --build-options

git version 2.12.2.windows.2
built from commit: 7723f38cfb0e80f010afaebdd9fec4d0822fd2e1
sizeof-long: 4
machine: x86_64

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

$ cmd.exe /c ver

Microsoft Windows [Version 6.1.7601]

• What options did you set as part of the installation? Or did you choose the
  defaults?

NOTE: This is what I have installed now to work around the issue. The bug report is about the winssl/SChannel CURL option!

# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"

Path Option: Cmd
Plink Path: C:\Program Files\TortoiseGit\bin\TortoiseGitPlink.exe
SSH Option: Plink
CURL Option: OpenSSL
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Performance Tweaks FSCache: Enabled
Use Credential Manager: Enabled
Enable Symlinks: Enabled

• Any other interesting things about your environment that might be related
  to the issue you're seeing?

Git and LFS server use a company-signed root certificate. OpenSSL needs a replace of the ca-bundle.crt file but curl/winssl works without this.

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

git bash, GitExtensions dialogs

• What commands did you run to trigger this issue? If you can provide a
  Minimal, Complete, and Verifiable example
  this will help us understand the issue.

"git lfs pull" when LFS really has to load some files.

• What did you expect to occur after running these commands?

The Files are loaded.

• What actually happened instead?

Certificate check failure in git-lfs. Non-LFS git command work and do not show a certificate failure.
Using the curl/openssl option and adding the company root certificate to the ca bundle, it works for git-lfs.

• If the problem was occurring with a specific repository, can you provide the
  URL to that repository to help us with testing?

Sorry, internal company services.
But the problem should appear with any self-created CA and a server-certificate issued by it. Add the self-created CA to the Windows Certificate Store and install git with curl/winssl option.

[shiftkey]

Certificate check failure in git-lfs. Non-LFS git command work and do not show a certificate failure.

This doesn't feel like it's a Git for Windows issue if the non-LFS commands work fine. But let's get to the bottom of what you're seeing.

Using the curl/openssl option and adding the company root certificate to the ca bundle, it works for git-lfs.

This needs more information, in particular:

• are you reinstalling with the option disabled (choosing the default)?
• where is the location of the certificate bundle you are modifying?
• are you setting any environment variables to point to the bundle location?

But the problem should appear with any self-created CA and a server-certificate issued by it.

Please provide some example output you are seeing by setting GIT_TRACE=1 and GIT_CURL_VERBOSE=1 before running the command that you're seeing fail. Certificates are fickle things when they don't work, so any extra context you can provide would be 💎

[dscho]

I agree that this may be a Git-LFS problem. @taylorr may be able to shed light into this: does Git LFS use OpenSSL internally for HTTPS transport? If so, is there an option to use Windows-native Secure Channel instead?

[hsattler]

I reinstalled with the OpenSSL option and replace C:\Program Files\Git\mingw64\ssl\certs\ca-bundle.crt. Replacing this file make LFS work with both options but it defeats the purpose of the SChannel option when LFS is used for a repository. I am not setting any environment variables (except the two mentioned by you for the output below).

LFS fails with:
trace git-lfs: tq: sending batch of size 78
trace git-lfs: api: batch 78 files
trace git-lfs: HTTP: POST https://xxxxx/yyyyy/zzzzz.git/info/lfs/objects/batch

POST /yyyyy/zzzzz.git/info/lfs/objects/batch HTTP/1.1
Host: xxxxx
Accept: application/vnd.git-lfs+json; charset=utf-8
Content-Length: 6902
Content-Type: application/vnd.git-lfs+json; charset=utf-8
User-Agent: git-lfs/2.0.2 (GitHub; windows amd64; go 1.8; git 85e2aec4)

trace git-lfs: api error: Post https://xxxxx/yyyyy/zzzzz.git/info/lfs/objects/batch: x509: certificate signed by unknown authority

Git LFS: (0 of 578 files) 0 B / 796.48 MB
batch response: Post https://xxxxx/yyyyy/zzzzz.git/info/lfs/objects/batch: x509: certificate signed by unknown authority

[dscho]

@hsattler again, this is most likely a problem with Git-LFS, and you are more likely to get answers there about the OpenSSL vs Secure Channel support. We do not even build Git-LFS in the Git for Windows project...

[hsattler]

My suggestions if LFS is not going to fix this:

• A warning should appear in the installer, or
• If LFS install item is selected, do not offer the winssl option.

[hsattler]

I opened a git-lfs issue: git-lfs/git-lfs#2208

[dscho]

Yes, opening that ticket should have come before making plans in case Git-LFS does not fix that: you need to give them a chance to know about the problem in the first place.

[hsattler]

It seems that http.sslcainfo config item is the cause of the problem.

Can the git-for-windows installer omit this setting when installing cURL that uses Windows Certificate Store?

Or maybe install both cURL versions and switch between them like git lfs does?

[dscho]

Can the git-for-windows installer omit this setting when installing cURL that uses Windows Certificate Store?

Sure. It should also remove it in case that the user chose Secure Channel.

Maybe you want to give that a try? It'd be relatively easy: first, make your first custom installer, then modify /usr/src/build-extra/installer/install.iss somewhere around this line: https://github.com/git-for-windows/build-extra/blob/87224a59a550ef324b09d173e7a481caa324436b/installer/install.iss#L1830-L1832 (I imagine an if [Secure channel enabled] then ... else before that line would do the job).

[dscho]

Or maybe install both cURL versions and switch between them like git lfs does?

I forgot to answer this question. It would be possible to install both, but that would not solve your problem: Git LFS does not use cURL. Instead, it uses the presence of the file indicated by http.sslcainfo for a sign that it should tell the Go runtime to use those certificates, and it is a bug in the Go runtime that it ignores the Windows Certificate Store whenever any certificates are configured explicitly.

In my previous comment, I indicated how to fix your problem. Please give it a try.

[hsattler]

I will try to tackle it after my vacation => July. Thanks for the hints.

IMHO, ignoring the Windows Certificate Store when setting a set of wanted CAs is the right thing to do. Some kind of HPKP.

[dscho]

@hsattler it's October now. Should we just close this as a WontFix?

[hsattler]

Currently, I do not have the time to look at this. I just remove the config line after installation. But it's still wrong that the installer puts it there and thus make git lfs fail. But if you are not going to change this, marking this bug "won't fix" is probably the right thing.

[dscho]

Closing this bug in favor of #1409.