Error downloading git repo over HTTP without a commit hash

[krzysieki]

Yarn version: 0.17.10

I want to report a bug.

Current behavior
An attempt to install dependency from a git repo with specified branch, fails with message:

error Refusing to download the git repo http://<xxxxxxxxxxxx>.git over HTTP without a commit hash

Steps to reproduce: call yarn in project with package.json containing:

...
"devDependencies": {
    "name.of.the.dependency": "git+http://url.of.the.dependency/git.repo.name.git#develop"
  },
...

or call command:
yarn add git+http://url.of.the.dependency/git.repo.name.git#develop

Expected behavior
Dependency specified by git url should be installed. Branch specified after # should be used.
Command:
yarn upgrade git+http://url.of.the.dependency/git.repo.name.git#develop
Should install dependency using latest commit from a specified branch (skipping yarn cache)

Additional info
In yarn@0.16.* command yarn add worked as expected in specified scenario.

[mrapczynski]

For private packages internal to our organization, publishing to an NPM repo adds unnecessary overhead. We definitely make heavy use of Git repos for long term storage. It would be great to have this fixed sooner than later, at least to return to parity with npm

[gagoar]

this is happening for me as well with version 0.17.x and 0.18.x

    at MessageError (/usr/lib/node_modules/yarn/lib-legacy/errors.js:8:10)
    at SecurityError (/usr/lib/node_modules/yarn/lib-legacy/errors.js:15:1)
    at /usr/lib/node_modules/yarn/lib-legacy/util/git.js:173:17
    at next (native)
    at step (/usr/lib/node_modules/yarn/node_modules/babel-runtime/helpers/asyncToGenerator.js:17:30)
    at /usr/lib/node_modules/yarn/node_modules/babel-runtime/helpers/asyncToGenerator.js:28:13
    at run (/usr/lib/node_modules/yarn/node_modules/babel-runtime/node_modules/core-js/library/modules/es6.promise.js:87:22)
    at /usr/lib/node_modules/yarn/node_modules/babel-runtime/node_modules/core-js/library/modules/es6.promise.js:100:28
    at flush (/usr/lib/node_modules/yarn/node_modules/babel-runtime/node_modules/core-js/library/modules/_microtask.js:18:9)
    at nextTickCallbackWith0Args (node.js:420:9)
error Refusing to download the git repo http://ikt.pm2.io/ikt.git over HTTP without a commit hash```

[anatolii-zolotar]

Excuse me, is there any solution except of specifying dependency version?
I have similar definition in package.json dependencies:
"foobar": "git+http://customserver.net/path/to/public/repo.git"
and it works only if I append commit hash. But I don't think that it's acceptable in my case.

[fc]

The error may be misleading... if you inspect the error it could be a slightly different problem:

{ Error: Command failed.
Exit code: 128
Command: git
Arguments: ls-remote -t https://ikt.pm2.io/ikt.git
Directory: /Users/some-awesome-person/dev/frontend-app/tools/proxy
Output:
fatal: unable to access 'https://ikt.pm2.io/ikt.git/': SSL certificate problem: Invalid certificate chain
    at MessageError (/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/lib/errors.js:8:5)
    at SpawnError (/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/lib/errors.js:18:1)
    at ChildProcess.proc.on.code (/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/lib/util/child.js:107:15)
    at emitTwo (events.js:106:13)
    at ChildProcess.emit (events.js:191:7)
    at maybeClose (internal/child_process.js:852:16)
    at Socket.<anonymous> (internal/child_process.js:323:11)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at Pipe._handle.close [as _onclose] (net.js:492:12) code: undefined, EXIT_CODE: 128 }
error Refusing to download the git repo http://ikt.pm2.io/ikt.git over HTTP without a commit hash

The above output comes from here:
https://github.com/yarnpkg/yarn/blob/master/src/util/git.js#L91

So I set git config --global http.sslVerify false and run again to get a new error (remember to re-enable this afterward:

fatal: repository 'https://ikt.pm2.io/ikt.git/' not found
...

It appears yarn is converting anything that is http to https which may not always be desired here:
https://github.com/yarnpkg/yarn/blob/master/src/util/git.js#L118

By yarn automatically converting http to https it causes the error for me and maybe for you too.

A PR would just need to remove the auto conversion to https before it checks that the git repos exists but still check the git repos exists whether it is http or https.

[fabiosantoscode]

This is still a standing issue :( I've been affected many times thus far and only now decided to look for the issue. Does anyone have any news?

[fc]

@fabiosantoscode it looks like the error is triggered here:

yarn/src/util/git.js

Line 151 in 67dfc7a

throw new SecurityError(reporter.lang('refusingDownloadHTTPWithoutCommit', ref));

It looks like it is likely executing this command (which does not fail for me when running on the command line):

git ls-remote -t https://github.com/mjohnston/scroller

You could put a trace statement here to see what actual error is being generated:

yarn/src/util/git.js

Line 113 in 67dfc7a

return false;

I would just specify the hash if you can though.

[dancrumb]

Still present in version 0.27.5

[fabiosantoscode]

Specifying the hash would pin the commit forever, and in a project with a dependency in a separate repository you want it to roll forward.

This failure looks like it's by design, as yarn is simply refusing to clone the repository without a hash, which is more secure but not always what one would want.

The alternative, which is to specify a github tarball URL, also only works if you pin the version. Otherwise, a commit comes in, the tarball for master changes, and the hash doesn't match yarn.lock anymore and the install fails.

[daviskoh]

any updates?

[bakso]

+1

[chaliy]

We also have problems with this issue.

[deminoth]

sudo xcodebuild -license resolves it for me.

[BYK]

This should be fixed on 1.0+. Is anyone willing to confirm?

[kronnakrit]

I still got this error

error Refusing to download the git repo {"hostname":"bitbucket.org","protocol":"https:","repository":"https://git@bitbucket.org:xxxx/xxxx.git"} over HTTPS without a commit hash - possible certificate error?

v1.1.0

[Neats29]

+1

[reergymerej]

FWIW, I had the same issue until I changed my password to something without #.

[ianp]

@BYK I can confirm that this is still present in 1.2.0 (also, changing passwords didn't do anything for us)

[Neats29]

My colleague and I worked through this issue all day so bit hard to pinpoint exactly what fixed it but I believe the following should help.

If you are on macOS Sierra, it might be a SSH agent forwarding issue, so follow the steps bellow:
https://developer.github.com/v3/guides/using-ssh-agent-forwarding/

If you get to the end and running /usr/bin/ssh-add -K yourkey asks you for a passphrase you may have to generate a new SSH and pass a passphrase and try this again (don't forget to update the new key on github)

Then test to see if it has worked using:
https://help.github.com/articles/error-permission-denied-publickey/#make-sure-you-have-a-key-that-is-being-used

Now try yarn upgrade <private repo>

if you get this error: fatal: could not read Username for 'https://github.com': terminal prompts disabled

follow these steps:
https://stackoverflow.com/a/34885303/4548085

ps. I'm using yarn version 1.2.0. Hope this helps.

[musti2]

Still an issue with v.1.2.0

[BorntraegerMarc]

also happens when running yarn with docker container node:6.11.1-alpine
yarn version: 1.1.0

[jeffmwells]

+1

[BYK]

So I think the current behavior was intentional due to security concerns. Maybe we should understand why this is needed in the first place?

[japboy]

@BorntraegerMarc it probably happens because alpine doesn't have git package as default.

this might help you:

RUN apk update && \
    apk upgrade && \
    apk add --no-cache git

[teodorpatras]

If you're on OSX, xcode-select --install will most probably fix the problem. I just updated from Sierra to High Sierra and had the same issue. Running the previous command fixed it for me.

[lost-valley]

My team had a similar issue, and in our case it was caused by a colon instead of slash in the package.json. Strange thing is, that the "colon version" worked for quite a time in our codebase.

Error:
"my-repo": "git+https://user:pass@bitbucket.org:my-account/my-repo.git#2.0.0"

Ok:
"my-repo": "git+https://user:pass@bitbucket.org/my-account/my-repo.git#2.0.0"

[adjavaherian]

This doesn't seem to be a problem on my localhost, but when i build in a container (node:carbon) its an issue. In this case, you can copy your yarn.lock into the container and it should have the hashes for the sem-versions.

[yairopro]

I'm still having the issue.
My project have got protected dependencies (by proptected I mean it needs authentication usename/password).
In order to add protected projects using yarn, I've found a solution:
Deleting the yarn.lock & adding the git dependency's url with a username and his password like this https://username:password@project-host.com/path/to/project.git inside the package.json.
But as you can guess, it's not acceptable to write a the username with its password in the package.json.

When cloning protected projects, git ask user to enter his username & his password:

$ git clone https://project-host.com/path/to/project.git
Cloning into 'project'...
Username for 'https://bitbucket.org': 
Password for 'https://myUsername@bitbucket.org': 

Why yarn can't do this?

Version 1.3.2

[bradisbell]

Adding to this... it seems that Yarn on Windows uses its own Git rather than what's in the path. Therefore, if you've configured Git to use PuTTY and its agent, you can't expect Yarn to work this way. This has thrown the error in question for me.

[ChadTaljaardt]

We have just received the same error, and we can do git clone just fine on that repo.

[yairopro]

@bradisbell I didn't configured Git to use PuTTY. If you see my post, in my question, I'am using an https url. Furthermore, this is the default git behavior when you clone private repo (protected with credentials I mean). I didn't make any configuration. So it should work this way even in yarn's own git.

[jeppe-style]

I had a similar issue with a private repo over HTTPS. Instead of adding the username and password to the URL I added the username to the global .gitconfig (default ~/.gitconfig on Mac) like so:

[credential "https://example.com"]
	username = me

It is documented under gitcredentials. The first time it is used it will ask for the password, so by first cloning the repo and entering the password it can be saved for the future in the Keychain (not sure for other systems). Then it worked without a problem with yarn using #branch.

[benzen9113]

Stil has no workaround but should append commit hash to url?

[pameljoel]

I fixed it on my mac os machine by changing the node version to 8

[ancadenisa]

I had this kind of error (not the text after "Fatal")
.
Solved by updating to the latest git version.

[DavitVosk]

I was getting such kind of problem when trying to yarn add my any private repository