Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Git could not be found when running in electron #96

Open
waltonseymour opened this issue May 27, 2017 · 36 comments
Open

Git could not be found when running in electron #96

waltonseymour opened this issue May 27, 2017 · 36 comments

Comments

@waltonseymour
Copy link

waltonseymour commented May 27, 2017

Git is installed, but am unable to use dugite in an electron-renderer process.

Git could not be found. This is most likely a problem in dugite itself

@shiftkey
Copy link
Member

@waltonseymour apologies for the delay on this. There's an internal check that if we don't find Git at the expected path, but of course we don't actually include that path in the error message 🤦‍♂️ .

We expect Git to be installed atnode_modules/dugite/git/ relative to your application root, but the subdirectory path to the git executable depend on the platform you're running on.

I've opened #96 to add this diagnostic message, and 1.29.0 is up on NPM which has this change. Let me know how this changes the behaviour for you.

I expect there's some extra work here (like how we move the Git install around at package time) that you might need to do, but this should be a good first step.

@waltonseymour
Copy link
Author

Hey @shiftkey thanks for getting back to me.

I checked in node_modules/dugite/git/ and the binary is actually there, so I'm not sure what the exact issue is. It works fine if I run it from the electron main process.
Additionally, I'm able to use child_process to shell out to other commands in the electron-renderer process.

I'm assuming something is wrong with the path to the binary in the renderer process, whether it be webpack or something else.

@shiftkey
Copy link
Member

I'm assuming something is wrong with the path to the binary in the renderer process, whether it be webpack or something else.

Let me know what error you do see. A recent change in #85 introduced a LOCAL_GIT_DIRECTORY environment variable as a way to override the defaults and point to an external Git install - not sure if that's relevant here.

@gengjiawen
Copy link

I run into this problem too, but the official github desktop is working fine. I doubt it is caused by packager (using electron-builder) config error. But I can't find it for now.

@shiftkey
Copy link
Member

@gengjiawen this is probably because we explicitly copy the package into the output folder before packaging, and then set the environment variable in the renderer process.

For other setups or tools, I'm happy to provide some documentation and guidance but I need examples that I can work with so I can ensure I'm documenting the right setup.

@gengjiawen
Copy link

If you are interested, you can check this repo, https://github.com/gengjiawen/desktop. You can fork this repo and add those two lines back, gengjiawen/desktop@6b156ad. If you run the app, clone a repo, it's will raise a ENOENT error indicating the git exe is not exist, but it actually exist.

@gengjiawen
Copy link

If you want to check the exe only, you can download binary here, https://github.com/gengjiawen/desktop/releases. I have remove the minify in production mode. You can check the log in develop tools.

@gengjiawen
Copy link

From my observation, if i bundle the git using build.js, git clone and git config set seems not working, but the git executable does exist, but got ENOENT exception executing git spawn.When you open an existing repo, it seems that git blame and git pull working.

Today I tried remove the copy phase, since it's a dependency. Electron-builder will automatically bundle it.
But the app can't the git executable. Maybe the path is wrong. I will update the version to latest and see what happened.
image

@gengjiawen
Copy link

gengjiawen commented Jun 18, 2017

I change a little source code to debug. I find it even if the process.env.LOCAL_GIT_DIRECTORY path not exist, it still use it. But the hot reloading is not working on this project (a little complicated for me). I will do more test using vue with electron.

function resolveGitDir() {
    if (process.env.LOCAL_GIT_DIRECTORY) {
      console.log('using local');
      return path.resolve(process.env.LOCAL_GIT_DIRECTORY);
    }
    else {
        const s = path.sep;
        const gitDir = path.resolve(__dirname, '..', '..', 'git')
            .replace(/[\\\/]app.asar[\\\/]/, `${s}app.asar.unpacked${s}`);
        console.log(gitDir);
        return gitDir;
    }
}

@gengjiawen
Copy link

I have write a demo https://github.com/gengjiawen/dugite-vue. And when bundled in dir (run npm run build:dir) dugite not working. npm run dev works fine.

Cloning into 'dugite'...
warning: templates not found //share/git-core/templates
fatal: Unable to find remote helper for 'https'

@shiftkey
Copy link
Member

shiftkey commented Jun 19, 2017

@gengjiawen your repro is related to ASAR being enabled by default when you invoke electron-builder - there are a number of limitations which this introduces on the Node API, and I'm pretty sure this is affecting execFile when we try to invoke it. We've disabled this on Desktop for this exact reason.

I'm still digging into the root cause - there's lots of optimizations I've needed to unwind.

EDIT: here's my changes on top of master to get to this error.

  • corrected the usage of GitProcess.exec to pass in a directory
  • the dev tools aren't shown in the packaged app, bringing that back
  • disabled the removing of console and debugger messages - this helps with debugging

And now we have this error:

Error: spawn ENOTDIR
    at exports._errnoException (util.js:1050:11)
    at ChildProcess.spawn (internal/child_process.js:319:11)
    at exports.spawn (child_process.js:390:9)
    at Object.exports.execFile (child_process.js:155:15)
    at Object.module.(anonymous function) [as execFile] (ELECTRON_ASAR.js:200:20)
    at /Users/shiftkey/src/dugite-vue/build/mac/dugite-vue.app/Contents/Resources/app.asar/node_modules/dugite/build/lib/git-process.js:65:52
    at Promise (<anonymous>)
    at Function.exec (/Users/shiftkey/src/dugite-vue/build/mac/dugite-vue.app/Contents/Resources/app.asar/node_modules/dugite/build/lib/git-process.js:48:16)
    at i.download (file:///Users/shiftkey/src/dugite-vue/build/mac/dugite-vue.app/Contents/Resources/app.asar/dist/electron/renderer.js:1:89425)
    at n (file:///Users/shiftkey/src/dugite-vue/build/mac/dugite-vue.app/Contents/Resources/app.asar/dist/electron/renderer.js:1:8541)

This is something that @paulcbetts had a shot at fixing recently, but I don't think he succeeded. I think the dependency on spawn here is a dealbreaker because of this rule

There are Node APIs that can execute binaries like child_process.exec, child_process.spawn and child_process.execFile, but only execFile is supported to execute binaries inside asar archive.

This is because exec and spawn accept command instead of file as input, and commands are executed under shell. There is no reliable way to determine whether a command uses a file in asar archive, and even if we do, we can not be sure whether we can replace the path in command without side effects.

@gengjiawen
Copy link

The Error: spawn ENOTDIR error occured, I think it's because __dirname is inside asar.
The error fatal: Unable to find remote helper for 'https' I think maybe it's because git-clone can't link the right libexec, since we can see warning templates not found //share/git-core/templates.

@shiftkey
Copy link
Member

shiftkey commented Jun 21, 2017

The Error: spawn ENOTDIR error occured, I think it's because __dirname is inside asar.

Electron will unpack the necessary contents from the ASAR archive that it thinks it needs at runtime. Looking at this current build, it seems like everything is there that it should need:

$ ls -lR dugite-vue.app/Contents/Resources/app.asar.unpacked/node_modules/dugite/git 
total 0
drwxr-xr-x  6 shiftkey  staff  204 21 Jun 10:47 bin
drwxr-xr-x  3 shiftkey  staff  102 21 Jun 10:47 libexec
drwxr-xr-x  3 shiftkey  staff  102 21 Jun 10:47 share

dugite-vue.app/Contents/Resources/app.asar.unpacked/node_modules/dugite/git/bin:
total 11248
-rwxr-xr-x  1 shiftkey  staff  2720036 21 Jun 10:47 git
-rwxr-xr-x  1 shiftkey  staff      100 21 Jun 10:47 git-cvsserver
-rwxr-xr-x  1 shiftkey  staff  1465448 21 Jun 10:47 git-shell
-rwxr-xr-x  1 shiftkey  staff  1564536 21 Jun 10:47 git-upload-pack

dugite-vue.app/Contents/Resources/app.asar.unpacked/node_modules/dugite/git/libexec:
total 0
drwxr-xr-x  49 shiftkey  staff  1666 21 Jun 10:47 git-core

dugite-vue.app/Contents/Resources/app.asar.unpacked/node_modules/dugite/git/libexec/git-core:
total 65976
-rwxr-xr-x   1 shiftkey  staff  2720036 21 Jun 10:47 git
-rwxr-xr-x   1 shiftkey  staff      100 21 Jun 10:47 git-add--interactive
-rwxr-xr-x   1 shiftkey  staff      100 21 Jun 10:47 git-archimport
-rwxr-xr-x   1 shiftkey  staff    15984 21 Jun 10:47 git-bisect
-rwxr-xr-x   1 shiftkey  staff  1470264 21 Jun 10:47 git-credential-cache
-rwxr-xr-x   1 shiftkey  staff  1483192 21 Jun 10:47 git-credential-cache--daemon
-rwxr-xr-x   1 shiftkey  staff  1478288 21 Jun 10:47 git-credential-store
-rwxr-xr-x   1 shiftkey  staff      100 21 Jun 10:47 git-cvsexportcommit
-rwxr-xr-x   1 shiftkey  staff      100 21 Jun 10:47 git-cvsimport
-rwxr-xr-x   1 shiftkey  staff      100 21 Jun 10:47 git-cvsserver
-rwxr-xr-x   1 shiftkey  staff  1485544 21 Jun 10:47 git-daemon
-rwxr-xr-x   1 shiftkey  staff     2244 21 Jun 10:47 git-difftool--helper
-rwxr-xr-x   1 shiftkey  staff  1532020 21 Jun 10:47 git-fast-import
-rwxr-xr-x   1 shiftkey  staff    12913 21 Jun 10:47 git-filter-branch
-rwxr-xr-x   1 shiftkey  staff  1481640 21 Jun 10:47 git-http-backend
-rwxr-xr-x   1 shiftkey  staff  1611632 21 Jun 10:47 git-http-fetch
-rwxr-xr-x   1 shiftkey  staff  1625360 21 Jun 10:47 git-http-push
-rwxr-xr-x   1 shiftkey  staff  1622208 21 Jun 10:47 git-imap-send
-rwxr-xr-x   1 shiftkey  staff      100 21 Jun 10:47 git-instaweb
-rwxr-xr-x   1 shiftkey  staff  7762004 21 Jun 10:47 git-lfs
-rwxr-xr-x   1 shiftkey  staff     2477 21 Jun 10:47 git-merge-octopus
-rwxr-xr-x   1 shiftkey  staff     3707 21 Jun 10:47 git-merge-one-file
-rwxr-xr-x   1 shiftkey  staff      944 21 Jun 10:47 git-merge-resolve
-rwxr-xr-x   1 shiftkey  staff    10303 21 Jun 10:47 git-mergetool
-rw-r--r--   1 shiftkey  staff     8869 21 Jun 10:47 git-mergetool--lib
-rwxr-xr-x   1 shiftkey  staff   143085 21 Jun 10:47 git-p4
-rw-r--r--   1 shiftkey  staff     2650 21 Jun 10:47 git-parse-remote
-rwxr-xr-x   1 shiftkey  staff     3596 21 Jun 10:47 git-quiltimport
-rwxr-xr-x   1 shiftkey  staff    16280 21 Jun 10:47 git-rebase
-rw-r--r--   1 shiftkey  staff     2471 21 Jun 10:47 git-rebase--am
-rw-r--r--   1 shiftkey  staff    35663 21 Jun 10:47 git-rebase--interactive
-rw-r--r--   1 shiftkey  staff     3933 21 Jun 10:47 git-rebase--merge
-rwxr-xr-x   1 shiftkey  staff  1624888 21 Jun 10:47 git-remote-http
-rwxr-xr-x   1 shiftkey  staff  1496208 21 Jun 10:47 git-remote-testsvn
-rwxr-xr-x   1 shiftkey  staff     3698 21 Jun 10:47 git-request-pull
-rwxr-xr-x   1 shiftkey  staff      100 21 Jun 10:47 git-send-email
-rw-r--r--   1 shiftkey  staff     2374 21 Jun 10:47 git-sh-i18n
-rwxr-xr-x   1 shiftkey  staff  1465520 21 Jun 10:47 git-sh-i18n--envsubst
-rw-r--r--   1 shiftkey  staff     9252 21 Jun 10:47 git-sh-setup
-rwxr-xr-x   1 shiftkey  staff  1465448 21 Jun 10:47 git-shell
-rwxr-xr-x   1 shiftkey  staff  1465200 21 Jun 10:47 git-show-index
-rwxr-xr-x   1 shiftkey  staff    15609 21 Jun 10:47 git-stash
-rwxr-xr-x   1 shiftkey  staff    28086 21 Jun 10:47 git-submodule
-rwxr-xr-x   1 shiftkey  staff      100 21 Jun 10:47 git-svn
-rwxr-xr-x   1 shiftkey  staff  1564536 21 Jun 10:47 git-upload-pack
-rwxr-xr-x   1 shiftkey  staff     4401 21 Jun 10:47 git-web--browse
drwxr-xr-x  27 shiftkey  staff      918 21 Jun 10:47 mergetools

dugite-vue.app/Contents/Resources/app.asar.unpacked/node_modules/dugite/git/libexec/git-core/mergetools:
total 200
-rw-r--r--  1 shiftkey  staff  358 21 Jun 10:47 araxis
-rw-r--r--  1 shiftkey  staff  367 21 Jun 10:47 bc
-rw-r--r--  1 shiftkey  staff   24 21 Jun 10:47 bc3
-rw-r--r--  1 shiftkey  staff  353 21 Jun 10:47 codecompare
-rw-r--r--  1 shiftkey  staff  663 21 Jun 10:47 deltawalker
-rw-r--r--  1 shiftkey  staff  309 21 Jun 10:47 diffmerge
-rw-r--r--  1 shiftkey  staff  248 21 Jun 10:47 diffuse
-rw-r--r--  1 shiftkey  staff  306 21 Jun 10:47 ecmerge
-rw-r--r--  1 shiftkey  staff  438 21 Jun 10:47 emerge
-rw-r--r--  1 shiftkey  staff  336 21 Jun 10:47 examdiff
-rw-r--r--  1 shiftkey  staff   29 21 Jun 10:47 gvimdiff
-rw-r--r--  1 shiftkey  staff   29 21 Jun 10:47 gvimdiff2
-rw-r--r--  1 shiftkey  staff   29 21 Jun 10:47 gvimdiff3
-rw-r--r--  1 shiftkey  staff  522 21 Jun 10:47 kdiff3
-rw-r--r--  1 shiftkey  staff  117 21 Jun 10:47 kompare
-rw-r--r--  1 shiftkey  staff  843 21 Jun 10:47 meld
-rw-r--r--  1 shiftkey  staff  267 21 Jun 10:47 opendiff
-rw-r--r--  1 shiftkey  staff  617 21 Jun 10:47 p4merge
-rw-r--r--  1 shiftkey  staff  258 21 Jun 10:47 tkdiff
-rw-r--r--  1 shiftkey  staff  602 21 Jun 10:47 tortoisemerge
-rw-r--r--  1 shiftkey  staff  890 21 Jun 10:47 vimdiff
-rw-r--r--  1 shiftkey  staff   29 21 Jun 10:47 vimdiff2
-rw-r--r--  1 shiftkey  staff   29 21 Jun 10:47 vimdiff3
-rw-r--r--  1 shiftkey  staff  361 21 Jun 10:47 winmerge
-rw-r--r--  1 shiftkey  staff  584 21 Jun 10:47 xxdiff

dugite-vue.app/Contents/Resources/app.asar.unpacked/node_modules/dugite/git/share:
total 0
drwxr-xr-x  3 shiftkey  staff  102 21 Jun 10:47 git-core

dugite-vue.app/Contents/Resources/app.asar.unpacked/node_modules/dugite/git/share/git-core:
total 0
drwxr-xr-x  6 shiftkey  staff  204 21 Jun 10:47 templates

dugite-vue.app/Contents/Resources/app.asar.unpacked/node_modules/dugite/git/share/git-core/templates:
total 8
drwxr-xr-x   2 shiftkey  staff   68 21 Jun 10:47 branches
-rw-r--r--   1 shiftkey  staff   73 21 Jun 10:47 description
drwxr-xr-x  12 shiftkey  staff  408 21 Jun 10:47 hooks
drwxr-xr-x   3 shiftkey  staff  102 21 Jun 10:47 info

dugite-vue.app/Contents/Resources/app.asar.unpacked/node_modules/dugite/git/share/git-core/templates/branches:

dugite-vue.app/Contents/Resources/app.asar.unpacked/node_modules/dugite/git/share/git-core/templates/hooks:
total 88
-rwxr-xr-x  1 shiftkey  staff   478 21 Jun 10:47 applypatch-msg.sample
-rwxr-xr-x  1 shiftkey  staff   896 21 Jun 10:47 commit-msg.sample
-rwxr-xr-x  1 shiftkey  staff   189 21 Jun 10:47 post-update.sample
-rwxr-xr-x  1 shiftkey  staff   424 21 Jun 10:47 pre-applypatch.sample
-rwxr-xr-x  1 shiftkey  staff  1642 21 Jun 10:47 pre-commit.sample
-rwxr-xr-x  1 shiftkey  staff  1348 21 Jun 10:47 pre-push.sample
-rwxr-xr-x  1 shiftkey  staff  4951 21 Jun 10:47 pre-rebase.sample
-rwxr-xr-x  1 shiftkey  staff   544 21 Jun 10:47 pre-receive.sample
-rwxr-xr-x  1 shiftkey  staff  1239 21 Jun 10:47 prepare-commit-msg.sample
-rwxr-xr-x  1 shiftkey  staff  3610 21 Jun 10:47 update.sample

dugite-vue.app/Contents/Resources/app.asar.unpacked/node_modules/dugite/git/share/git-core/templates/info:
total 8
-rw-r--r--  1 shiftkey  staff  240 21 Jun 10:47 exclude

And just before launching git.exe I can see that it's location is inside app.asar.unpacked:

screen shot 2017-06-21 at 11 19 18 am

And then I get the error with the ERROR spawn ENOTDIR.

The error fatal: Unable to find remote helper for 'https' I think maybe it's because git-clone can't link the right libexec

That's this code, which looks fine for Linux.

since we can see warning templates not found //share/git-core/templates.

The // here suggests that something with the environment variables isn't rooted correctly (this code), but this is just a warning and shouldn't affect the cloning process.

@gengjiawen
Copy link

gengjiawen commented Jun 21, 2017

I don't quite follow you, If i replace the __dirname with a valid dir, I get the error fatal: Unable to find remote helper for 'https', not ERROR spawn ENOTDIR.

And if you go to the dir like dugite-vue/build/mac/dugite-vue.app/Contents/Resources/app.asar.unpacked/node_modules/dugite/git/bin, and execute ./git clone https://github.com/desktop/dugite. You get the same error fatal: Unable to find remote helper for 'https'.
So I still suspect there is something wrong with linking libexec path.

@shiftkey
Copy link
Member

I don't quite follow you, If i replace the __dirname with a valid dir, I get the error fatal: Unable to find remote helper for 'https', not ERROR spawn ENOTDIR.

Aha, this is the bit I was missing. I assumed __dirname would be a valid path but switching it over to os.tmpdir() now has me at this same error. Thanks for clarifying!

And if you go to the dir like dugite-vue/build/mac/dugite-vue.app/Contents/Resources/app.asar.unpacked/node_modules/dugite/git/bin, and execute ./git clone https://github.com/desktop/dugite. You get the same error fatal: Unable to find remote helper for 'https'.

This is because you need to set GIT_EXEC_PATH to the root of the Git installation before executing Git. That's the expected behaviour, but we're on the right track now...

@shiftkey
Copy link
Member

Looks like the ASAR unpacking is ignoring shortcuts. This is from the unpacked dugite-native archive:

screen shot 2017-06-21 at 12 02 22 pm

And this is from the unpacked git-core directory inside asar.app.unpacked:

dugite-vue.app/Contents/Resources/app.asar.unpacked/node_modules/dugite/git/libexec/git-core:
...
-rw-r--r--   1 shiftkey  staff    35663 21 Jun 10:47 git-rebase--interactive
-rw-r--r--   1 shiftkey  staff     3933 21 Jun 10:47 git-rebase--merge
-rwxr-xr-x   1 shiftkey  staff  1624888 21 Jun 10:47 git-remote-http
-rwxr-xr-x   1 shiftkey  staff  1496208 21 Jun 10:47 git-remote-testsvn
-rwxr-xr-x   1 shiftkey  staff     3698 21 Jun 10:47 git-request-pull
...

@gengjiawen
Copy link

gengjiawen commented Jun 21, 2017

You won't believe it, change the repo url to http, and everything works...

@shiftkey
Copy link
Member

@gengjiawen cool - sounds like this should all just work if we can tell the ASAR packing to also include shortcuts!

@gengjiawen
Copy link

I am not following you again, https still not works for now. Why would http and https works on npm run dev, but https not works on npm run build:dir.

@gengjiawen
Copy link

I see, it's not bunding git-remote-https.

@shiftkey
Copy link
Member

I see, it's not bunding git-remote-https.

Precisely. Digging into whether this is a limitation of asar, or whether electron-builder is just ignoring these (for whatever reason).

@shiftkey
Copy link
Member

@gengjiawen and here's the issue - electron-userland/electron-builder#1376

@gengjiawen
Copy link

gengjiawen commented Jun 21, 2017

set "asar": false works :)

@shiftkey
Copy link
Member

@gengjiawen yes, and that's what we're doing in Desktop currently. I'd love to fix this properly so we can enable this behaviour, so I'm gonna go down this rabbit hole and see if I can propose a patch to electron-builder.

@gengjiawen
Copy link

As you might know, I have fork the desktop and change the packager to electron-builder to use travis to build linux distro. To achieve this , do i need to comment this line :

process.env['LOCAL_GIT_DIRECTORY'] = Path.resolve(__dirname, 'git')

@shiftkey
Copy link
Member

@gengjiawen I'm not sure if you can remove it, but it's probably the wrong path for your setup

@shiftkey
Copy link
Member

@gengjiawen electron-packager seems to work as expected (it bundles the git-remote-https shortcut too).

@gengjiawen
Copy link

Okay, but i will stay with electron-builder since it has many convenient feature, such as publish to github release and aws.

gengjiawen added a commit to gengjiawen/dugite-vue that referenced this issue Jun 21, 2017
@gengjiawen
Copy link

gengjiawen commented Jun 21, 2017

Hey, I find that on desktop, You need to change the source code this to make it it work, it looks like the __dirname is different frome vue-dugite. Any thought or do we need a fix ?

function resolveGitDir() {
    if (process.env.LOCAL_GIT_DIRECTORY) {
      console.log('using env');
      return path.resolve(process.env.LOCAL_GIT_DIRECTORY);
    }
    else {
      console.log(`using guess dirname ${__dirname}`);
      const s = path.sep;
      const dir = path.resolve(__dirname, '..', 'node_modules', 'dugite', 'git').replace(/[\\\/]app.asar[\\\/]/, `${s}app.asar.unpacked${s}`);
      console.log(dir);
      return dir;
    }
}

@shiftkey
Copy link
Member

@gengjiawen so the problem isn't with either packager, but with the asar library itself: electron/asar#76 (comment)

@shiftkey
Copy link
Member

Any thought or do we need a fix ?

Are you running this with asar enabled?

@gengjiawen
Copy link

nope

@shiftkey
Copy link
Member

Are you moving Git or the related JS files around during your build process? What's __dirname in your situation? If that's something strange then I'd recommend setting LOCAL_GIT_DIRECTORY instead of trusting the defaults.

@gengjiawen
Copy link

I didn't move anything.

In dugite-vue, it's somthing like dugite-vue/build/mac/dugite-vue.app/Contents/Resources/app/node_modules/dugite/build/lib.

In desktop, it's something like desktop/dist/mac/GithubDesktop.app/Contents/Resources/app/dist.

Maybe it's related to webpack config when get __dirname.

Anyway, using LOCAL_GIT_DIRECTORY do fix this.

@gengjiawen
Copy link

If this turns out related to webpack config. I think we may need to fix this.

@tombowers
Copy link

I was getting a similar error - Git could not be found at the expected path, building electron with webpack using the electron-renderer target. This is due to the __dirname variable being set incorrectly. I fixed it by adding the following setting to my webpack config.

node: {
    __dirname: true
}

Webpack mocks __dirname by default resulting in just a /. True compiles __dirname to a string of the context path, and false leaves __dirname as it is in node.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants