bash on windows has error #778

freshcn · 2017-03-08T17:17:02Z

I use glide in bash on windows. when i exec glide install.i got the follow error message

[INFO]  --> Exporting golang.org/x/net
[INFO]  --> Exporting golang.org/x/crypto/acme/autocert
[INFO]  --> Exporting golang.org/x/crypto
[INFO]  Replacing existing vendor dependencies
[ERROR] remove /tmp/glide-vendor220918663/vendor/github.com/google/go-github/github: directory not empty
[ERROR] Unable to export dependencies to vendor directory: remove /mnt/c/Users/Administrator/go/src/github.com/freshcn/go-id-builder/vendor/github.com/google/go-github/github: directory not empty

there has the same error on v0.12.3 and 0.13.0-dev

The text was updated successfully, but these errors were encountered:

bacongobbler · 2017-03-10T21:41:30Z

do you have an example glide.yaml and glide.lock that can reproduce this failure?

freshcn · 2017-03-11T09:02:02Z

this is my glide.yaml and glide.lock. they can work on ubuntu and mac.

glide.yaml

package: git.coding.net/dolist/api.git
import:
- package: github.com/freshcn/async
- package: github.com/freshcn/log
- package: github.com/freshcn/tools
  subpackages:
  - uuid
  - vaildate
- package: github.com/go-sql-driver/mysql
  version: ^1.2.0
- package: github.com/go-xorm/core
  version: ^0.5.6
- package: github.com/go-xorm/xorm
  version: ^0.5.6
- package: github.com/namsral/flag
- package: gopkg.in/ini.v1
  version: ^1.21.1
- package: gopkg.in/iris-contrib/middleware.v5
  version: ^5.0.0
  subpackages:
  - logger
- package: gopkg.in/kataras/iris.v5
  version: ^5.0.0
- package: gopkg.in/redis.v5
  version: ^5.0.1

glide.lock

hash: f3c7280548e096dae033969bab54d1ebd8925c7bb7bd1758ba5b223b78b5f9f0
updated: 2017-02-18T02:01:18.477596607+08:00
imports:
- name: github.com/fatih/structs
  version: dc3312cb1a4513a366c4c9e622ad55c32df12ed3
- name: github.com/freshcn/async
  version: c3953276c8fc222c13d454dee39e2eed6ba3f5ed
- name: github.com/freshcn/log
  version: 98911aaee264b8c69ffa041c63a71574d951c8bc
- name: github.com/freshcn/tools
  version: cdcebbb7db5ed8db845c644c4e9197e11778a0d5
  subpackages:
  - uuid
  - vaildate
- name: github.com/geekypanda/httpcache
  version: 76ba6c68462ae362cda7564c44492b95322b363a
  subpackages:
  - internal
  - internal/fhttp
  - internal/fhttp/rule
  - internal/nethttp
  - internal/nethttp/rule
  - internal/server
- name: github.com/go-sql-driver/mysql
  version: a0583e0143b1624142adab07e0e97fe106d99561
- name: github.com/go-xorm/builder
  version: 867edcc549127f15667ece8876c3a60093c9634b
- name: github.com/go-xorm/core
  version: 5bf745d7d163f4380e6c2bba8c4afa60534dd087
- name: github.com/go-xorm/xorm
  version: d75356fc733fce7683c6d961fc2aeb16ededc8ef
- name: github.com/google/go-github
  version: f7fcf6f52ff94adf1cc0ded41e7768d2ad729972
  subpackages:
  - github
- name: github.com/google/go-querystring
  version: 9235644dd9e52eeae6fa48efd539fdc351a0af53
  subpackages:
  - query
- name: github.com/gorilla/websocket
  version: 2aff870ef83507abb46472eae82449f7a4c464c5
- name: github.com/hashicorp/go-version
  version: e96d3840402619007766590ecea8dd7af1292276
- name: github.com/imdario/mergo
  version: 50d4dbd4eb0e84778abe37cefef140271d96fade
- name: github.com/iris-contrib/formBinder
  version: 81b6a071e35797b83562caf4b3cad24dc01912dc
- name: github.com/iris-contrib/lego
  version: 095d7f6459c501cb15319aa2754afa221b81a3ec
  subpackages:
  - acme
- name: github.com/iris-contrib/letsencrypt
  version: 1a3e5c619a13b307df3b1b4da7cb7e57d2e156dd
- name: github.com/kataras/go-errors
  version: 0f977b82cc78d5d31bb75fb6f903ad9e852c8bbd
- name: github.com/kataras/go-fs
  version: c029e113d9faaf558b730f06041c8bf9545a3502
- name: github.com/kataras/go-options
  version: 23b556c1b935c594ec6d71ff81ead4dbeec3aa8d
- name: github.com/kataras/go-serializer
  version: 0bd874a15c70db74ef2e668e5eeda27041f03b81
  subpackages:
  - data
  - json
  - jsonp
  - markdown
  - text
  - xml
- name: github.com/kataras/go-sessions
  version: 697680208eb3db6a76240f2556eaa7d4909f2556
- name: github.com/kataras/go-template
  version: 457f21178102f4688603eccbb4f2e8d5ae1023bf
  subpackages:
  - html
- name: github.com/klauspost/compress
  version: 8df558b6cb6f9b445f9586446cfe7223e7d8bd6b
  subpackages:
  - flate
  - gzip
  - zlib
- name: github.com/klauspost/cpuid
  version: 09cded8978dc9e80714c4d85b0322337b0a1e5e0
- name: github.com/klauspost/crc32
  version: cb6bfca970f6908083f26f39a79009d608efd5cd
- name: github.com/microcosm-cc/bluemonday
  version: f77f16ffc87a6a58814e64ae72d55f9c41374e6d
- name: github.com/miekg/dns
  version: 58f52c57ce9df13460ac68200cef30a008b9c468
- name: github.com/namsral/flag
  version: 91bb473a91a4a5e1ea554fb8ddce84f832be9aca
- name: github.com/russross/blackfriday
  version: 5f33e7b7878355cd2b7e6b8eefc48a5472c69f70
- name: github.com/satori/go.uuid
  version: b061729afc07e77a8aa4fad0a2fd840958f1942a
- name: github.com/shurcooL/sanitized_anchor_name
  version: 1dba4b3954bc059efc3991ec364f9f9a35f597d2
- name: github.com/square/go-jose
  version: 63d7cfa0284d0bc9bf41d58f802037559c45ce8f
  subpackages:
  - cipher
  - json
- name: github.com/valyala/bytebufferpool
  version: e746df99fe4a3986f4d4f79e13c1e0117ce9c2f7
- name: github.com/valyala/fasthttp
  version: 4ee6517626b0549c4a4c85ad7b91acdec1d2af24
  subpackages:
  - fasthttpadaptor
  - fasthttputil
- name: golang.org/x/crypto
  version: bed12803fa9663d7aa2c2346b0c634ad2dcd43b7
  subpackages:
  - acme/autocert
  - ocsp
- name: golang.org/x/crypto/acme/autocert
  version: bed12803fa9663d7aa2c2346b0c634ad2dcd43b7
- name: golang.org/x/net
  version: 41bba8d80bbfab43231ffdf4c210037baae5f6a3
  subpackages:
  - context
  - html
  - html/atom
  - publicsuffix
- name: golang.org/x/time
  version: f51c12702a4d776e4c1fa9b0fabab841babae631
  subpackages:
  - rate
- name: gopkg.in/ini.v1
  version: e3c2d47c61e5333f9aa2974695dd94396eb69c75
- name: gopkg.in/iris-contrib/middleware.v5
  version: eff973db50aaea60190e75f8f5dc343f7de34b38
  subpackages:
  - logger
- name: gopkg.in/iris-contrib/websocket.v5
  version: bc14117ad25db88dd667f03170dc6db1e5c1f78f
- name: gopkg.in/kataras/go-errors.v0
  version: d2aaffdebcb40c69c4cb6a95e436430cde192435
- name: gopkg.in/kataras/go-fs.v0
  version: ecf3587588f7218a21a991a1c0a9f4bdebd20cc1
- name: gopkg.in/kataras/go-websocket.v0
  version: cb1681badd5582ccaebaf02f452e2d5a98f1a440
- name: gopkg.in/kataras/iris.v5
  version: 547c5360493789dc4f0e65894f14af086c62d03c
  subpackages:
  - utils
- name: gopkg.in/redis.v5
  version: 2fe9c5cc0a8da9faeba3a45511a4ece8f7e37dc2
  subpackages:
  - internal
  - internal/consistenthash
  - internal/hashtag
  - internal/pool
  - internal/proto
testImports: []

mattfarina · 2017-03-21T02:49:28Z

Is this bash installed with git? What windows version and bash version?

joesteele · 2017-04-05T03:37:37Z

I believe this is related to #654.

Using the example docker glide.yaml provided in the comments, I'm able to reproduce this on my Windows machine (on the recent Creator's Update) using the WSL (bash v4.3.11(1)-release).

Using this same glide.yaml on my mac laptop works fine with the same glide and go versions (glide v0.12.3 and go v1.8).

package: .
import:
- package: github.com/docker/docker
  version: master

eliothedeman · 2017-04-16T09:51:44Z

I see the same thing running WSL on kernal 4.4.0-43-Microsoft on most recent update of windows 10 glide 0.12.3 and go 1.8.1

djensen47 · 2017-04-26T06:05:58Z

Is this bash installed with git? What windows version and bash version?

No, this is on "Bash on Ubuntu for Windows" also known as Windows Subsystem for Linux (WSL).

It looks like some folks are seeing this issue on Vagrant as well.

kumarharsh · 2017-04-27T14:47:44Z

I get this same error on powershell too. It's not a WSL problem, but rather something to do with windows filesystem locking some file while glide is doing it's thing. And it's always random repositories, it's not just due to any one dependency. So there's not real way to stop this from happening apart from this:

rm -r ./vendor; glide install

which sucks!

thomastaylor312 · 2017-05-21T08:40:43Z

I was able to duplicate this problem as well when doing a glide install in the Helm repo using WSL. Mine seems to hang on the same repo every time /tmp/glide-vendor123664865/vendor/golang.org/x/sys/unix

vidaoaaron · 2017-06-02T04:35:25Z

Also seeing this consistently on the same repo

$ glide -v
glide version 0.13.0-dev

[ERROR] remove /tmp/glide-vendor361549384/vendor/google.golang.org/api: directory not empty                                          
[ERROR] Unable to export dependencies to vendor directory: remove /root/gopath/src/github.com/xx/yy/vendor/google.golang.org/api: directory not empty

kumarharsh · 2017-06-05T06:55:19Z

Replying to #778 (comment), @mattfarina this happens even with the latest Git for Windows version (2.13.0 at the time of writing this), and with powershell or cmd or git bash (mingw64 bash). Glide version is the latest (but it's still lacking those windows fixes merged into master) (0.12.1 at the time of writing).

To easily reproduce the issue, running a simple glide update after glide install on this glide.yaml should throw the error on Windows.

djensen47 · 2017-06-07T12:39:02Z

I'm using glide version v0.12.1-80-g4b407b3 from master and I'm still seeing these issues.

BhallalDev · 2017-06-07T14:02:01Z

I'm also using glide version v0.12.3 from master and facing this issue consistently

djensen47 · 2017-06-07T19:56:49Z

Are there any workarounds?

I noticed this only happens for me if a dependency that I'm importing has files in it's /vendor dir.

djensen47 · 2017-06-07T21:49:15Z

Also, I know that other folks said they couldn't get this working on Windows with Git Bash. After clearing my glide cache, it worked for me on Git Bash!

Should we file something against microsoft/BashOnWindows?

I ask because in the early days of WSL (BashOnWindows) there was an issue in nodejs where you could not watch files and it turned out to be a WSL issue IIRC related to file descriptors.

kumarharsh · 2017-06-08T07:39:17Z

Should we file something against microsoft/BashOnWindows?

@djensen47 - not sure about that. I get same errors using powershell or cmd or git's mingw bash. But I also don't know if WSL would affect normal powershell either, so ¯_(ツ)_/¯

djensen47 · 2017-06-08T07:47:42Z

@kumarharsh Did you try clearing your glide cache? I was having issues with Git (mingw) Bash at first but clearing the cache fixed it.

kumarharsh · 2017-06-08T10:34:44Z

@djensen47 I tried clearing cache and running glide update again. Still the same error.

[ERROR] Unable to export dependencies to vendor directory: remove C:\Workspace...\vendor\github.com\chzyer\readline\example: The directory is not empty.

mattfarina · 2017-06-24T13:36:41Z

I've been able to reproduce on projects with a large number of dependencies. Projects I've tried with a smaller set of dependencies have not exhibited the behavior. Is this how it's worked for others?

mattfarina · 2017-06-24T14:31:37Z

The issue appears to be that os.RemoveAll is unable to remove some files. This had previously been an issue in Go and some other projects detect Windows to handle this case separately. The issue is closed.

Unless someone beats me to it, I'll be:

Filing an issue against Go about this
Implementing a work around

kumarharsh · 2017-06-24T16:48:23Z

@mattfarina Yes, it happens consistently with large number of dependencies.

thomastaylor312 · 2017-06-24T20:12:08Z

@mattfarina That seems to be the case for me. Thanks for looking into it!

mattfarina · 2017-06-27T12:28:42Z

Quick update... I think I have a fix but testing it has been a little hard. On Windows (cmd.exe, etc) I'm having trouble reproducing the original issue to prove it's fixed with the change. It's intermittent for me. Under Bash... the linux subsystem setup to work with the filesystem is really slow which has made testing on projects with lots of files (where this shows up) and proving the fix worked to be slow.

It's just taking a little longer than I expected.

kumarharsh · 2017-06-27T12:31:08Z

@mattfarina - I have a system where I always get this error. Maybe if you can share the code/glide binary, I can try it out and report if all goes well

mattfarina · 2017-06-27T12:40:17Z

When I return home, in a little bit, I'll post the code for review. Thanks.

thomastaylor312 · 2017-06-27T20:04:02Z

I can test as well

djensen47 · 2017-06-27T22:21:11Z

@mattfarina I can also test. If you push your updates to a branch, I'll compile and test. Thanks!

mattfarina · 2017-06-29T01:51:54Z

Sorry for the delay. Took a little longer to get to the bottom of this than I expected.

First, the Go issue is at golang/go#20841 along with details to reproduce it outside of Glide.

Second, there is a pull request that needs testing at #857.

Feedback is requested. Is there a better way to handle this?

marct83 · 2017-07-01T05:36:48Z

I can test this I'm just not sure how install your fix on my system. I'm running windows linux subsystem.

djensen47 · 2017-07-01T23:10:52Z

$ git clone git@github.com:Masterminds/glide.git $GOPATH/src/github.com/Masterminds/glide
$ cd $GOPATH/src/github.com/Masterminds/glide
$ git checkout -b windows-many-files origin/fix/windows-many-files
$ make build
$ mv glide $GOPATH/bin

Go to your project and use the new glide but be sure that you're using the new one...

$ which glide

Next, I cleared everything away.

$ rm -rf vendor
$ glide cc

@mattfarina My first attempt at glide cc failed with this error:

[ERROR] Unable to clear the cache: remove .glide/cache/src/https-github.com-fatih-color/vendor/golang.org/x/sys/unix: directory not empty

I tried again and it worked the second time.

The moment of truth ...

$ glide update

🥁 (to be continued, that's supposed to be a drumroll while I'm waiting for the command to finish)

djensen47 · 2017-07-01T23:17:44Z

SUCCESS! It worked! 🕺 🎉 🎆

@mattfarina Thank you! I feel like I should buy you a beer, or coffee, or artisan water, or whatever is your beverage of choice.

marct83 · 2017-07-02T05:49:23Z

@djensen47

I'm running into this issue when I try the clone:

Cloning into '/root/Code/go/github.com/Masterminds/glide'...
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

djensen47 · 2017-07-02T06:55:14Z

Oh, clone using the https address instead. `git clone https://github.com/Masterminds/glide.git`

…

On Sat, Jul 1, 2017, 10:49 PM marct83 ***@***.***> wrote: @djensen47 <https://github.com/djensen47> I'm running into this issue when I try the clone: Cloning into '/root/Code/go/github.com/Masterminds/glide'... Permission denied (publickey). fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#778 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAgN-f2ehDfk6Zc9MWv0j2meCUip06EIks5sJy9ngaJpZM4MXDzl> .

marct83 · 2017-07-02T07:14:44Z

Closer...

make build
go build -o glide -ldflags "-X main.version=v0.12.1-81-ga7ffb17" glide.go
glide.go:24:2: cannot find package "github.com/Masterminds/glide/action" in any of:
/usr/lib/go-1.6/src/github.com/Masterminds/glide/action (from $GOROOT)
/root/marcb/Code/go/src/github.com/Masterminds/glide/action (from $GOPATH)
glide.go:25:2: cannot find package "github.com/Masterminds/glide/cache" in any of:
/usr/lib/go-1.6/src/github.com/Masterminds/glide/cache (from $GOROOT)
/root/marcb/Code/go/src/github.com/Masterminds/glide/cache (from $GOPATH)
glide.go:26:2: cannot find package "github.com/Masterminds/glide/msg" in any of:
/usr/lib/go-1.6/src/github.com/Masterminds/glide/msg (from $GOROOT)
/root/marcb/Code/go/src/github.com/Masterminds/glide/msg (from $GOPATH)
glide.go:27:2: cannot find package "github.com/Masterminds/glide/path" in any of:
/usr/lib/go-1.6/src/github.com/Masterminds/glide/path (from $GOROOT)
/root/marcb/Code/go/src/github.com/Masterminds/glide/path (from $GOPATH)
glide.go:28:2: cannot find package "github.com/Masterminds/glide/repo" in any of:
/usr/lib/go-1.6/src/github.com/Masterminds/glide/repo (from $GOROOT)
/root/marcb/Code/go/src/github.com/Masterminds/glide/repo (from $GOPATH)
glide.go:29:2: cannot find package "github.com/Masterminds/glide/util" in any of:
/usr/lib/go-1.6/src/github.com/Masterminds/glide/util (from $GOROOT)
/root/marcb/Code/go/src/github.com/Masterminds/glide/util (from $GOPATH)
glide.go:31:2: cannot find package "github.com/codegangsta/cli" in any of:
/usr/lib/go-1.6/src/github.com/codegangsta/cli (from $GOROOT)
/root/marcb/Code/go/src/github.com/codegangsta/cli (from $GOPATH)
Makefile:8: recipe for target 'build' failed
make: *** [build] Error 1

djensen47 · 2017-07-02T07:25:38Z

My instructions had a typo. I was missing /src after $GOPATH. I updated the instructions and they should work now. You'll have to start from the top though.

marct83 · 2017-07-02T16:32:51Z

Ok so I was able to get it built. When I do a which glide I'm getting /usr/bin/glide. How do I change it to use the new glide? Thanks for the help.

marct83 · 2017-07-04T04:25:44Z

FYI I got this to work. Tested working.

kumarharsh · 2017-07-05T12:23:56Z

It works for me too on Windows proper (on powershell)!

Fixed #778: Removing and moving large sets of files fails on Windows

thomastaylor312 · 2017-07-07T05:17:54Z

Thanks for patching this @mattfarina!

kumarharsh mentioned this issue Apr 27, 2017

VCS-76: Fix submodule handling for Windows Masterminds/vcs#77

Merged

mattfarina self-assigned this Jun 24, 2017

mattfarina added the bug label Jun 24, 2017

mattfarina closed this as completed in a7ffb17 Jul 7, 2017

mattfarina added a commit that referenced this issue Jul 7, 2017

Merge pull request #857 from Masterminds/fix/windows-many-files

245cace

Fixed #778: Removing and moving large sets of files fails on Windows

bash on windows has error #778

bash on windows has error #778

Comments

freshcn commented Mar 8, 2017 • edited Loading

bacongobbler commented Mar 10, 2017

freshcn commented Mar 11, 2017

mattfarina commented Mar 21, 2017

joesteele commented Apr 5, 2017

eliothedeman commented Apr 16, 2017

djensen47 commented Apr 26, 2017

kumarharsh commented Apr 27, 2017 • edited Loading

thomastaylor312 commented May 21, 2017

vidaoaaron commented Jun 2, 2017 • edited Loading

kumarharsh commented Jun 5, 2017 • edited Loading

djensen47 commented Jun 7, 2017

BhallalDev commented Jun 7, 2017

djensen47 commented Jun 7, 2017

djensen47 commented Jun 7, 2017

kumarharsh commented Jun 8, 2017

djensen47 commented Jun 8, 2017

kumarharsh commented Jun 8, 2017

mattfarina commented Jun 24, 2017

mattfarina commented Jun 24, 2017

kumarharsh commented Jun 24, 2017

thomastaylor312 commented Jun 24, 2017

mattfarina commented Jun 27, 2017

kumarharsh commented Jun 27, 2017

mattfarina commented Jun 27, 2017

thomastaylor312 commented Jun 27, 2017

djensen47 commented Jun 27, 2017

mattfarina commented Jun 29, 2017

marct83 commented Jul 1, 2017

djensen47 commented Jul 1, 2017 • edited Loading

djensen47 commented Jul 1, 2017 • edited Loading

marct83 commented Jul 2, 2017

djensen47 commented Jul 2, 2017 via email

marct83 commented Jul 2, 2017

djensen47 commented Jul 2, 2017 • edited Loading

marct83 commented Jul 2, 2017 • edited Loading

marct83 commented Jul 4, 2017

kumarharsh commented Jul 5, 2017

thomastaylor312 commented Jul 7, 2017

freshcn commented Mar 8, 2017 •

edited

Loading

kumarharsh commented Apr 27, 2017 •

edited

Loading

vidaoaaron commented Jun 2, 2017 •

edited

Loading

kumarharsh commented Jun 5, 2017 •

edited

Loading

djensen47 commented Jul 1, 2017 •

edited

Loading

djensen47 commented Jul 1, 2017 •

edited

Loading

djensen47 commented Jul 2, 2017 •

edited

Loading

marct83 commented Jul 2, 2017 •

edited

Loading