cmd/go: return best-effort results despite inconsistent vendoring

[stamblerre]

Following up on #39100, is it possible for the go command to return partial results even with an inconsistent vendor folder? This is done in several (arguably smaller) cases, like #35973 and #38846.

If go list could return partial results along with the error, that would make the experience in gopls much better for users. Right now, our best option will be to show the user a pop-up with the error message and a button that, when clicked, runs go mod vendor. This work-around is fine for now, but I wonder if there is a better longterm solution.

/cc @bcmills @jayconrod @heschik

[jayconrod]

cc @matloob

Tentatively milestoning for 1.16. @bcmills could confirm feasibility of this though.

I think this is possible, but it will make looking up which module provides each package unreliable. We'd probably only want to do this with go list -e. That would work for gopls though, right?

[stamblerre]

Thanks!

I think this is possible, but it will make looking up which module provides each package unreliable. We'd probably only want to do this with go list -e. That would work for gopls though, right?

Yep, gopls uses -e.

[bcmills]

Yes, I think this is feasible. The fix will be to move the inconsistent vendoring error from one that occurs at module loading time to one that applies to individual packages and/or patterns.

I think it will suffice to apply the error to every package loaded from the vendor directory, plus any wildcard patterns that could match packages outside of the standard library. All other packages (such as those loaded from std and from the main module) should be unaffected by inconsistent vendoring anyway.

[dlsniper]

I have a similar problem, but I'm not sure if my use-case fits in this issue or needs to be addressed in a separate issue.

When I list the dependencies of a Go modules based project, e.g. Delve, with GOPROXY=off by running this command:

go list -m -json -mod=vendor all

I receive the following error:

go list -m: can't compute 'all' using the vendor directory
        (Use -mod=mod or -mod=readonly to bypass.)

If I use the suggested -mod=mod or -mod=readonly then I'll receive an error similar to this:

go: github.com/cosiner/argv@v0.1.0: module lookup disabled by GOPROXY=off

When compiling the application with a command like the one below, the command finishes successfully:

go build -mod=vendor -o=my-delve github.com/go-delve/delve/cmd/dlv

If I set the GOPROXY=proxy.golang.org,direct, then go list downloads the dependencies and then lists them from the GOMODCACHE. This is not what I want or need.

So, the problem seems to be go lists's inability to list the dependencies when using -mod=vendor and/or work with GOPROXY=off. The -e flag does not seem to do anything in this case.

As such, I propose that go list should continue to work and list the vendored modules/dependencies when GOPROXY=off and -mod=vendor flags are present. If I need to add -e to the go list command, that's OK too.

This problem also hampers users of editors such as GoLand. It is impossible for the IDE to list the vendored dependencies and have access to their version, without parsing the vendor/modules.txt file. To the best of my knowledge, the file format is not stable, or well documented, which means it can change at any time (please correct me if my assumption is incorrect).

[dmitshur]

I understand there's not enough time to include this issue in the Go 1.16 release, so I'll reset the milestone to Backlog. Please update it as needed.

[mtalhasaleem502]

Is this issue resolved in go 1.16.2 ??
I am facing this issue in go 1.15.7 should I change the version or not??

[jayconrod]

@mtalhasaleem502 No, this issue is still open.

You can resolve this problem in your module by running go mod vendor to make the vendor/modules.txt file consistent with go.mod.

[mtalhasaleem502]

I have done this but its still giving the same issue in go.mod file.

[bcmills]

@mtalhasaleem502, please open a separate issue with steps to reproduce the problem.

[mtalhasaleem502]

@bcmills yes I have open the new issue #45103

[ianlancetaylor]

Is this issue likely to be done for 1.17? Thanks.

[bcmills]

I don't think this will be done for 1.17. Moving to Backlog, but I might reassess it alongside the changes for #36876.

[AbbyDeng]

@mtalhasaleem502 No, this issue is still open.

You can resolve this problem in your module by running go mod vendor to make the vendor/modules.txt file consistent with go.mod.

Hey Jay, I read a few threads you guys had and I'm facing a similar issue here: basically running go mod vendor could not fix the inconsistency between vendor and go.mod.

I'm happy to elaborate on what I'm trying to do here if this info could be helpful for you guys and get some insights from you:
I'm playing around with go-gorm/playground: https://github.com/go-gorm/playground
as the step 4, I should be able to run ./test.sh, however, it always shows:

no required module provides package

I read one thread where you suggested running go mod tidy then go mod vendor, this is what I got:

Don't know if this is related?
If not, I can delete this post, if so, do you have any idea how to fix this? or we need to wait until Go 1.17?

Thank you so much! :)

[seankhliao]

@AbbyDeng no, that test script edits go.mod and should rerun go mod tidy and go mod vendor after it does so

[pigletfly]

any updates ? I'm using go1.17.5,re-run go mod tidy go mod vendor doesn't work.Is there a workaround?

[padraicbc]

any updates ? I'm using go1.17.5,re-run go mod tidy go mod vendor doesn't work.Is there a workaround?

golang/vscode-go#1595 (comment)

[pjweinb]

This seems to be still happening.

[petkostas]

I am getting the same issue here (and I am pretty new with GO)

[Jarvan-via]

Removing gopls from setting.json did the trick for me.