x/tools/cmd/stringer: does no longer work in package with uninstalled dependencies

[eliaperantoni]

What version of Go are you using (go version)?

$ go version
go version go1.11.4 windows/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Elia\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\Users\Elia\go
set GOPROXY=
set GORACE=
set GOROOT=C:\Go
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\Elia\AppData\Local\Temp\go-build407096830=/tmp/go-build -gno-record-gcc-switches

What did you do?

Ran stringer -type MyType in the package where struct MyType is declared.

What did you expect to see?

stringer running correctly and producing a generated code file like it always did so far

What did you see instead?

This error

PS C:\Users\Elia\go\src\gitlab.com\eliaperantoni\helloworld> stringer -type MyType
stringer: go [list -e -json -compiled=true -test=false -export=false -deps=true -find=false -tags= -- .]: exit status 1: go build github.com/sirupsen/logrus: no Go files in
go build github.com/goburrow/modbus: no Go files in

It seems like stringer is no longer able to operate when searching for a type declaration in a package that has uninstalled dependencies like it did before.

Installing dependencies before running stringer makes it work correctly.

Why would anyone need this?

I'm building docker images for my go application that doesn't use any dependency managment tool yet. The build process:

1. Installs some essential dependencies like stringer itself
2. Runs go generate
3. Then installs all dependencies using go get -d ./.... Because of this there's no need to update the Dockerfile after adding or removing dependencies.

Running go generate before go get -d ./...is necessary because otherwise some directories would end up having no go files, causing go get to fail.

When was it working?

I've manually installed a version of golang.org/x/tools that goes back to the first week of march 2019 and that one works without any problems. It seems like golang.org/x/tools/go/packages has been updated ever since (while golang.org/x/tools/cmd/stringer hasn't) but I can't pinpoint the line or commit that is causing the issue.

Maybe this is even some intentional behaviour?

[andybons]

@matloob @ianthehat

[matloob]

I'm sorry, I don't completely understand: what do you mean by uninstalled dependencies? Do you mean that the dependencies of the package itself are missing?

[matloob]

@jba since he helped merge in @suzmue's work on stringer.

[eliaperantoni]

@matloob Yes, by that I mean packages that I import in my project but are actually not installed on the system.

This happens i.e. when building a Docker image for a Go project that uses stringer.

...
COPY . .
RUN go generate
RUN go get -d ./...
...

Using RUN go get -d ./... is easier to use than managing dependencies inside Dockerfile but it must be run after go generate (which includes some stringer steps) otherwise some packages will have no Go file and the build process will exit

This might be a edge case but I thought that pointing out that it used to work and now it doesn't was the right thing to do.

[eliaperantoni]

To reproduce:

git clone https://github.com/eliaperantoni/stringerIssue31216.git
cd stringerIssue31216
go mod download
go run golang.org/x/tools/cmd/stringer -type MyType

It seems like running stringer the first time makes it crash but running it again works correctly.
To make it crash again you have to edit root.go and change the imported package name to any other string.
I have no clue why it only works the second time but I think it always crashed in my real world scenario, no matter how many times I ran it.
I hope this is still useful.

[eliaperantoni]

It looks like the bug first appears in golang/tools@dbeab5a

@matloob

[matloob]

Hm, this is interesting:

git clone https://github.com/eliaperantoni/stringerIssue31216.git
cd stringerIssue31216
go list -e -compiled .; echo $? #  exit status 1
go list -e -compiled .; echo $? # exit status 0

I don't think go list should have that behavior. Though we'll need to put in a temporary fix for the mean time

edit: added "-e", though the behavior is present both with and without -e

[matloob]

I want to make sure I understand the problem: You're trying to run stringer on a package that has a dependency that doesn't exist right? I'm going to do some research to figure out how stringer handles packages it can't type-check.

[matloob]

@bcmills @jayconrod for the go list behavior in #31216 (comment). Is that expected?

[eliaperantoni]

@matloob It's correct. In my example I've used a meaningless string for the package name (you can also use completely random strings that don't make any sense for the import path actually and the issue remains) while in my real world scenario I was dealing with real libraries (such as logrus) that were imported but were not installed on the system yet

[bcmills]

@matloob, go list -e -compiled failing is #26755.

If that snippet is accurate, the 0 exit status from the second call may be another bug; we'll need to investigate one the first one is fixed.

[matloob]

Are you using a meaningless string so you can replicate after the first time you stringer?

That is, were you also seeing the issue when using a package that does exist? I think the go list behavior in the comment above explains the behavior with a package that doesn't exist, but if it's happening with a package that does exist, there's another, bigger problem.

Could you try running stringer with a empty GOCACHE and GOPATH as follows?:

GOCACHE=$(mktemp -d) GOPATH=$(mktemp -d) go run golang.org/x/tools/cmd/stringer -type MyType

I'm interested if you still see the error in that case