cmd/go: test coverpkg=all ./... with test only packages will fail to build

[AlexRouSg]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

1.11

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

linux amd64

What did you do?

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

Ran go test -coverpkg=all ./... with the following folder structure:

├── foo.go
├── foo_test.go
└── tests
    └── tests_test.go

Content of files is irrelevant.
Apparently go test -cover ./... works, so it's specific to -coverpkg
Also deleting foo_test.go works

What did you expect to see?

ok      _/<HOME>/go/src/foo   0.001s  coverage: 0.0% of statements [no tests to run]
ok      _/<HOME>/go/src/foo/tests     0.004s  coverage: 0.0% of statements [no tests to run]

What did you see instead?

go build _/<HOME>/go/src/foo/tests: no non-test Go files in go/src/foo/tests
FAIL    _/<HOME>/go/src/foo [build failed]
ok      _/<HOME>/go/src/foo/tests     0.001s  coverage: 0.0% of statements in all [no tests to run]

[FiloSottile]

/cc @bcmills @rsc

[vvakame]

I met this issue in real world.
This issue is very painful...
because, unrelated packages will be build failed without an error message...
I found go build go.mercari.io/datastore/testbed: no non-test Go files in /go/src/go.mercari.io/datastore/testbed message, but I can't found relation between this message to build faled. and I see it occasionally, but I never had a problem until now.

https://github.com/mercari/datastore/tree/v1.4.0/testbed
https://circleci.com/gh/mercari/datastore/219

$ go test go.mercari.io/datastore go.mercari.io/datastore/aedatastore go.mercari.io/datastore/aeprodtest go.mercari.io/datastore/boom go.mercari.io/datastore/clouddatastore go.mercari.io/datastore/dsmiddleware go.mercari.io/datastore/dsmiddleware/aememcache go.mercari.io/datastore/dsmiddleware/chaosrpc go.mercari.io/datastore/dsmiddleware/dslog go.mercari.io/datastore/dsmiddleware/fishbone go.mercari.io/datastore/dsmiddleware/localcache go.mercari.io/datastore/dsmiddleware/noop go.mercari.io/datastore/dsmiddleware/rediscache go.mercari.io/datastore/dsmiddleware/rpcretry go.mercari.io/datastore/dsmiddleware/storagecache go.mercari.io/datastore/internal go.mercari.io/datastore/internal/shared go.mercari.io/datastore/internal/testutils go.mercari.io/datastore/testbed go.mercari.io/datastore/testsuite go.mercari.io/datastore/testsuite/dsmiddleware/dslog go.mercari.io/datastore/testsuite/dsmiddleware/fishbone go.mercari.io/datastore/testsuite/dsmiddleware/localcache go.mercari.io/datastore/testsuite/dsmiddleware/rpcretry go.mercari.io/datastore/testsuite/favcliptools go.mercari.io/datastore/testsuite/realworld/recursivebatch go.mercari.io/datastore/testsuite/realworld/tbf -count 1 -p 1 -coverpkg=go.mercari.io/datastore/... -covermode=atomic -coverprofile=coverage.txt
go build go.mercari.io/datastore/testbed: no non-test Go files in /go/src/go.mercari.io/datastore/testbed
FAIL	go.mercari.io/datastore [build failed]
FAIL	go.mercari.io/datastore/aedatastore [build failed]
?   	go.mercari.io/datastore/aeprodtest	[no test files]
FAIL	go.mercari.io/datastore/boom [build failed]
FAIL	go.mercari.io/datastore/clouddatastore [build failed]
?   	go.mercari.io/datastore/dsmiddleware	[no test files]
FAIL	go.mercari.io/datastore/dsmiddleware/aememcache [build failed]
FAIL	go.mercari.io/datastore/dsmiddleware/chaosrpc [build failed]
FAIL	go.mercari.io/datastore/dsmiddleware/dslog [build failed]
FAIL	go.mercari.io/datastore/dsmiddleware/fishbone [build failed]
FAIL	go.mercari.io/datastore/dsmiddleware/localcache [build failed]
?   	go.mercari.io/datastore/dsmiddleware/noop	[no test files]
FAIL	go.mercari.io/datastore/dsmiddleware/rediscache [build failed]
FAIL	go.mercari.io/datastore/dsmiddleware/rpcretry [build failed]
FAIL	go.mercari.io/datastore/dsmiddleware/storagecache [build failed]
?   	go.mercari.io/datastore/internal	[no test files]
?   	go.mercari.io/datastore/internal/shared	[no test files]
?   	go.mercari.io/datastore/internal/testutils	[no test files]
ok  	go.mercari.io/datastore/testbed	6.887s	coverage: 0.7% of statements in go.mercari.io/datastore/...
?   	go.mercari.io/datastore/testsuite	[no test files]
?   	go.mercari.io/datastore/testsuite/dsmiddleware/dslog	[no test files]
?   	go.mercari.io/datastore/testsuite/dsmiddleware/fishbone	[no test files]
?   	go.mercari.io/datastore/testsuite/dsmiddleware/localcache	[no test files]
?   	go.mercari.io/datastore/testsuite/dsmiddleware/rpcretry	[no test files]
?   	go.mercari.io/datastore/testsuite/favcliptools	[no test files]
?   	go.mercari.io/datastore/testsuite/realworld/recursivebatch	[no test files]
?   	go.mercari.io/datastore/testsuite/realworld/tbf	[no test files]

[gopherbot]

Change https://golang.org/cl/164198 mentions this issue: cmd/go: ignore test only packages with -coverpkg

[BinaryHexer]

This is something that has just bit me as well.

Anyways if anyone is stuck because of this like me, here is tiny workaround:

go test -v -race -coverpkg=$(go list ./... grep -v "/test") -coverprofile=coverage.txt $(go list ./...)

The above workaround only works when your directory structure is as follows:

├── foo.go
└── test
    └── foo_test.go
├── bar.go
└── test
    └── bar_test.go

That is your test only package are always inside a directory named test.
If you have different naming style please modify the grep -v "/test" to suit your use case.

[byxor]

I found a tool called go-acc that solved this problem for me. 🎉

# Download and install go-acc
$ go get -u github.com/ory/go-acc

# Run it against some package(s)
$ go-acc github.com/some/package
$ go-acc .
$ go-acc ./...
$ go-acc $(glide novendor)

Here's the blogpost I found this tool on: https://www.ory.sh/golang-go-code-coverage-accurate

Thanks ory; if you see this, you saved me hours of pain.