cmd/go: duplicate symbols when including two main packages in -coverpkg in module mode

[lshulyay]

Note that I am not sure if this is a bug or expected behaviour, only that this does not happen in the latest stable 1.11 release and does on 1.12rc1.

What version of Go are you using (go version)?

$ go version
go version go1.12rc1 linux/amd64

Does this issue reproduce with the latest release?

Not the latest stable release (1.11.5)

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/lazer/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/lazer/go"
GOPROXY=""
GORACE=""
GOROOT="/home/lazer/sdk/go1.12rc1"
GOTMPDIR=""
GOTOOLDIR="/home/lazer/sdk/go1.12rc1/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/lazer/Dev/gosnaillife/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build721704492=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I ran the following command:

go1.12rc1 test -race -v -coverpkg gitlab.com/drakonka/gosnaillife/cmd/snaillifecli,gitlab.com/drakonka/gosnaillife/cmd/snaillifesrv, -coverprofile allcoverage-rc.out ./...

Both of the above are main packages producing two binaries.

What did you expect to see?

I expected the tests to run and pass, with coverage output, as they do on Go 1.11.5

What did you see instead?

The following error:

2019/02/24 11:39:04 duplicate symbol runtime/debug.modinfo (types 30 and 30) in gitlab.com/drakonka/gosnaillife/cmd/snaillifecli and $WORK/b293/_pkg_.a(_go_.o)

I run Go RC and Go 1.11 side by side as part of my CI, which is how I noticed this issue. I am able to reproduce it locally as well. Full logs for the successful run of Go 1.11 and the failure run of Go 1.12rc1 can be found here:

• https://gitlab.com/drakonka/gosnaillife/-/jobs/166407007
• https://gitlab.com/drakonka/gosnaillife/-/jobs/166407009

[mvdan]

I found the same regression just a few days ago, but hadn't filed an issue yet. This is impeding my daily development, and I'd consider it a regression introduced in 1.12.

The 1.12 release will be out any day now though, so it's probably more realistic to target a backport to 1.12.1. /cc @bcmills @ianlancetaylor @hyangah since this very likely relates to the recent modinfo changes in the cycle.

[bcmills]

I'd consider it a regression introduced in 1.12.

This is almost certainly a bad interaction between #23910, //go:linkname, and the fix for #27584. Note that the former dates to at least Go 1.10, and can be a problem for any program that happens to use //go:linkname in package main (although those are hopefully rare).

Before the fix for #27584, the error (and redefinition) still existed, but because the variable was initialized in an init function, instead of a compile-time error one of the values of debug.modinfo would be overwritten at run-time.

[bcmills]

Given that we do not currently merge coverage profiles from multiple processes (#28235), I'm not sure why you would want to include two different main packages in the same -coverpkg argument (except, perhaps, to simplify the patterns passed to -coverpkg).

The workaround for now is to include at most one main package in the -coverpkg arguments, or to run coverage in GOPATH mode.

[bcmills]

CC @jayconrod

[schoentoon]

Seems very similar to the issues I had when Golang 1.11 came out #23910. I guess I'm just disabling -coverpkg on CI for now.

[lshulyay]

Seems very similar to the issues I had when Golang 1.11 came out #23910. I guess I'm just disabling -coverpkg on CI for now.

Depending on the details of your setup that seems like it might be overkill when you can just remove the conflicting main packages from -coverpkg and still get coverage reporting, right?

[markus-wa]

Same issue on my end https://travis-ci.org/markus-wa/demoinfocs-golang/jobs/499992529

I see the milestone is 1.13, does that mean it won't be fixed in 1.12.x?

[mvdan]

@bcmills I didn't know this was silently broken in 1.11; thanks for pointing this out. I think temporarily working around the issue for now is acceptable, but perhaps we should provide a simple example or one-liner that people could switch to.

[bcmills]

@mvdan , the one-liner is GO111MODULE=off. (There is a reason I wasn't comfortable turning it on-by-default in 1.12, and bugs like this one are that reason: they're unusual but not esoteric.)

[mvdan]

Thanks for the tip! If the bug(s) can be fixed in time for 1.13, then the timing with GO111MODULE=on would be ideal :)

[markus-wa]

the one-liner is GO111MODULE=off

@bcmills but with that set to off, go, understandably, isn't able to find dependencies. - .travis.yml

Am I missing something obvious here?

I'm not sure why you would want to include two different main packages in the same -coverpkg argument

In my case I have a couple of examples for my lib and each of them has it's own main, then I just do go test -coverpkg=./... ./...

[gopherbot]

Change https://golang.org/cl/164877 mentions this issue: cmd/go: avoid link errors when -coverpkg covers main packages

[sjansen]

I was bitten by the same problem: multiple main packages in subdirectories below examples/.

If you're slightly insane, here's a work around:

go test -race \
    -coverpkg .,$(
        ls -1d */ \
        | sed -E \
            -e :a \
            -e '$!N ; s|\n|,| ; ta' \
            -e 's#dist/|docs/|examples/|scripts/##g' \
            -e 's/,,/,/g ; s/^,// ; s/,$//' \
            -e 's|/|/...|g'
    ) \
    -coverprofile=dist/coverage.txt \
    -tags integration ./...

[markus-wa]

@sjansen You can also use go list instead, if that's more sane.

go test -v -coverprofile=coverage.out -coverpkg=$(go list ./... | grep -v 'examples/' | awk -vORS=, '{ print $1 }' | sed 's/,$/\n/')

I'm still a bit bummed to lose coverage on the examples though 😄.

[lshulyay]

I've been doing it like this:

CVPKG=$(go list ./... | grep -v -e mocks -e snaillifecli -e snaillifesrv | tr '\n' ',') 

go test -race -v -coverpkg $CVPKG -coverprofile allcoverage-stable.out ./... 

[bcmills]

@gopherbot, please backport to 1.12. This is a significant regression from GOPATH mode and the fix is straightforward.

[gopherbot]

Backport issue(s) opened: #30684 (for 1.12).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

[jba]

I can still reproduce this in 1.12.1:

> docker run -it golang:1.12.1
# cd ..
# git clone https://github.com/google/go-cloud
# cd go-cloud
# export GO111MODULE=on
# go test -coverpkg=./... ./...
...
2019/03/18 12:52:58 duplicate symbol runtime/debug.modinfo (types 30 and 30) in gocloud.dev/samples/blobdownload and $WORK/b466/_pkg_.a(_go_.o)  

(Note: I had to run the go test line several times because I would get occasional "Could not resolve host: github.com" errors. Not sure whether that's specific to my setup.)

[jayconrod]

Reproduced. Trying to narrow it down.

It's possible this is a new issue. The _go_.o part of the message is suspicious. I don't think that file should be getting linked.

[jba]

It might be useful to know that this repo has nested modules.

[gopherbot]

Change https://golang.org/cl/168200 mentions this issue: cmd/go: avoid link error when -coverpkg covers main packages (more)

[jayconrod]

@gopherbot Please backport to 1.12. This is a more complete fix for an issue that was supposed to be fixed in 1.12.1.

[dmitshur]

For posterity, the second backport issue was created manually: #30937 (for 1.12).

[imsodin]

I am encountering errors on go1.12.5 just like previously reported.

$ go version
go version go1.12.5 linux/amd64

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/simon/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/media/ext4_data/Coding/go"
GOPROXY=""
GORACE=""
GOROOT="/media/ext4_data/Linux/source/go"
GOTMPDIR=""
GOTOOLDIR="/media/ext4_data/Linux/source/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/simon/tmp/syncthing/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build248314634=/tmp/go-build -gno-record-gcc-switches"

Running all tests with -coverpkg on all packages

git clone https://github.com/syncthing/syncthing.git
cd syncthing
go test -short -coverprofile coverage.out -coverpkg ./cmd/...,./lib/... ./cmd/... ./lib/...

results in errors like previously reported:

2019/05/15 16:02:50 duplicate symbol go.constinfo.main (types 45 and 45) in github.com/syncthing/syncthing/cmd/strelaypoolsrv and /home/simon/.cache/go-build/87/87afda35dba0b8de44e2f9d449eda7ce4caef7ed781612883b4e457eab1ee3d5-d(_go_.o)

The same also happens with just ./.... I cleaned out ~/.cache/go-build for good measure, didn't change anything.

To make sure it isn't some weird local problem of mine I also ran it on Syncthing's CI and reproduced the issue there as well: https://build.syncthing.net/viewLog.html?buildId=39242&buildTypeId=Syncthing_Coverage&tab=buildLog

[bcmills]

@imsodin, please file a separate issue — there may be a similar bug involving the -coverprofile flag.