Go codegen doesn't work

[rdeknijf]

Go codegen doesn't work

$ pants export-codegen ::

11:11:48.11 [ERROR] 1 Exception encountered:

Engine traceback:
in `export-codegen` goal

ProcessExecutionFailure: Process 'Build Go gRPC protobuf plugin for `protoc`.' failed with exit code 1.
stdout:

stderr:
go: google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.2.0: loading deprecation for google.golang.org/grpc/cmd/protoc-gen-go-grpc: module lookup disabled by GOPROXY=off

So, export-codegen doesn't work. Also, the docs describe that Pants auto-regenerates any protobuf output. But it doesn't.
If you remove the pkg/person.pb.go file, and run

pants test :: tests fail. If you run protoc --go_out=. --go_opt=paths=source_relative person.proto` 

(in pkg), tests are good again.

So, about GOPROXY: I've tried setting it multiple ways, but Pants seems to ignore it all. I think there may be a relation with: #21527

Pants version
Tested on 2.22 and 2.23

OS
Both Linux and Mac.

Additional info
Here's a minimal reproduction repo: https://github.com/rdeknijf/pants-repro-protoc

[tdyas]

Based on my initial investigation, this seems to be caused by go wanting to check the latest version of the protoc-gen-go-grpc module's go.mod to see if it has any deprecation comments. Given the Go backend ordinarily sets GOPROXY=off to prevent networks calls to the Go module proxy, that setting will prevent go from being able to contact the Go module proxy to get deprecation information for the google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.2.0 module.

The GOPROXY=off setting only makes sense though when the Go backend is building and linking Go code which has already been resolved to sources. The Go protobuf backend just invokes go install which does all the building and linking of a binary. In that case, it doesn't make sense to set GOPROXY=off.

See:

pants/src/python/pants/backend/codegen/protobuf/go/rules.py

Lines 604 to 615 in 020b2fb

	Get(
	ProcessResult,
	GoSdkProcess(
	[
	"install",
	"google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.2.0",
	],
	input_digest=download_sources_result.output_digest,
	output_files=["gopath/bin/protoc-gen-go-grpc"],
	description="Build Go gRPC protobuf plugin for `protoc`.",
	),
	),

The rules for GoSdkProcess are what set GOPROXY=off. See

pants/src/python/pants/backend/go/util_rules/sdk.py

Line 56 in 9d1d887

else FrozenDict({**(env or {}), "GOPROXY": "off"})

.

Probably just need to set allow_downloads=True on the use of GoSdkProcess and that will avoid setting GOPROXY=off.

[geoah]

Are there any known workarounds to get export-codegen working with go at the moment?

[tdyas]

Are there any known workarounds to get export-codegen working with go at the moment?

@geoah: Are you able to run Pants from sources and try out #21614 to see if it works for you?

[geoah]

@tdyas I'll try to get this done in the next 24h and get back to you! Thank you for looking into this!

[tdyas]

@geoah: Any word on whether this worked for you?