GOPACKAGESDRIVER with bzlmod breaks

[alan910127]

What version of rules_go are you using?

0.41.0

What version of gazelle are you using?

0.32.0

What version of Bazel are you using?

Bazelisk version: v1.17.0
Build label: 6.3.2

Does this issue reproduce with the latest releases of all the above?

Yes. All the above are the latest releases.

What operating system and processor architecture are you using?

OS: WSL Ubuntu-22.04 (Microsoft Windows 11 Home 10.0.22621 Build 22621)
Arch: amd64

Any other potentially useful information about your toolchain?

No. I just created a brand new project with local running Bazel.

What did you do?

MODULE.bazel

"Playing with Bazel and Go"

module(name = "bazel-go", version = "0.0.1")

bazel_dep(name = "rules_go", version = "0.41.0")
bazel_dep(name = "gazelle", version = "0.32.0")

go_sdk = use_extension("@rules_go//go:extensions.bzl", "go_sdk")
go_sdk.download(version = "1.21.0", name = "go_sdk")

Editor

Follow the instructions from the Editor Setup section in the Wiki.
With the lines changed:

• tools/gopackagesdriver.sh

  #!/usr/bin/env bash
  exec bazel run -- @rules_go//go/tools/gopackagesdriver "${@}"

• .vscode/settings.json

  {
      "go.goroot": "${workspaceFolder}/bazel-${workspaceFolderBasename}/external/rules_go~0.41.0~go_sdk~go_sdk",
      // ...the rest are omitted
  }

What did you expect to see?

The errors gone in VSCode, and be able to interact with gopls.

What did you see instead?

The error message is copied from the error notification:
(Manual line breaks, some of them might be wrong)

Error loading workspace: 
packages.Load error: 
/home/alan/code/bazel-go/tools/gopackagesdriver.sh: 
exit status 2: 
Loading:  
Loading:  
Loading: 0 packages loaded 
Analyzing: target @rules_go~0.41.0//go/tools/gopackagesdriver:gopackagesdriver (0 packages loaded, 0 targets configured) 
INFO: Analyzed target @rules_go~0.41.0//go/tools/gopackagesdriver:gopackagesdriver (0 packages loaded, 0 targets configured). 
INFO: Found 1 target... [0 / 1] 
[Prepa] BazelWorkspaceStatusAction stable-status.txt 
Target @rules_go~0.41.0//go/tools/gopackagesdriver:gopackagesdriver up-to-date:    bazel-bin/external/rules_go~0.41.0/go/tools/gopackagesdriver/gopackagesdriver_/gopackagesdriver 
INFO: Elapsed time: 0.164s, Critical Path: 0.00s 
INFO: 1 process: 1 internal. 
INFO: Build completed successfully, 1 total action 
INFO: Running command line: bazel-bin/external/rules_go~0.41.0/go/tools/gopackagesdriver/gopackagesdriver_/gopackagesdriver ./... builtin 
Running: [bazel info --tool_tag=gopackagesdriver --ui_actions_shown=0] 
Running: [bazel query --tool_tag=gopackagesdriver --ui_actions_shown=0 --ui_event_filters=-info,-stderr --noshow_progress --order_output=no --output=label --nodep_deps --noimplicit_deps --notool_deps kind("go_library|go_test", ...) union @@rules_go~0.41.0//:stdlib] 
Running: [bazel build --tool_tag=gopackagesdriver --ui_actions_shown=0 --show_result=0 --build_event_json_file=/tmp/gopackagesdriver_bep_3802961558 --build_event_json_file_path_conversion=no --experimental_convenience_symlinks=ignore --ui_event_filters=-info,-stderr --noshow_progress --aspects=@@rules_go~0.41.0//go/tools/gopackagesdriver:aspect.bzl%go_pkg_info_aspect --output_groups=go_pkg_driver_json_file,go_pkg_driver_stdlib_json_file,go_pkg_driver_srcs --keep_going //packages/greet:greet_lib //packages/greet:greet_test @rules_go//:stdlib] 
panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x549bed]  

goroutine 1 [running]: 
main.(*PackageRegistry).walk(0xc0001a4470, 0xc000289a90?, {0xc00001b140?, 0x12?}) 	
external/rules_go~0.41.0/go/tools/gopackagesdriver/packageregistry.go:79 +0x4d main.(*PackageRegistry).Match(0xc0001a4470, {0xc00019c300, 0x3, 0x11?}) 	
external/rules_go~0.41.0/go/tools/gopackagesdriver/packageregistry.go:112 +0x306 main.(*JSONPackagesDriver).GetResponse(0xc0001ca200?, {0xc00019c300?, 0xc00013d400?, 0xc00019c300?}) 	
external/rules_go~0.41.0/go/tools/gopackagesdriver/json_packages_driver.go:51 +0x1f main.run() 	
external/rules_go~0.41.0/go/tools/gopackagesdriver/main.go:111 +0x558 main.main() 	
external/rules_go~0.41.0/go/tools/gopackagesdriver/main.go:115 +0x1f 

[alan910127]

After I downgrade the rules_go to 0.39.1, the problems disappears. So this issue might be duplicated with #3604 .

[alan910127]

UPDATE

After carefully investigating the issue, I found that the problem appears to be related to the bzlmod system and the new "canonical name" approach, which uses double @ symbols at the beginning. This change seems to have introduced the following two issues:

1. The stdlib label does not match the expected RulesGoStdlibLabel.
2. The local package cannot be found since only one @ sign is added to the label, but the IDs are in the form of @@//....

The problems stem from the fact that VSCode only passes the "apparent name" to the driver.

Update Details

To address these issues, I applied a patch to the packageregistry.go file within the gopackagesdriver tool. This patch adjusts the labels to work coherently with the new bzlmod system.

Patch:

diff --color -Naur old/packageregistry.go new/packageregistry.go
--- old/packageregistry.go	2023-08-16 11:17:09.488012010 +0800
+++ new/packageregistry.go	2023-08-16 11:16:45.308021486 +0800
@@ -89,10 +89,10 @@
 
 	for _, label := range labels {
 		if !strings.HasPrefix(label, "@") {
-			label = fmt.Sprintf("@%s", label)
+			label = fmt.Sprintf("@@%s", label)
 		}
 
-		if label == RulesGoStdlibLabel {
+		if strings.HasSuffix(label, "//:stdlib") {
 			// For stdlib, we need to append all the subpackages as roots
 			// since RulesGoStdLibLabel doesn't actually show up in the stdlib pkg.json
 			for _, pkg := range pr.packagesByID {

However, it's important to note that this patch is currently hard-coded and doesn't account for the old WORKSPACE system.

[andrewmbenton]

Assuming you're correct that this is specific to the bzlmod context, I think this is a more accurate description of the problem than what's written in #3604 comments.

I am using bzlmod and also experiencing this issue, as described in my comment on #3604.

[fmeum]

Interesting, we do have logic that is supposed to handle this: https://github.com/bazelbuild/rules_go/blob/f64211a8687352323b973f9a1e6457a60478b061/go/tools/gopackagesdriver/BUILD.bazel#L36

Could you post the exact labels that would be correct and the ones that are emitted?

Edit: Looks like ec5a27c could actually have made this worse?

[alan910127]

This is what I got:

• @rules_go//:stdlib in the labels parameter of the method PackageRegistry.Match
• @@rules_go~0.41.0//:stdlib in RulesGoStdlibLabel

[fmeum]

This needs to be solved in Bazel. I will submit a PR and try to get it into 6.4.0.

[alan910127]

Thanks a lot!