Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/go/packages: Package.Module is unset for std and cmd modules #65816

Open
adonovan opened this issue Feb 20, 2024 · 5 comments
Open

x/tools/go/packages: Package.Module is unset for std and cmd modules #65816

adonovan opened this issue Feb 20, 2024 · 5 comments
Labels
NeedsFix The path to resolution is known, but the work has not been done. Tools This label describes issues relating to any tools in the x/tools repository.
Milestone

Comments

@adonovan
Copy link
Member

The gopackages command (patched as shown at the end) displays the module of each package:

xtools$ gopackages  golang.org/x/telemetry
Go package "golang.org/x/telemetry":
	module golang.org/x/telemetry@v0.0.0-20240209200032-7b892fcb8a78
	package telemetry
	has no exported type info
	file /Users/adonovan/go/pkg/mod/golang.org/x/telemetry@v0.0.0-20240209200032-7b892fcb8a78/doc.go
	file /Users/adonovan/go/pkg/mod/golang.org/x/telemetry@v0.0.0-20240209200032-7b892fcb8a78/mode.go
	file /Users/adonovan/go/pkg/mod/golang.org/x/telemetry@v0.0.0-20240209200032-7b892fcb8a78/types.go
	import "golang.org/x/telemetry/internal/telemetry"

xtools$ gopackages ./gopls
Go command "golang.org/x/tools/gopls":
	module golang.org/x/tools/gopls@
	package main
	has no exported type info
	file /Users/adonovan/w/xtools/gopls/main.go
	import "context"
	import "golang.org/x/tools/gopls/internal/cmd"
	import "golang.org/x/tools/gopls/internal/hooks"
	import "golang.org/x/tools/gopls/internal/telemetry"
	import "golang.org/x/tools/gopls/internal/version"
	import "golang.org/x/tools/internal/tool"
	import "os"

But for standard packages and tools, it shows nothing:

xtools$ gopackages errors
Go package "errors":
	package errors
	has no exported type info
	file /Users/adonovan/w/goroot/src/errors/errors.go
	file /Users/adonovan/w/goroot/src/errors/join.go
	file /Users/adonovan/w/goroot/src/errors/wrap.go
	import "internal/reflectlite"
	import "unsafe"

xtools$ gopackages cmd/link
Go command "cmd/link":
	package main
	has no exported type info
	file /Users/adonovan/w/goroot/src/cmd/link/doc.go
	file /Users/adonovan/w/goroot/src/cmd/link/main.go
	import "cmd/internal/sys"
	import "cmd/link/internal/amd64"
	import "cmd/link/internal/arm"
	import "cmd/link/internal/arm64"
	import "cmd/link/internal/ld"
	import "cmd/link/internal/loong64"
	import "cmd/link/internal/mips"
	import "cmd/link/internal/mips64"
	import "cmd/link/internal/ppc64"
	import "cmd/link/internal/riscv64"
	import "cmd/link/internal/s390x"
	import "cmd/link/internal/wasm"
	import "cmd/link/internal/x86"
	import "fmt"
	import "internal/buildcfg"
	import "os"

This seems like a bug. Even if "go list" doesn't report Module information for standard packages, go/packages should probably synthesize something. Either way, it should document what it actually does.

diff --git a/go/packages/gopackages/main.go b/go/packages/gopackages/main.go
index bf0b5043fc..706f13a99a 100644
--- a/go/packages/gopackages/main.go
+++ b/go/packages/gopackages/main.go
@@ -104,6 +104,7 @@ func (app *application) Run(ctx context.Context, args ...string) error {
        default:
                return tool.CommandLineErrorf("invalid mode: %s", app.Mode)
        }
+       cfg.Mode |= packages.NeedModule
 
        lpkgs, err := packages.Load(cfg, args...)
        if err != nil {
@@ -162,6 +163,9 @@ func (app *application) print(lpkg *packages.Package) {
                kind += "package"
        }
        fmt.Printf("Go %s %q:\n", kind, lpkg.ID) // unique ID
+       if mod := lpkg.Module; mod != nil {
+               fmt.Printf("\tmodule %s@%s\n", mod.Path, mod.Version)
+       }
        fmt.Printf("\tpackage %s\n", lpkg.Name)
 
        // characterize type info
@gopherbot gopherbot added the Tools This label describes issues relating to any tools in the x/tools repository. label Feb 20, 2024
@gopherbot gopherbot added this to the Unreleased milestone Feb 20, 2024
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/565477 mentions this issue: go/packages/gopackages: display module

@prattmic prattmic added the NeedsFix The path to resolution is known, but the work has not been done. label Feb 20, 2024
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/565475 mentions this issue: gopls/internal/cache: fix two bugs related to workspace packages

gopherbot pushed a commit to golang/tools that referenced this issue Feb 20, 2024
Fix two bugs related to workspace packages that contributed to
golang/go#65801.

The first is that we didn't consider packages with open files to be
workspace packages. This was pre-existing behavior, but in the past we
simply relied on diagnoseChangedFiles to provide diagnostics for open
packages, even though we didn't consider them to be part of the
workspace. That led to races and inconsistent behavior: a change in one
file would wipe out diagnostics in another, and so on. It's much simpler
to say that if the package is open, it is part of the workspace. This
leads to consistent behavior, no matter where diagnostics originate.

The second bug is related to loading std and cmd. The new workspace
package heuristics relied on go/packages.Package.Module to determine if
a package is included in the workspace. For std and cmd, this field is
nil (golang/go#65816), apparently due to limitations of `go list`. As a
result, we were finding no workspace packages for std or cmd. Fix this
by falling back to searching for the relevant go.mod file in the
filesystem. Unfortunately this required reinstating the lockedSnapshot
file source.

These bugs revealed a rather large gap in our test coverage for std. Add
a test that verifies that we compute std workspace packages.

Fixes golang/go#65801

Change-Id: Ic454d4a43e34af10e1224755a09d6c94c728c97d
Reviewed-on: https://go-review.googlesource.com/c/tools/+/565475
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/565457 mentions this issue: [gopls-release-branch.0.15] gopls/internal/cache: fix two bugs related to workspace packages

gopherbot pushed a commit to golang/tools that referenced this issue Feb 20, 2024
Updates golang/go#65816

Change-Id: I76db031fc8c1b23caeabe0360db34e1126184e2f
Reviewed-on: https://go-review.googlesource.com/c/tools/+/565477
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
gopherbot pushed a commit to golang/tools that referenced this issue Feb 20, 2024
…d to workspace packages

Fix two bugs related to workspace packages that contributed to
golang/go#65801.

The first is that we didn't consider packages with open files to be
workspace packages. This was pre-existing behavior, but in the past we
simply relied on diagnoseChangedFiles to provide diagnostics for open
packages, even though we didn't consider them to be part of the
workspace. That led to races and inconsistent behavior: a change in one
file would wipe out diagnostics in another, and so on. It's much simpler
to say that if the package is open, it is part of the workspace. This
leads to consistent behavior, no matter where diagnostics originate.

The second bug is related to loading std and cmd. The new workspace
package heuristics relied on go/packages.Package.Module to determine if
a package is included in the workspace. For std and cmd, this field is
nil (golang/go#65816), apparently due to limitations of `go list`. As a
result, we were finding no workspace packages for std or cmd. Fix this
by falling back to searching for the relevant go.mod file in the
filesystem. Unfortunately this required reinstating the lockedSnapshot
file source.

These bugs revealed a rather large gap in our test coverage for std. Add
a test that verifies that we compute std workspace packages.

Fixes golang/go#65801

Change-Id: Ic454d4a43e34af10e1224755a09d6c94c728c97d
Reviewed-on: https://go-review.googlesource.com/c/tools/+/565475
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 607b664)
Reviewed-on: https://go-review.googlesource.com/c/tools/+/565457
@dominikh
Copy link
Member

Related: #61174 (comment)

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/588141 mentions this issue: go/packages: document fields that are part of JSON schema

gopherbot pushed a commit to golang/tools that referenced this issue May 29, 2024
Package is both the "cooked" result data type of a Load
call, and the "raw" JSON schema used by DriverResponse.

This change documents the fields that are part of the
protocol, and ensures that the others are omitted from
the JSON encoding. (They are populated by the post-
processing done by 'refine', if the appropriate Need
bits are set.)

Also
- document that there are a number of open bugs
  in places where it may be likely to help,
  particularly Mode-related issues.
- document that Load returns new Packages,
  using distinct symbol realms (types.Importers).
- document Overlays in slightly more detail.

Fixes golang/go#67614
Fixes golang/go#61418
Fixes golang/go#67601
Fixes golang/go#43850
Updates golang/go#65816
Updates golang/go#58726
Updates golang/go#56677
Updates golang/go#48226
Updates golang/go#63517
Updates golang/go#56633

Change-Id: I2f5f2567baf61512042fc344fca56494f0f5e638
Reviewed-on: https://go-review.googlesource.com/c/tools/+/588141
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
github-merge-queue bot pushed a commit to DataDog/orchestrion that referenced this issue Nov 4, 2024
Fixes #371 

See golang/go#65816

---------

Signed-off-by: Eliott Bouhana <eliott.bouhana@datadoghq.com>
Co-authored-by: Romain Marcadier <romain@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsFix The path to resolution is known, but the work has not been done. Tools This label describes issues relating to any tools in the x/tools repository.
Projects
None yet
Development

No branches or pull requests

4 participants