cmd/go: support multiple main packages with -pgo=auto #58099
Comments
prattmic
added
the
NeedsFix
bcmills
added
GoCommand
|
I hope everone here understands that this N-fold complete rebuild doesn't scale. Having 100s of services and 1000s of packages you end up in extremely large caches (100x than normal) or basically dong every build without cache, or mix of both. |
|
Change https://go.dev/cl/472357 mentions this issue: |
|
Change https://go.dev/cl/472358 mentions this issue: |
|
Change https://go.dev/cl/473275 mentions this issue: |
Currently, the PGO profile path is global for a single go command invocation, as it applies to all packages being built (or none). With -pgo=auto mode with multiple main packages, packages from a single go command invocation could have different profiles. So it is necessary that the PGO profile path is per package, which is this CL does. For #58099. Change-Id: I148a15970ec907272db85b4b27ad6b08c41d6c0c Reviewed-on: https://go-review.googlesource.com/c/go/+/472357 Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Bryan Mills <bcmills@google.com>
…nting In -pgo=auto mode, a package may be built multiple times. E.g. for go build -pgo=auto cmd/a cmd/b and both cmd/a and cmd/b imports package p, p may be built twice, one using a's profile, one using b's. If we need to print p, e.g. in "go list -deps" or when there is a build failure, p will be printed twice, and currently we don't distinguish them. We have a precedence for a similar case: for testing, there is the original package, and the (internal) test version of the package (which includes _test.go files). Packages that import the package under testing may also have two versions (one imports the original, one imports the testing version). In printing, the go command distinguishes them by adding a "[p.test]" suffix for the latter, as they are specifically built for the p.test binary. We do the similar. When a package needs to be compiled multiple times for different main packages, we attach the main package's import path, like "p [cmd/a]" for package p built specifically for cmd/a. For #58099. Change-Id: I4a040cf17e1dceb5ca1810c217f16e734c858ab6 Reviewed-on: https://go-review.googlesource.com/c/go/+/473275 Reviewed-by: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Cherry Mui <cherryyz@google.com> Reviewed-by: Bryan Mills <bcmills@google.com>
|
Just tracking: this doesn't actually make |
|
Change https://go.dev/cl/474236 mentions this issue: |
Updates #58099. Updates #55022. Change-Id: I32eacdf9f008d16566e0b30230ecc25d110a9811 Reviewed-on: https://go-review.googlesource.com/c/go/+/474236 Run-TryBot: Cherry Mui <cherryyz@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
|
Change https://go.dev/cl/495477 mentions this issue: |
Updates #58099. Change-Id: I95c0397add696f677c86ab7618482e07eb4e9fda Reviewed-on: https://go-review.googlesource.com/c/go/+/495477 Run-TryBot: Cherry Mui <cherryyz@google.com> Reviewed-by: Austin Clements <austin@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
In Go 1.20,
go build -pgo=autocan only be used with a single main package. Specifying multiple main packages (e.g.,go build -pgo=auto cmd/compile cmd/link) is an error.To support this, cmd/go needs to learn to build the transitive dependencies N times, once for each different profile used by each main package.
Follow up to #55022.
cc @cherrymui @aclements
The text was updated successfully, but these errors were encountered: