cmd/go: mod tidy reports toolchain not available with 'go 1.21'

[matthewhughes-uw]

Note: I think this is a question of: is "go 1.21" a valid go directive? since there was no 1.21 release, only 1.21.0, perhaps not

What version of Go are you using (go version)?

$ go version
go version go1.21.0 linux/amd64

Does this issue reproduce with the latest release?

Yes, reproduced on Go 1.21.0

What operating system and processor architecture are you using (go env)?

go env Output

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/root/.cache/go-build'
GOENV='/root/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.21.0'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='0'
GOMOD='/proj/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build435086183=/tmp/go-build -gno-record-gcc-switches'

What did you do?

Given a basic go.mod

$ cat go.mod 
module example.com/foo

go 1.21

Then run:

$ GOTOOLCHAIN="go1.20+auto" go mod tidy
go: downloading go1.21 (linux/amd64)
go: download go1.21 for linux/amd64: toolchain not available

What did you expect to see?

go mod tidy runs successfully. I guess I expect it would pick go1.21.0 as the toolchain if no toolchain is available as 1.21

What did you see instead?

The error output above with a non-zero exit code

More details

The issue was seen when running dependabot on a repo with a go 1.21 directive in go.mod, the GOTOOLCHAIN above was taken from their usage https://github.com/dependabot/dependabot-core/blob/08ac25ebd773cede0c00be9a98e5bb03b680870b/go_modules/Dockerfile#L34

Running the above command with GODEBUG=http2debug=1 I see it:

• requests go.dev/dl/mod/golang.org/toolchain/@v/v0.0.1-go1.21.linux-amd64.zip which returns Location: https://go.dev/dl/mod/golang.org/toolchain/@v/v0.0.1-go1.21.linux-amd64.zip
• requesting that URL then returns location: https://dl.google.com/go/v0.0.1-go1.21.linux-amd64.zip
• requesting that URL gives a 404

Updating the directive to: go 1.21.0 will permit things to run fine.

Update: closed per #62278 (comment)

[seankhliao]

how did you end up with go 1.21 as a directive?

[matthewhughes-uw]

how did you end up with go 1.21 as a directive?

I can't remember exactly, but I think it was rather manual like: go1.21.0 mod edit -go=1.21

[laboger]

I am seeing this problem on linux/ppc64le.
I have built a toolchain from the latest go1.21 branch and set up my PATH to use it.
When I cd'ed to cmd/compile/internal/ssa in a toolchain from master that changes the behavior to what is shown above.
I found it when I tried to do 'go generate' but I get the same behavior from just doing 'go version' from within that directory.
If I cd back to another directory then it provides the correct go version.

$ go version
go version go1.21.0 linux/ppc64le
$ cd ~/golang/plain/go/src/cmd/compile/internal/ssa
$ pwd
/home/boger/golang/plain/go/src/cmd/compile/internal/ssa
$ go version
go: downloading go1.22 (linux/ppc64le)
go: download go1.22 for linux/ppc64le: toolchain not available
$ cd ~/gotests
$ go version
go version go1.21.0 linux/ppc64le

[bcmills]

@matthewhughes-uw, this is as expected: go 1.21 was a development version of the language, for which there is no downloadable release (because it is not a specific version). The release version would be go 1.21.0.

In general if you want to specify a development version in the go line, you must also give a concrete toolchain version. So either of these should work:

go 1.21.0

or

go 1.21
toolchain go1.21.0

[bcmills]

@laboger, the problem you are seeing is similar. go1.21.0 cannot upgrade automatically to a toolchain that supports go 1.22, because no such toolchain has been released. In order to compile a package in a go 1.22 module, you need to use a development build of the toolchain that supports that language version.

In particular, if you are working within $GOROOT/src you should be using exactly the toolchain built by make.bash within that GOROOT — you may need to re-run make.bash and/or check your $PATH. 😅

[bcmills]

As far as I can tell this is all working as designed. @matthewhughes-uw, are there specific documentation changes that would have helped you solve or avoid this problem?

[matthewhughes-uw]

As far as I can tell this is all working as designed

👍

@matthewhughes-uw, are there specific documentation changes that would have helped you solve or avoid this problem?

I think the main issue was my assumption that, previously I could just specify a language version (per https://go.dev/doc/toolchain#version) i.e. go 1.X in my go.mod e.g. go 1.20 but with Go 1.21 I can no longer do this. The docs say:

The version must be a valid Go release version, such as 1.9, 1.14, or 1.21rc1

and within the linked page

The syntax ‘1.N’ is called a “language version”. It denotes the overall family of Go releases implementing that version of the Go language and standard library.

so I guess it wasn't clear to me that a "language version" != a "Go release version" (though it was true for every language version before Go 1.21)

In general if you want to specify a development version in the go line, you must also give a concrete toolchain version. So either of these should work:

🤔 Maybe it's worth documenting something like "If you want to specify a 'language version' in go.mod you should (must? But again, only true for Go 1.21 at the moment) also specify a toolchain version"?

[prymitive]

I think the main issue was my assumption that, previously I could just specify a language version (per https://go.dev/doc/toolchain#version) i.e. go 1.X in my go.mod e.g. go 1.20 but with Go 1.21 I can no longer do this.

To add to this: what really happened here that causes a bit of confusion is that the value of go keyword in go.mod changed it's format.

Before 1.21 release X.Y.Z value (1.20.1) would be invalid so you couldn't put it there.

With 1.21 release you most likely want to put X.Y.Z there since most user would want to put there a real working version, not a dev release.
To add to confusion X.Y value technically works, but it may or may not depending on what was released.

So the way I see it we went from never put .0 to always put .0 (0 being point release number).

Having now also toolchain (which from docs sounds like it's optional) just means that people need to wrap their heads around more complicated set of options to set, as per example from #62278 (comment).

[pellared]

So the way I see it we went from never put .0 to always put .0 (0 being point release number).

If this is the case, I think that the Go docs/blogs/reference should recommend it.

Right now, I think that the documentation explains well how the Go modules + Go toolchain are supposed to work, but it misses clear recommendations that the Go developers should follow.

Examples that may even to discourage putting .0:

• https://go.dev/ref/mod#go-mod-file has examples with go 1.12 and go 1.14. There is not a single example with go 1.x.y (where x >= 21)
• In https://go.dev/doc/modules/managing-dependencies we have go.mod with go 1.16.

However, I am not sure if this would be a correct recommendation. What would be wrong if the go.mod have

go 1.22

toolchain 1.23.3

This way:

1. binaries build in other modules may use Go 1.22 and still use packages from "this" module as a dependency
2. binaries (main packages) build in "this" module are going to use Go 1.23.3 or newer (e.g. making sure that a binary with latest security fixes is used)

At the same time

go 1.22.0

toolchain 1.23.3

also does not look wrong.

Having guidelines maybe would also reduce the risk of having issues like this one: grpc/grpc-go#7831

Moreover, I am not sure it this was a good change in golang.org/x/mod: https://go-review.googlesource.com/c/mod/+/605935.