cmd/vet: disable checks based on go.mod go version

[rsc]

New vet checks can make old code's tests stop passing. An example is #57059, where the printf format checker got more precise in Go 1.18. Code that tested fine with Go 1.17 stops passing tests in Go 1.18.

Vet needs to know which Go version each package was written for anyway. Perhaps it should use that information to disable checks that are newer than that version, so that tests don't start failing due to new vet reports until you update the go.mod go line.

(Related to #56986.)

[timothy-king]

For the go command, we can forward the module information from go/packages. I am not really sure where one gets the go version of the package from bazel builds yet. sdk_version is on go_cross_binary.

It would be nice if knew we were going to use this before we commit to supporting it. I am not sure we would use this for #57059 at the moment. A range variable semantic change would require this for loopclosure (related #56010).

A somewhat related problem is rolling out precision improvements to existing checks incrementally. Perhaps using the version number would have been a good way to gate the t.Parallel improvement to loopclosure? It gives a good and consistent story of "as a part of updating to the new version, you must also fix new warning from new checks". A problem with doing this is that this particular check can still flag issues in older code. It seems a pity to not warn and get those unit tests fixed due to the go.mod file not being updated.

[rsc]

I would use it for any new analysis that we add in a particular version of Go. I think we could have used this for the Printf check when we did the expansion in Go 1.18 to look at constant values.

The analysis doesn't run when the go.mod go line says an earlier version than when it was introduced. That way if people update to a new Go toolchain and run 'go test ./...', the test failures don't happen until they update the go.mod to opt in to the new release semantics. I would have done the same for the loopclosure fix. If people want the warnings, they are welcome to update the Go version and get them. If they just want to build some old code with a new toolchain, we should make that the least disruptive we can. We want to prioritize compatibility over new test errors.

Everything is going to have to be updated to understand per-module language versions when it's time for the for loop fix. The go command and the compiler obviously already understand that. We will need to pass it to vet as well. And Bazel will need to understand it too, I assume by gazelle writing something per BUILD file or per Go rule. So the version is going to be there already.

[timothy-king]

I believe all of the work for this has been done. Okay to close?

edit: I was only focused on the part that forwards the package version information to vet. That part is done. Turning off checks in detail based on the version has not yet been done.

[ChrisHines]

I have a question about this change.

Background: I help maintain a large number of (private) Go modules that we generally upgrade the version of Go at the same time for all of them. In the past, prior to actually upgrading them I would run go vet across all the modules with the new tool chain to see how much we needed to update before bumping the Go version. We would usually fix those issues before actually upgrading Go to avoid a scramble.

Do I understand correctly that with this change, running a new version of go vet is no longer sufficient to discover what code needs updating in the above situation? We would first need to run go get go@1.N+1 in the module and then revert any changes that made if we aren't ready to upgrade.

Although that isn't terribly hard and it's something that only happens once every ~6 months, it would be nice to have a way to override the version specified in the go.mod file. Something like staticcheck has: https://staticcheck.io/docs/running-staticcheck/cli/#go.

I would also like to mention that anyone not aware of this change may be surprised by following the same procedure as before, see no warnings from go vet and falsely believe there are no issues to address before upgrading. This change should be prominently mentioned in the release notes and the go vet docs updated appropriately.

[timothy-king]

We would first need to run go get go@1.N+1 in the module and then revert any changes that made if we aren't ready to upgrade.

In some cases, yes. These are related loop variable scope at the moment.

Each file now has an associated GoVersion. There are fine grained controls with //go:build but most are expected to be derived from go.mod versions. To get the new loop variable scope changes the GoVersion >= 1.22 in the file. So some vet checks are pegged to specific GoVersions. An example is loopclosure no longer warning after an upgrade. So that is less warnings on go get go@1.N+1. In the other direction, to resolve #66387 would result in copylock giving more warnings because there is a new implicit assignment in 3-clause for loops with GoVersion >= 1.22. The new stdversions checker #46136 will start warning based on the file's GoVersion. This will issue fewer warnings when upgrading a go.mod file.

So so far most of what is there should be not be too visible to most people when upgrading the go.mod to go1.N+1. The exceptions mean you have a serious problem, and hopefully the bumpiness when updating in those cases is warranted.

I would also like to mention that anyone not aware of this change may be surprised by following the same procedure as before, see no warnings from go vet and falsely believe there are no issues to address before upgrading. This change should be prominently mentioned in the release notes and the go vet docs updated appropriately.

If we did start gating new vet checks (or improved precision) on the GoVersion, upgrading the go.mod to go1.N+1 would become more visible. Maybe that would require more documentation? Blog post?

Thank you for pointing this out.

[ChrisHines]

@timothy-king I've read your post a couple times and I am having trouble interpreting your argument. I think you are saying essentially that go vet checks rarely find new problems in old code so the practice of running go vet from tool chain vN+1 before upgrading isn't usually needed. Apologies if I've misinterpreted your comments.

I don't find that point persuasive, though. On more than one occasion go vet vN+1 has found more problems in our code than go vet vN. From that experience I have learned that it is prudent to look before we leap and work to fix the problematic code before we upgrade the tool chain. Keeping our CI builds rolling along smoothly is valuable to the developers on the project. So regardless of whether go vet vN+1 will actually find anything we don't know until we try. So my feedback here is about the extra steps to perform that check this change implies compared to the status quo.

[timothy-king]

@ChrisHines The point I am making is a bit different. I am claiming that expected impact of changing a go.mod file to go1.N+1 has limited risk of new diagnostics at least given what cmd/vet does as of 1.22 (and into 1.23). As of today, changing the module version is expected to produce fewer diagnostics for a fixed toolchain.

Now I do hope to address a false negative in an existing checker so that there will be a new diagnostic (but hopefully only in rare and serious cases). So this may stop being true. My point was the concern "We would first need to run go get go@1.N+1 in the module and then revert any changes that made if we aren't ready to upgrade" is somewhat limited today.

So my feedback here is about the extra steps to perform that check this change implies compared to the status quo.

I agree. This is an implication of gating new vet checks to the go.mod file (what this issue is proposing to do). This might still be the right thing to do if the toolchain is And if Go does this, it seems like there should be documentation around best practices.

[findleyr]

See also #71487 (a dupe I just filed).

It sounds like there is a real need for this, and as @ChrisHines has pointed out (along with @ianlancetaylor on the dupe), we probably also want to add some way to set the applicable Go version while running vet.

If we do this, here are some action items. Dumping them here as I've been thinking about them:

• Update cmd/vet/README, go tool vet, and the vet package documentation to describe how vet behaves with respect to language versions.
• Add a -go argument to vet to control the 'version' of the checks. (The implementation of such a flag may be tricky). (EDIT: let's treat this as a separate concern -- see below)
• Add some documentation on vet best practices. Not sure of the best location for this, but the package documentation seems like a reasonable home (and is the first Google result for golang vet).

[ChrisHines]

FWIW, I've started experimenting with 1.24rc2 on some of our code at $work today.

I encountered examples of go vet flagging non-constant fmt.Errorf(s) calls and because that check runs as part of go test I had to fix them for tests to pass. My main goal was to test the new go get -tool features, so I had run go get go@1.24rc2 first and that's when I discovered the go vet warnings. But as I mentioned before, the ability to scan all our code for these issues without modifying any files would be ideal.

[findleyr]

@ChrisHines yes, that check in particular is likely to produce findings in existing code, and was in fact the reason we restarted this conversation (#71485). In go1.24rc3, it will only apply to language versions >= 1.24 (as set by go.mod, go.work, or //go:build directives).

Discussing with @adonovan, we think this policy change seems relatively uncontroversial (and reversible), but the -go flag is much less clear (Should it be -lang? But it's not a language version. Do we really need this, or can users just run with a go 1.24 go.workgo.mod file?). Therefore, let's separate the discussion of gating checks, from changes to the vet command line.

@aclements if you are OK with this policy change, we can update the docs to reflect it. Or if you feel it needs more discussion, we can bump it to the proposal committee.

[aclements]

Just to reiterate, the policy would be: any newly added or strengthened vet checks would need to be gated on the Go version they are released in. Correct?

Is this going to make maintenance of vet more difficult? It suggests really any change to vet needs to be carefully gated. New analyzers are pretty easy, but changes to existing analyzers may not be.

Is there a horizon for how many versions back we support? I think the answer is no, because the toolchain itself supports all past language versions, so there's no requirement that a user ever update their go version.

I'm not sure I quite understand the need for a -go/-lang flag. Is the proposed toolchain upgrade process: install toolchain vN+1, run go vet -go vN+1, fix errors, run go get go@vN+1, run go test all, fix test failures, commit? Why is that better than: install toolchain vN+1, run go get go@vN+1, run go test all, fix vet and test failures, commit?

[ianlancetaylor]

I'm not sure I quite understand the need for a -go/-lang flag.

For a change like the one that woke this issue up again, #60529, the vet check is perfectly reasonable for old code. The reason we are gating it on the language version is so that when we do a new tool release, we don't surprisingly break existing code before people are ready to upgrade. But there is another class of people who for whatever reason can't yet upgrade to a new version of the compiler, but can run the new version of vet. They would benefit from getting the new reports, which are, after all, still valid concerns about their code.

[ChrisHines]

My desire for a -go flag derives from maintaining a large code base spread across roughly 100 modules. With that many modules it takes time to go through and fix all the errors and we want to do that before running go get go@vN+1.

I don't want to have to worry about forgetting to revert the changes. I might also want to run the go vet check on code that has some uncommitted changes and don't want to deal with reverting only the changes resulting from a go upgrade I'm not planning to keep at the moment.

We target upgrading to the latest major version of Go ~3 months after the first release of that version. A few weeks before we plan to upgrade we start investigating the impact and working through the list in parallel with other work. As part of that process I typically run a script that collects all the new go vet errors into a single text file to get a preview of how many errors we need to fix before we can upgrade. We also have to take into account the dependency graph of our modules because we have automated bumping the version of our internal modules after successful builds and transitively go getting the new versions in all the dependent modules. As a result we need to be pretty sure that all the issues are dealt with before we start committing go major version upgrades.

[ChrisHines]

Ideally I'd like to run something like the following to get the list of problems to fix before upgrading and it wouldn't modify any files in the module.

GOTOOLCHAIN=govN+1 go vet -go=vN+1

Then I can fix those issues over time, committing them incrementally across many modules, before running go get go@vN+1 and committing that.

[findleyr]

Just to reiterate, the policy would be: any newly added or strengthened vet checks would need to be gated on the Go version they are released in. Correct?

Is this going to make maintenance of vet more difficult?

That's correct, and yes, this will make maintenance of vet more difficult. I think if we decide on this policy, we could make such gating easier, for example by making it table-driven, but it is nevertheless true that vet will get more complicated as it has a new dimension to worry about.

An alternative would be to simply not add checks to vet that don't warrant immediate fixes. The two checks that led to these discussions (#57059 and #60529) both highlight real but perhaps mild bugs. However, I think doing this would really limit the amount of help vet can offer in writing correct code.

I wonder if a better path forward might be to simply make it easier to fix vet errors. #60529 came with a fix (and #57059 could have included a fix). If the error message suggested running go vet -fix to address the new findings, perhaps it would significantly reduce the cost of the new checks.

We may also learn from the 1.24 release. I conservatively added version gating to the check in #60529. I wonder if it will be confusing for users to encounter new vet errors based on their go.mod version rather than go command version.

The correct decision here is less clear to me now than it was last week.

[thepudds]

Overall, I think this would be a very good change from the community perspective, and very much in keeping with the increased focus on backwards compatibility (#56986 and friends).

Is there a horizon for how many versions back we support? I think the answer is no, because the toolchain itself supports all past language versions, so there's no requirement that a user ever update their go version.

Rather than support back forever, I think it could be reasonable to pick some horizon for how many versions back to support.

There is some precedent for doing so. The GODEBUG compatibility settings can age out after 4 releases / two years (for many of them; some are supported longer). go mod tidy supports a -compat flag that allows for older behavior. The proposal for that (#46141) says it is "best effort", and I think it's imperfect if you go back too far.

I think the main goal here could be to reduce how often a vet-related disruption catches people somewhat by surprise or with short notice. Even just supporting back to the prior major release would be an improvement and give people more time to adjust. Or it could be something like the norm is to support back 2 releases, but explicitly state in some cases it might only be 1 or 0 if the complexity tradeoff is not warranted or the risk is estimated to be low enough.

[ChrisHines]

#60529 came with a fix

It did? I think that's worth adding to the release notes.

If the error message suggested running go vet -fix to address the new findings, perhaps it would significantly reduce the cost of the new checks.

That would help, but I would want to be able to run the fix without first running go get go@vN+1. I want to fix the problems before upgrading Go so that I can perform the upgrade across multiple modules that depend on each other without worrying about a lower level module forcing a Go version upgrade on dependent modules before they are all fixed.