x/tools/gopls: clarify supported operating systems

[findleyr]

We have experienced a fair amount of friction with gopls tests on e.g. plan9, android, illumos, and solaris. While test failures on these operating systems occasionally turn up real gopls bugs, in my experience failures are more often due to builder problems, resource constraints, and/or bugs in the toolchain. Given that (I assume) few if any of our users are on some of these operating systems, it is almost certain that our resources devoted to these builders results in a net loss for our users.

Unlike other projects on the Go team, gopls is designed for a relatively narrow use-case: development environments. We should not expend resources on ports that are unlikely to serve this use-case for a large number of users. However, we do not specify supported operating systems in our README.

We should specify precisely which operating systems we aim to support, and disable builders for others. At the very least, we will support linux, windows, darwin, and other BSD-like operating systems. I am open to others, but do not want to commit to support if there is no demand.

CC @bcmills @adonovan

[gopherbot]

Change https://go.dev/cl/492495 mentions this issue: gopls/internal/lsp/filecache: skip TestConcurrency on plan9 builder

[bcmills]

I agree that we do not need to explicitly support android, although after https://go.dev/cl/488655 I suspect that the two Android-specific skips in gopls might no longer be necessary.

(The only other recent changes that have been needed for Android are also needed to support go test on the gopls module when run as a dependency of another module, which I think we should aim to support, provided that that module also requires an appropriate version of x/tools.)

[bcmills]

The problems on illumos and solaris seem to be due to builder and/or kernel issues. Those have been reported to the port maintainers (as #59379 and #58782 respectively), so per our porting policy I think we can safely ignore gopls failures on those builders until those issues are addressed.

Beyond the builder-specific issues, most of the API workarounds needed for solaris also apply to aix, our other SVR4-based platform.

[bcmills]

As for Plan 9, well, I'm not sure. 😅 The plan9-arm builder, at least, seems to do ok with the gopls tests most of the time.

We occasionally need to add test skips for lack of symlink support, but in theory we also need those for certain versions and/or configurations of Windows, and those skip checks can call general testenv functions that are not specific to Plan 9.

Beyond that we have only two plan9-specific workarounds in place:

• One test skip for x/tools/gopls/internal/regtest/modfile: TestUnknownRevision consistently failing on plan9-386 and plan9-amd64 builders #50477, which remains undiagnosed.
• And one retry loop in (*Workdir).ReadFile to allow gopls to read files that are concurrently opened by go commands.

To be honest, though, I'm not entirely sure why we don't need an analogous retry loop on Windows. 😅

[bcmills]

We should specify precisely which operating systems we aim to support, and disable builders for others.

That may be tricky. Do we currently have a mechanism to skip only a specific submodule within a repo?
I hesitate to drop all of x/tools on a given platform just because gopls doesn't support it.

[findleyr]

@bcmills I think there's a trap in thinking "well, things seem to be working OK now, so why turn them off". Considerable resources (including a lot of your time specifically!) has been spent tracking down these flakes and reporting bugs upstream. While it is greatly appreciated, we have limited resources and the assertion of this issue is that we would be better off focusing those resources on higher priority environments.

That may be tricky. Do we currently have a mechanism to skip only a specific submodule within a repo?
I hesitate to drop all of x/tools on a given platform just because gopls doesn't support it.

I would definitely not turn off all of x/tools. I think we could probably make a coordinator change to skip just x/tools/gopls.

[findleyr]

I think there's a trap in thinking "well, things seem to be working OK now, so why turn them off". Considerable resources (including a lot of your time specifically!) has been spent tracking down these flakes and reporting bugs upstream. While it is greatly appreciated, we have limited resources and the assertion of this issue is that we would be better off focusing those resources on higher priority environments.

Some clarification after re-reading: I don't think that's what you're saying, but we've made this mistake in the past. Also: your efforts thus far have been in service of the important goal of test hygiene and engineering standards. This issue is about committing to that same goal.

[bcmills]

I think mostly what I'm saying is: given that we probably do want to keep testing the rest of x/tools on most platforms, and given the “curb cut” effects of many of the fixes (for example: fixing the tests on Android also fixes them when run from the module cache; fixing symlink issues on plan9 also fixes them in certain Windows configurations), the engineering cost of skipping gopls tests on the less-supported builders might turn out to be higher than the cost of not skipping them.

But I agree that we should be mindful of the costs in both directions.

[bsiegert]

Please don't drop support for the BSDs (FreeBSD, NetBSD, OpenBSD).

[findleyr]

@bsiegert we can do that, but can you explain a bit more? e.g. do you think this affects a large number of users? Is it strategically important to support these GOOS?

[PaluMacil]

@bsiegert we can do that, but can you explain a bit more? e.g. do you think this affects a large number of users? Is it strategically important to support these GOOS?

I don't use a BSD, but I know that these are platforms commonly used as everyday desktops besides just servers. While I do not expect someone to sit down and write software on Android, and I have never met someone with a need for Solaris, plan9, etc, I would expect that a lot of BSD users are writing software while sitting at a BSD desktop.

[findleyr]

@PaluMacil thanks. Yes, I think there are probably some users on BSD, whereas I expect ~zero users on Android or Plan9. Also, generally speaking, BSDs have been less of a problem for us than others.