x/build: decide status of Android TryBots

[bcmills]

The android-amd64-emu builder currently runs as a TryBot in the default set. However, android/amd64 is not a first-class port, and has some open issues that can cause test failures in TryBot runs (notably #51001, #52724, #42212).

Discussion #53060 proposes to run only first-class ports as TryBots, but I think we should reevaluate the status of android TryBot in particular independent of that discussion, because we have specific known failure modes affecting the android TryBot today.

CL 407615 would remove this TryBot for the go repo, and #25963 discusses misc-compile TryBots that could plausibly take its place.

In order to avoid TryBot failure noise when the tree reopens for Go 1.20, we should decide on the status of this TryBot (and update cmd/coordinator accordingly) before completing the Go 1.19 release cycle.

(CC @eliben @golang/android @golang/release)

[gopherbot]

Change https://go.dev/cl/412174 mentions this issue: dashboard: add known issues for android-*-emu

[gopherbot]

Change https://go.dev/cl/407615 mentions this issue: dashboard: exclude android-amd64-emu from main go repo's TryBot set

[dmitshur]

As a small correction, only the android-amd64-emu builder is configured as a TryBot. The android-386-emu one is a post-submit only builder for all repos.

[bcmills]

Thanks. Updated the first comment accordingly.

[heschi]

We think that this should be removed from the TryBot set: it's pretty noisy and there isn't a clear path to fixing it. If nobody disagrees we'll do that.

cc @eliasnaur, who may be interested

[dmitshur]

I think in the current state it makes sense for the android-amd64-emu builder to be post-submit only.

If I understand correctly, @cherrymui may support that too, based on this comment on CL 407615. (I'll update that CL once this issue moves to NeedsFix state.)

[changkun]

Just clarify questions:
What's the exact consequence of removing a builder from TryBots?
Does it mean that this port will never be tested unless explicitly requested?
Is it going to be similar to how iOS living today? (i.e. unless try=ios, the builder will only be tested after a CL is submitted?)
Instead of removing it from TryBots will at least require doing a proper upgrade, is that correct?

[dmitshur]

Is it going to be similar to how iOS living today? (i.e. unless try=ios, the builder will only be tested after a CL is submitted?)

Yes, this is correct. "Removing the builder from the TryBot set" would mean it will not run automatically before CL submission when Run-TryBot+1 vote is set, but can be requested via SlowBots syntax (i.e., TRY=android or TRY=android-amd64). The android-amd64-emu builder will run after a CL is submitted (aka post-submit builds, as visible on the build dashboard), that's not changing here. This is indeed very similar to the iOS situation which doesn't have any builders in the default TryBot set, only as post-submit builders.

I think this answers other questions, but let me know if I missed something. (I didn't quite understand what "a proper upgrade" refers to in the last question.)