-
Notifications
You must be signed in to change notification settings - Fork 17.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
build: build failure on darwin builders since CL 536399 [consistent failure] #70402
Comments
Found new dashboard test flakes for:
2024-11-17 14:32 gotip-darwin-amd64-nocgo go@f9a95b1b [build] (log)
2024-11-17 14:32 gotip-darwin-amd64-nocgo go@44d4b699 [build] (log)
2024-11-17 20:17 gotip-darwin-amd64-nocgo go@63f762bc [build] (log)
2024-11-17 20:21 gotip-darwin-amd64-nocgo go@a867e5e5 [build] (log)
|
Found new dashboard test flakes for:
2024-11-17 21:09 gotip-darwin-amd64-nocgo go@01e1e5c2 [build] (log)
|
Found new dashboard test flakes for:
2024-11-17 20:16 gotip-darwin-amd64-nocgo go@04807d3a [build] (log)
|
cc @aclements |
Found new dashboard test flakes for:
2024-11-17 14:32 gotip-darwin-amd64-longtest go@f9a95b1b [build] (log)
2024-11-17 14:32 gotip-darwin-amd64_13 go@f9a95b1b [build] (log)
2024-11-17 14:32 gotip-darwin-amd64_14 go@f9a95b1b [build] (log)
2024-11-17 14:32 gotip-darwin-arm64_13 go@f9a95b1b [build] (log)
2024-11-17 14:32 gotip-darwin-amd64-longtest go@44d4b699 [build] (log)
2024-11-17 14:32 gotip-darwin-amd64_13 go@44d4b699 [build] (log)
2024-11-17 14:32 gotip-darwin-amd64_14 go@44d4b699 [build] (log)
2024-11-17 14:32 gotip-darwin-arm64_13 go@44d4b699 [build] (log)
2024-11-17 20:16 gotip-darwin-amd64_13 go@04807d3a [build] (log)
2024-11-17 20:16 gotip-darwin-amd64_14 go@04807d3a [build] (log)
2024-11-17 20:16 gotip-darwin-arm64_13 go@04807d3a [build] (log)
2024-11-17 20:17 gotip-darwin-amd64-longtest go@63f762bc [build] (log)
2024-11-17 20:17 gotip-darwin-amd64_13 go@63f762bc [build] (log)
2024-11-17 20:17 gotip-darwin-amd64_14 go@63f762bc [build] (log)
2024-11-17 20:17 gotip-darwin-arm64_13 go@63f762bc [build] (log)
2024-11-17 20:21 gotip-darwin-amd64-longtest go@a867e5e5 [build] (log)
2024-11-17 20:21 gotip-darwin-amd64_13 go@a867e5e5 [build] (log)
2024-11-17 20:21 gotip-darwin-amd64_14 go@a867e5e5 [build] (log)
2024-11-17 20:21 gotip-darwin-arm64_13 go@a867e5e5 [build] (log)
2024-11-17 21:09 gotip-darwin-amd64-longtest go@01e1e5c2 [build] (log)
2024-11-17 21:09 gotip-darwin-amd64_13 go@01e1e5c2 [build] (log)
2024-11-17 21:09 gotip-darwin-amd64_14 go@01e1e5c2 [build] (log)
2024-11-17 21:09 gotip-darwin-arm64_13 go@01e1e5c2 [build] (log)
2024-11-18 02:08 gotip-darwin-amd64_13 go@d1180dbd [build] (log)
2024-11-18 02:08 gotip-darwin-arm64_13 go@d1180dbd [build] (log)
|
Looking closely at https://ci.chromium.org/ui/p/golang/builders/ci/gotip-darwin-amd64_13/b8730968502183490337/overview I think what's going on is that we have this one new line in the middle of the
And this new "build-output" action is confusing the LUCI test reader:
Prior to this change (example), the dist JSON output just had text embedded in it:
|
Found new dashboard test flakes for:
2024-11-18 02:08 gotip-darwin-amd64-nocgo go@d1180dbd [build] (log)
2024-11-18 02:08 gotip-darwin-amd64_14 go@d1180dbd [build] (log)
2024-11-18 02:09 gotip-darwin-amd64-nocgo go@90b1dc01 [build] (log)
2024-11-18 02:09 gotip-darwin-amd64_13 go@90b1dc01 [build] (log)
2024-11-18 02:09 gotip-darwin-arm64_13 go@90b1dc01 [build] (log)
|
Change https://go.dev/cl/628955 mentions this issue: |
Marking as a release blocker because we either need to re-enable build JSON from |
Found new dashboard test flakes for:
2024-11-18 02:09 gotip-darwin-amd64-longtest go@90b1dc01 [build] (log)
2024-11-18 02:09 gotip-darwin-amd64_14 go@90b1dc01 [build] (log)
|
Unfortunately, this is tripping up the LUCI test output processor, so we need to disable it until we can figure that out. For #70402. Updates #62067. Cq-Include-Trybots: luci.golang.try:gotip-darwin-arm64_13,gotip-linux-amd64-longtest Change-Id: I9ae722218e98b8060b8b4c46358f23381ac8537a Reviewed-on: https://go-review.googlesource.com/c/go/+/628955 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
Found new dashboard test flakes for:
2024-11-18 02:08 gotip-darwin-amd64-longtest go@d1180dbd [build] (log)
|
Updating this with some details that I'm seeing so far. The For TestEvent objects, the test ID was constructed from the Package and Test fields. The new BuildEvent objects populate the ImportPath field instead of a Package field, and so they end up having an empty ID associated, which get rejected by the server. The fix for this particular problem should be, when the Package field is empty, to consider the new ImportPath field and use it to generate the ID. As @mknyszek pointed out, result_adapter already intends to ignore unknown actions, so those might not need any special handling to get the baseline behavior back. I'll look more into that after handling the test IDs. |
Change https://go.dev/cl/629335 mentions this issue: |
I'm not sure if that's the right fix. The If a build event never matches up with a test, then it might make sense to construct a result for it independently using the |
Change https://go.dev/cl/629375 mentions this issue: |
Two reasons: to buy more time to work on the result_adapter/ResultDB side (beyond just the bare minimum behavior of not regressing beyond what we had before), and to test that using a GODEBUG for this works as we expect. It already uncovered the need to merge multiple GODEBUG settings that we hadn't run into earlier. For golang/go#70402. Change-Id: I9b86f6a5462637eef84f3efca9fc31d11657d756 Reviewed-on: https://go-review.googlesource.com/c/build/+/629375 Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Michael Knyszek <mknyszek@google.com>
Updated plan, which takes into account https://pkg.go.dev/cmd/go@master#hdr-Build__json_encoding and #70402 (comment), is to determine that a given JSON object is a BuildEvent rather than a TestEvent by checking its Action field and seeing a "build-" prefix, and then attach the build output to the test record corresponding to the package. Out of the options currently available at https://pkg.go.dev/go.chromium.org/luci/resultdb/proto/v1#TestStatus, TestStatus_FAIL will do. So the main observable difference will be that the package build error output will be attached to the package's test record, instead of only being visible in the non-structured output. I have a working prototype of that, need to clean it up a bit before mailing to handle edge cases. Edit: Filed #70435 to track the LUCI side of this work. |
Issue created automatically to collect these failures.
Example (log):
— watchflakes
The text was updated successfully, but these errors were encountered: