go/test2json: panic in goroutine results in missing Action=Fail TestEvent

[dnephin]

Related to #37555

What version of Go are you using (go version)?

$ go version
go version go1.14.2 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOOS=linux
GOARCH=amd64

What did you do?

package mypkg

import (
	"testing"
	"time"
)

func Test_PanicInGoroutine(t *testing.T) {
	go func() { panic("panicing in a goroutine") }()
	time.Sleep(10 * time.Millisecond)
}

$ go test 
panic: panicing in a goroutine

goroutine 7 [running]:
gotest.tools/gotestsum/tmp2.Test_PanicInGoroutine.func1()
	/home/daniel/pers/code/gotestsum/tmp2/main_test.go:9 +0x39
created by gotest.tools/gotestsum/tmp2.Test_PanicInGoroutine
	/home/daniel/pers/code/gotestsum/tmp2/main_test.go:9 +0x35
FAIL	gotest.tools/gotestsum/tmp2	0.003s
FAIL

$ go test -json
{"Time":"2020-04-11T18:36:29.705478289-04:00","Action":"run","Package":"gotest.tools/gotestsum/tmp2","Test":"Test_PanicInGoroutine"}
{"Time":"2020-04-11T18:36:29.705575001-04:00","Action":"output","Package":"gotest.tools/gotestsum/tmp2","Test":"Test_PanicInGoroutine","Output":"=== RUN   Test_PanicInGoroutine\n"}
{"Time":"2020-04-11T18:36:29.707652936-04:00","Action":"output","Package":"gotest.tools/gotestsum/tmp2","Test":"Test_PanicInGoroutine","Output":"panic: panicing in a goroutine\n"}
{"Time":"2020-04-11T18:36:29.707663408-04:00","Action":"output","Package":"gotest.tools/gotestsum/tmp2","Test":"Test_PanicInGoroutine","Output":"\n"}
{"Time":"2020-04-11T18:36:29.707666872-04:00","Action":"output","Package":"gotest.tools/gotestsum/tmp2","Test":"Test_PanicInGoroutine","Output":"goroutine 7 [running]:\n"}
{"Time":"2020-04-11T18:36:29.707669584-04:00","Action":"output","Package":"gotest.tools/gotestsum/tmp2","Test":"Test_PanicInGoroutine","Output":"gotest.tools/gotestsum/tmp2.Test_PanicInGoroutine.func1()\n"}
{"Time":"2020-04-11T18:36:29.707675539-04:00","Action":"output","Package":"gotest.tools/gotestsum/tmp2","Test":"Test_PanicInGoroutine","Output":"\t/home/daniel/pers/code/gotestsum/tmp2/main_test.go:9 +0x39\n"}
{"Time":"2020-04-11T18:36:29.707688443-04:00","Action":"output","Package":"gotest.tools/gotestsum/tmp2","Test":"Test_PanicInGoroutine","Output":"created by gotest.tools/gotestsum/tmp2.Test_PanicInGoroutine\n"}
{"Time":"2020-04-11T18:36:29.707695646-04:00","Action":"output","Package":"gotest.tools/gotestsum/tmp2","Test":"Test_PanicInGoroutine","Output":"\t/home/daniel/pers/code/gotestsum/tmp2/main_test.go:9 +0x35\n"}
{"Time":"2020-04-11T18:36:29.707843506-04:00","Action":"output","Package":"gotest.tools/gotestsum/tmp2","Output":"FAIL\tgotest.tools/gotestsum/tmp2\t0.003s\n"}
{"Time":"2020-04-11T18:36:29.707855313-04:00","Action":"fail","Package":"gotest.tools/gotestsum/tmp2","Elapsed":0.003}

What did you expect to see?

In go1.13 and earlier I received the following TestEvent:

{"Time":"2020-04-11T22:37:41.54673436Z","Action":"fail","Package":"gotest.tools/gotestsum/tmp2","Test":"Test_PanicInGoroutine","Elapsed":0.004}

What did you see instead?

That event was missing. There is an Action=fail for the package, but none for the test.

[andybons]

@bcmills @jayconrod @matloob

[bcmills]

The new output seems correct to me. The goroutine that panics may not be related in any way to the test that was running during the panic, so the failure should not be attributed to that specific test.

(Consider, for example: https://play.golang.org/p/C9JJsDkzOhM.)

[jayconrod]

+1. Not sure whether this was changed intentionally, but the new behavior seems more correct. A panic in a goroutine other than the one running a test can't always be attributed to a specific test. It could have been started by a parallel test, a previous test, TestMain, or something else.

[dnephin]

Shouldn't any test which has an Action=run have a corresponding Action=pass or fail ? That seemed to be the case in previous versions of Go. This test has failed, but there is no Action=fail for the test.

The goroutine that panics may not be related in any way to the test that was running during the panic, so the failure should not be attributed to that specific test.

The panic is already associated with the test. The output TestEvent has the test name on it.

[bcmills]

Shouldn't any test which has an Action=run have a corresponding Action=pass or fail ?

No. If the process exits before the test function completes, and nothing in the test so far has invoked (*T).Fail then the test has neither passed nor failed, so it would be inaccurate to report any result for it. (#29062 is related.)

[bcmills]

The panic is already associated with the test. The output TestEvent has the test name on it.

Aha, that is a bug!

[bcmills]

But maybe that bug is more helpful than it is harmful: the goroutine isn't necessarily associated with the currently-running test, but background-goroutine panics probably do relate to the running test more often than not.