go test -coverprofile fails under Go 1.20 #199
Comments
|
You're absolutely right, and I've been noticing these failures with Go tip for some weeks now. I wrote our code to collect and merge coverage profiles years ago, before upstream planned on doing the same, but properly :) So our code is not just obsolete, but also apparently broken now. I need to delete it, and check that the new |
Our code was a fairly hacky version of what Go 1.20 does for us, since we had to externally reach into the testing internals to do the right thing before and after each program execution. With Go 1.20, all we actually need to do is ensure that the GOCOVERDIR environment variable is forwarded properly. With that, test binaries will know how to produce multiple coverage profiles, and "go test" will know how to collect and merge them. We could keep our hacky workaround for the sake of "deep" coverage information for Go 1.19, but that doesn't seem worthwhile. The old mechanism caused test flakes like rogpeppe#130, is incompatible with Go 1.20, and is overall worse than what Go 1.20 can do. If a user wants code coverage with the latest testscript version, they can use Go 1.20. If they are stuck on Go 1.19 and need code coverage, I imagine they can also stick to a slightly older testscript version. On a main package, the old testscript with Go 1.19 reports: PASS coverage: 8.0% of statements total coverage: 90.1% of statements ok mvdan.cc/sh/v3/cmd/shfmt 0.063s The new testscript with Go 1.20 reports: PASS mvdan.cc/sh/v3/cmd/shfmt coverage: 90.1% of statements ok mvdan.cc/sh/v3/cmd/shfmt 0.047s Fixes rogpeppe#130, as the RemoveAll call is no longer present. Fixes rogpeppe#161, as the API is now deprecated. Fixes rogpeppe#199, as "go test -coverprofile" now works on Go 1.20.
Our code was a fairly hacky version of what Go 1.20 does for us, since we had to externally reach into the testing internals to do the right thing before and after each program execution. With Go 1.20, all we actually need to do is ensure that the GOCOVERDIR environment variable is forwarded properly. With that, test binaries will know how to produce multiple coverage profiles, and "go test" will know how to collect and merge them. We could keep our hacky workaround for the sake of "deep" coverage information for Go 1.19, but that doesn't seem worthwhile. The old mechanism caused test flakes like rogpeppe#130, is incompatible with Go 1.20, and is overall worse than what Go 1.20 can do. If a user wants code coverage with the latest testscript version, they can use Go 1.20. If they are stuck on Go 1.19 and need code coverage, I imagine they can also stick to a slightly older testscript version. On a main package, the old testscript with Go 1.19 reports: PASS coverage: 8.0% of statements total coverage: 90.1% of statements ok mvdan.cc/sh/v3/cmd/shfmt 0.063s The new testscript with Go 1.20 reports: PASS mvdan.cc/sh/v3/cmd/shfmt coverage: 90.1% of statements ok mvdan.cc/sh/v3/cmd/shfmt 0.047s Fixes rogpeppe#130, as the RemoveAll call is no longer present. Fixes rogpeppe#161, as the API is now deprecated. Fixes rogpeppe#199, as "go test -coverprofile" now works on Go 1.20.
Our code was a fairly hacky version of what Go 1.20 does for us, since we had to externally reach into the testing internals to do the right thing before and after each program execution. With Go 1.20, all we actually need to do is ensure that the GOCOVERDIR environment variable is forwarded properly. With that, test binaries will know how to produce multiple coverage profiles, and "go test" will know how to collect and merge them. We could keep our hacky workaround for the sake of "deep" coverage information for Go 1.19, but that doesn't seem worthwhile. The old mechanism caused test flakes like rogpeppe#130, is incompatible with Go 1.20, and is overall worse than what Go 1.20 can do. If a user wants code coverage with the latest testscript version, they can use Go 1.20. If they are stuck on Go 1.19 and need code coverage, I imagine they can also stick to a slightly older testscript version. On a main package, the old testscript with Go 1.19 reports: PASS coverage: 8.0% of statements total coverage: 90.1% of statements ok mvdan.cc/sh/v3/cmd/shfmt 0.063s The new testscript with Go 1.20 reports: PASS mvdan.cc/sh/v3/cmd/shfmt coverage: 90.1% of statements ok mvdan.cc/sh/v3/cmd/shfmt 0.047s Fixes rogpeppe#130, as the RemoveAll call is no longer present. Fixes rogpeppe#161, as the API is now deprecated. Fixes rogpeppe#199, as "go test -coverprofile" now works on Go 1.20.
|
I was also getting this error. Updating to latest (v1.12.0) fixed the error, but I'm not getting the total coverage. In this walkthrough, it shows that running But mine shows with no Everything else works. I have multiple test passing and changing the expected values fails the tests as expected. It seems that everything is being run correctly. Do I need to enable something to get the total coverage? I'm using Go 1.21. |
|
I found the same in the example repo I set up a test repo. I can't get deep coverage, only one package down, it seems. |
|
Ignore the old output saying "total coverage"; the new output simply outputs that percentage directly. |
|
Ok, I can see that in my test repo. But the cover.out file still does not include the line coverage information for the other packages. I need the cover profile for all the packages to publish to my coverage reporter (SonarQube). |
|
But using my custom bash script it outputs the coverage profile correctly |
See |
|
|
I've upgraded go to 1.20 and suddenly I'm getting this error from testscript:
Seems like it might be related to coverage changes documented here: https://go.dev/testing/coverage/
Removing
-coverprofile-...flag allows my tests to pass.The text was updated successfully, but these errors were encountered: