Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add optional module-level Summary #108

Open
TheRealChssAddct opened this issue Jan 4, 2024 · 3 comments
Open

Add optional module-level Summary #108

TheRealChssAddct opened this issue Jan 4, 2024 · 3 comments

Comments

@TheRealChssAddct
Copy link

TheRealChssAddct commented Jan 4, 2024

Just found tparse and really like it. For modules with many packages, the package-level summary is extremely useful.

I would love to see an optional (default off) module-level summary table. Just a simple rollup aggregation of the stats in the Package summary.

Essentially, the module-level summary information provided at the end of gotestsum output, but in the nice tparse packages-level table format.
My preference would be in a separate table from the Packages summary

  • Tests summary
  • Packages summary
  • Module summary

But I can see some might like it as an additional row in the Packages summary, with a package name like Total or Aggregate, probably with a dividing line above it to delineate from package summary rows.

My main use-case: I am now using tparse in github actions which run unit tests and prevent a PR merge if any tests fail.
I debated briefly elevating the tparse output automatically to be added as a comment and rejected immediately as failure output way too much for a comment, plus with multiple test runs as new commits come in, the PR fills up with many very long comments.
I debated somewhat longer adding just the packages-level summary from the output as an automated comment, but also rejected: still too noisy on the PR. We use draft PRs to master that may stay open several weeks, while work proceeds with multiple intermediate PRs to a test branch.

If I had a module-level summary, I would definitely automate adding this as a PR comment. Even if many commits came in while the PR is open and thus many automated comments pile up, they are succinct - just an overall totals of pass, fail, skip.

@TheRealChssAddct TheRealChssAddct changed the title module-level summary Add optional module-level Summary Jan 4, 2024
@mfridman
Copy link
Owner

Thanks for filing an issue. Is it possible to dump (even just a repro) an example of go test -json with multiple modules?

I don't mind adding flagging and aggregating one level up (by module), but would be helpful to see the raw JSON we'd be working with...

@TheRealChssAddct
Copy link
Author

I can certainly do a bit of research. To be honest, the thought of multiple modules hadn't occurred to me. For my use case - a single module per repo - I expected the module level summary to have just a single row in the table. But sourcing multiple modules in a single repo is a legitimate arrangement per this documentation, so it must be considered. It might be that go test would have to be run individually for each module and the json output merged. I've never tried, but I'll find out!

@TheRealChssAddct
Copy link
Author

TheRealChssAddct commented Jan 10, 2024

Ok. My research confirms my suspicion. 'go test' will only test a single module. To test multiple modules, you would script running go test for each module using something like
find . -name go.mod -execdir go test ./... \;

So I suggest the complexity of merging multiple go test output files then feeding that to tparse isn't worth the effort (unless that's something already supported by tparse). Rather, module level summary would be expected to be a single row aggregating the action counts of all the packages (which would be the same as aggregating for all tests, regardless of package).

I'd be willing to contribute a branch/PR with this change, if you like the idea, tho it may be a couple/few weeks as we kick off Q1 work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants