proposal: testing: automatically scale benchmark metrics in results output

[jpap]

When performing a benchmark, we can currently use SetBytes(int64) to set the number of input or output bytes per run, and get a measurement like X MB/s in the output.

It would be nice to be able to set an arbitrary unit instead of "bytes", and to provide multiple scales instead of just "mega" (1e6).

Example 1: pixels per second (px/s): b.SetUnits(1920*1080, "px/s") when benchmarking a full-HD image would print to the output one of the following, depending on the scale of the result:

• "X px/s"
• "X kpx/s"
• "X Mpx/s"
• "X Gpx/s"

Example 2: frames per second (fps): b.SetUnits(1, "fps") when benchmarking one frame operation.

The existing implementation of SetBytes(b int64) could just call SetUnits(b, "B/s") for backwards compatibility.

[agnivade]

I believe this https://tip.golang.org/pkg/testing/#B.ReportMetric is what you want.

[toothrot]

@jpap Thanks for writing this up. Does @agnivade's suggestion work for you?

[jpap]

Thanks @agnivade, it's great that arbitrary units of work are supported in Go 1.13+.

@toothrot, the second part to this issue, auto-scaling the results {_, k, M, G} instead of scaling by 1e6 (M) uniformly as is done now, still remains and doesn't appear to be already implemented (having just checked tip).

[toothrot]

Great, glad to hear the arbitrary units work (#26037) will help! @jpap Let me know if I captured your request correctly in the title.

@aclements Thoughts on this?

[jpap]

@toothrot, it would be great if the unit-scaling applied to all reported metrics in the benchmark data format, not just bytes/sec (MB/s).

[rsc]

/cc @aclements for thoughts