libtest: Fix padding of benchmarks run as tests

[Enselic]

Summary

The first commit adds regression tests for libtest padding.

The second commit fixes padding for benches run as tests and updates the blessed output of the regression tests to make it clear what effect the fix has on padding.

Closes #104092 which is E-help-wanted and regression-from-stable-to-stable

More details

Before this fix we applied padding before manually doing what convert_benchmarks_to_tests() does which affects padding calculations. Instead use convert_benchmarks_to_tests() first if applicable and then apply padding afterwards so it becomes correct.

Benches should only be padded when run as benches to make it easy to compare the benchmark numbers. Not when run as tests.

r? @ghost until CI passes.

[Enselic]

r? @saethlin who last touched the code I am changing

[saethlin]

This is good work and very well-tested, but I'm not a member of the library team. I'm asking on Zulip about who should be approving this.

[Enselic]

r? @thomcc as discussed on Zulip.

[thomcc]

r? @thomcc

[thomcc]

Seems good to me! Thanks!

@bors r+

[bors]

📌 Commit 809b2ad has been approved by thomcc

It is now in the queue for this repository.

[bors]

⌛ Testing commit 809b2ad with merge f6543a7...

[bors]

💔 Test failed - checks-actions

[Enselic]

I had to replace \+ with \{1,\} in the normalization regex to make it work with both macOS/BSD sed and Linux/GNU sed.

@thomcc Can you r+ again please? Thanks! Diff from what you already approved.

[Enselic]

@thomcc Friendly ping in case you missed my first ping (the diff from what you already approved is trivial).

[ChrisDenton]

I'm happy to approve the diff.

@bors r=thomcc,ChrisDenton

[bors]

📌 Commit 12e6bcf has been approved by thomcc,ChrisDenton

It is now in the queue for this repository.

[bors]

⌛ Testing commit 12e6bcf with merge 5431404...

[bors]

☀️ Test successful - checks-actions
Approved by: thomcc,ChrisDenton
Pushing 5431404 to master...

[rust-timer]

Finished benchmarking commit (5431404): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-3.7%	[-3.7%, -3.7%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-3.7%	[-3.7%, -3.7%]	1

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.5%	[0.5%, 0.5%]	1
Regressions ❌ (secondary)	3.8%	[3.8%, 3.8%]	1
Improvements ✅ (primary)	-1.2%	[-1.3%, -1.1%]	2
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.6%	[-1.3%, 0.5%]	3

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 666.091s -> 666.39s (0.04%)
Artifact size: 308.42 MiB -> 308.38 MiB (-0.01%)