[compiletest] Ignore known paths when abbreviating output

[pietroalbini]

To prevent out of memory conditions, compiletest limits the amount of output a test can generate, abbreviating it if the test emits more than a threshold. While the behavior is desirable, it also causes some issues (like #96229, #94322 and #92211).

The latest one happened recently, when the src/test/ui/numeric/numeric-cast.rs test started to fail on systems where the path of the rust-lang/rust checkout is too long. This includes my own development machine and LLVM's CI. Rust's CI uses a pretty short directory name for the checkout, which hides these sort of problems until someone runs the test suite on their own computer.

When developing the fix I tried to find the most targeted fix that would prevent this class of failures from happening in the future, deferring the decision on if/how to redesign abbreviation to a later date. The solution I came up with was to ignore known base paths when calculating whether the output exceeds the abbreviation threshold, which removes this kind of nondeterminism.

This PR is best reviewed commit-by-commit.

[rust-highfive]

r? @Mark-Simulacrum

(rust-highfive has picked a reviewer for you, use r? to override)

[Mark-Simulacrum]

I'm OK with approving this with the nit fixed. Still feels a little unfortunate/off to me, but I'm not seeing massively better options, particularly if there's a soft goal of trying to prevent 'excessively large' outputs from rustc. (Not that several hundred kilobytes of JSON isn't already quite big for the ~1 KB input we typically have...).

[pietroalbini]

Addressed the last suggestion you wrote. When fixing it, the amount of length manipulations we're doing started worrying me without tests, so I added them in another commit.

Those tests also uncovered an existing bug (if the first extend was longer than the maximum size it would not be truncated, but its excess size would still be counted for the "skipped n bytes" message), which I then fixed 🙂

[Mark-Simulacrum]

FWIW, I think we should make the read2_abbreviated return an error or panic instead of inserting this skipped message -- that typically leads to a JSON de-serialization failure and a pretty opaque one at that, which I think isn't too helpful -- we can probably say something more useful like "captured X bytes, which is above the limit in compiletest. See abbreviated output: {output}".

(Can be out of scope for this PR).

[pietroalbini]

Yeah, let's do that in a separate PR though.

[pietroalbini]

@bors ready

[Mark-Simulacrum]

I think this is going to run into some problems if we have overlapping matches (e.g., one of the patterns is /home/mark and the other is /home/mark/rust, then we'll subtract out twice).

I think src_base and build_base won't do that in practice so we're OK for now, but ideally we'd either (a) assert that it doesn't happen or (b) somehow fix it. I'm not quite sure what the best approach to fixing it would be -- trying to detect this kind of overlap seems annoying.

[Mark-Simulacrum]

r=me if you don't want to deal with this additional problem, but maybe add a comment about it to where we define paths so we think about it.

[pietroalbini]

@bors r=Mark-Simulacrum

There is already a comment about that where the filtered paths are defined.

[bors]

📌 Commit 8ea9598 has been approved by Mark-Simulacrum

[bors]

⌛ Testing commit 8ea9598 with merge 6609c67...

[bors]

☀️ Test successful - checks-actions
Approved by: Mark-Simulacrum
Pushing 6609c67 to master...

[rust-timer]

Finished benchmarking commit (6609c67): comparison url.

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

• Primary benchmarks: mixed results
• Secondary benchmarks: 😿 relevant regression found

	mean1	max	count2
Regressions 😿 (primary)	0.1%	0.1%	1
Regressions 😿 (secondary)	2.4%	2.4%	1
Improvements 🎉 (primary)	-3.0%	-3.0%	1
Improvements 🎉 (secondary)	N/A	N/A	0
All 😿🎉 (primary)	-1.4%	-3.0%	2

Cycles

Results

• Primary benchmarks: 😿 relevant regression found
• Secondary benchmarks: mixed results

	mean1	max	count2
Regressions 😿 (primary)	3.0%	3.0%	1
Regressions 😿 (secondary)	2.2%	2.2%	1
Improvements 🎉 (primary)	N/A	N/A	0
Improvements 🎉 (secondary)	-3.6%	-3.6%	1
All 😿🎉 (primary)	3.0%	3.0%	1

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

@rustbot label: -perf-regression

Footnotes

1. the arithmetic mean of the percent change ↩ ↩²

2. number of relevant changes ↩ ↩²