[Perf] Windows/x64: Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common 2/28/2024 9:23:39 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.22621
Queue	OwlWindows
Baseline	8538e722e1f30c526827c7c9a6abfbee5ff3b164
Compare	5742895d7c7493dfae4ac40ab36019995d256dd1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ElementAt - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.60 ns	5.38 ns	3.36	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

System.Linq.Tests.Perf_Enumerable.ElementAt(input: IList)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.22621
Queue	OwlWindows
Baseline	8538e722e1f30c526827c7c9a6abfbee5ff3b164
Compare	4c90107c80a7f8eb2f38f1494b4e17d48d5c7828
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Backtracking - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	80.90 ns	98.95 ns	1.22	0.20	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: Compiled)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.22621
Queue	OwlWindows
Baseline	d06ebfee1cfd1ef437784013e93ccfcd31334ac0
Compare	5742895d7c7493dfae4ac40ab36019995d256dd1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.BenchF.Whetsto

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Test - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	573.27 ms	614.59 ms	1.07	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.Whetsto*'

Payloads

Baseline
Compare

Benchstone.BenchF.Whetsto.Test

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[DrewScoggins]

This is the range of commits, but there is nothing that seems to jump out, but we are seeing this regression across all of our configurations.

d06ebfe...e2acec9

[jeffschwMSFT]

perhaps there is more than 1 reason for these regressions. given there is a Linq regression, I think that this change should be considered: e101ae2

[ghost]

Tagging subscribers to this area: @dotnet/area-system-linq
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.22621
Queue	OwlWindows
Baseline	8538e722e1f30c526827c7c9a6abfbee5ff3b164
Compare	5742895d7c7493dfae4ac40ab36019995d256dd1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ElementAt - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.60 ns	5.38 ns	3.36	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

System.Linq.Tests.Perf_Enumerable.ElementAt(input: IList)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Author:	performanceautofiler[bot]
Assignees:	-
Labels:	area-System.Linq, os-windows, tenet-performance, tenet-performance-benchmarks, arch-x64, untriaged, runtime-coreclr, needs-area-label
Milestone:	-

[stephentoub]

perhaps there is more than 1 reason for these regressions. given there is a Linq regression, I think that this change should be considered: e101ae2

This could be related to the ElementAt test, and I can take a look at that one to see if I can repro. I don't think it could be related to the other two.

[eiriktsarpalis]

Presumably fixed by #99437

[stephentoub]

@eiriktsarpalis, what about the other tests?

[eiriktsarpalis]

I hadn't noticed that more regressions had been appended by the bot as a comment. Is that common?

[stephentoub]

All tests look to be back in normal ranges.