[Perf] Linux/x64: 6 System.Text.RegularExpressions.Tests Regressions

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	84b33395057737db3ea342a5151feb6b90c1b6f6
Compare	4e626e2dccf5060ab9c50dc3a9baab547ee2b31e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
IsMatch - Duration of single invocation ADX - Test Multi Config Graph	33.54 ns	38.30 ns	1.14	0.03	False
IsMatch - Duration of single invocation ADX - Test Multi Config Graph	34.70 ns	37.50 ns	1.08	0.04	False
IsMatch - Duration of single invocation ADX - Test Multi Config Graph	34.18 ns	38.46 ns	1.13	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 8, Options: Compiled)

ETL Files

Histogram

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 7, Options: Compiled)

ETL Files

Histogram

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 6, Options: Compiled)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	84b33395057737db3ea342a5151feb6b90c1b6f6
Compare	4e626e2dccf5060ab9c50dc3a9baab547ee2b31e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Count - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	322.30 ms	359.17 ms	1.11	0.02	False
Count - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	314.14 ms	344.52 ms	1.10	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{0,2}(Tom|Sawyer|Huckleberry|Finn)", Options: Compiled)

ETL Files

Histogram

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{2,4}(Tom|Sawyer|Huckleberry|Finn)", Options: Compiled)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	84b33395057737db3ea342a5151feb6b90c1b6f6
Compare	4e626e2dccf5060ab9c50dc3a9baab547ee2b31e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Email_IsNotMatch - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	117.60 ns	130.90 ns	1.11	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsNotMatch(Options: IgnoreCase, Compiled)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[LoopedBard3]

Other regressions:

• Windows x64: [Perf] Windows/x64: 1 Regression on 5/6/2024 7:04:45 PM perf-autofiling-issues#34325

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-text-regularexpressions
See info in area-owners.md if you want to be subscribed.

[LoopedBard3]

Potentially related to: #101899, FYI @stephentoub

[stephentoub]

@MihaZupan, is there anything you can think of to reduce the startup overhead of IndexOfAny{Except} with SearchValues further? The difference here essentially stems from replacing this:

int iteration = 0;
while (iteration < 12 && (uint)iteration < (uint)slice.Length && char.IsAsciiLetter(slice[iteration]))
{
    iteration++;
}

with this:

int iteration = slice.Slice(0, Math.Min(slice.Length, 12)).IndexOfAnyExcept(Utilities.s_asciiLetters);
if (iteration < 0)
{
    iteration = Math.Min(slice.Length, 12);
}

The new code is cleaner and better for longer searches, but if the match is found early, it obviously represents a (small) regression. We might just need to accept it.

I did notice that this SearchValues (SearchValues.Create("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz")) is producing a System.Buffers.AsciiCharSearchValues'1[System.Buffers.IndexOfAnyAsciiSearcher+Default]. Should we consider adding a RangeCharSearchValues variant that's ASCII case-insensitive, and would that help to lower the overheads here? Or more generally is there a way we could optimize the various SearchValues<char> variants for ASCII ordinal case-insensitivity?

[MihaZupan]

This is about how it could look like to have an RangeCharPackedIgnoreCase implementation:
main...MihaZupan:runtime:searchvalues-ignoreCaseRange

(note that these numbers are from an AVX2 system, might change a bit with Avx512)

Benchmarks

Method	Toolchain	Length	MatchAtStart	Mean	Error	Ratio
IndexOfAny	main	1	False	2.228 ns	0.0182 ns	1.00
IndexOfAny	pr	1	False	1.803 ns	0.0016 ns	0.81
IndexOfAnyExcept	main	1	False	2.291 ns	0.0315 ns	1.00
IndexOfAnyExcept	pr	1	False	1.462 ns	0.0187 ns	0.64
IndexOfAny	main	1	True	1.645 ns	0.0012 ns	1.00
IndexOfAny	pr	1	True	1.799 ns	0.0011 ns	1.09
IndexOfAnyExcept	main	1	True	1.819 ns	0.0016 ns	1.00
IndexOfAnyExcept	pr	1	True	1.528 ns	0.0012 ns	0.84
IndexOfAny	main	7	False	6.917 ns	0.0032 ns	1.00
IndexOfAny	pr	7	False	4.236 ns	0.0039 ns	0.61
IndexOfAnyExcept	main	7	False	6.911 ns	0.0030 ns	1.00
IndexOfAnyExcept	pr	7	False	4.022 ns	0.0032 ns	0.58
IndexOfAny	main	7	True	1.646 ns	0.0012 ns	1.00
IndexOfAny	pr	7	True	1.800 ns	0.0015 ns	1.09
IndexOfAnyExcept	main	7	True	1.820 ns	0.0011 ns	1.00
IndexOfAnyExcept	pr	7	True	1.528 ns	0.0013 ns	0.84
IndexOfAny	main	100	False	4.618 ns	0.0109 ns	1.00
IndexOfAny	pr	100	False	5.043 ns	0.0241 ns	1.09
IndexOfAnyExcept	main	100	False	4.826 ns	0.0074 ns	1.00
IndexOfAnyExcept	pr	100	False	5.205 ns	0.0255 ns	1.08
IndexOfAny	main	100	True	2.869 ns	0.0207 ns	1.00
IndexOfAny	pr	100	True	2.281 ns	0.0067 ns	0.80
IndexOfAnyExcept	main	100	True	2.460 ns	0.0051 ns	1.00
IndexOfAnyExcept	pr	100	True	2.085 ns	0.0019 ns	0.85
IndexOfAny	main	1000	False	32.786 ns	0.0479 ns	1.00
IndexOfAny	pr	1000	False	32.251 ns	0.0310 ns	0.98
IndexOfAnyExcept	main	1000	False	34.281 ns	0.0395 ns	1.00
IndexOfAnyExcept	pr	1000	False	31.651 ns	0.1198 ns	0.92
IndexOfAny	main	1000	True	2.814 ns	0.0018 ns	1.00
IndexOfAny	pr	1000	True	2.274 ns	0.0059 ns	0.81
IndexOfAnyExcept	main	1000	True	2.289 ns	0.0525 ns	1.00
IndexOfAnyExcept	pr	1000	True	2.082 ns	0.0011 ns	0.92
IndexOfAny	main	10000	False	275.551 ns	0.0975 ns	1.00
IndexOfAny	pr	10000	False	267.742 ns	0.2087 ns	0.97
IndexOfAnyExcept	main	10000	False	283.259 ns	0.1677 ns	1.00
IndexOfAnyExcept	pr	10000	False	292.667 ns	0.4272 ns	1.03
IndexOfAny	main	10000	True	2.817 ns	0.0013 ns	1.00
IndexOfAny	pr	10000	True	2.267 ns	0.0071 ns	0.80
IndexOfAnyExcept	main	10000	True	2.478 ns	0.0096 ns	1.00
IndexOfAnyExcept	pr	10000	True	2.086 ns	0.0020 ns	0.84

This implementation (specific to X86's Packed variants) would take very little code to add, but it's also quite close to the ASCII impl in perf.

[steveharter]

@MihaZupan is this still planned for v9?

[stephentoub]

I opened #105735 to revert #101899.