[Perf] Changes at 1/25/2022 9:10:05 PM

[performanceautofiler]

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	4d16d0934259b2b1b056823986bb0ad08578c5c0
Compare	82ff4f89e5997ade30ac1a9de8f343c321dd10a9
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Count - Duration of single invocation	63.15 μs	158.33 μs	2.51	0.02	True
Count - Duration of single invocation	63.17 μs	156.70 μs	2.48	0.07	True
Count - Duration of single invocation	100.51 μs	193.19 μs	1.92	0.00	True
Count - Duration of single invocation	162.62 μs	261.46 μs	1.61	0.04	True
Count - Duration of single invocation	208.96 μs	306.95 μs	1.47	0.01	True
Count - Duration of single invocation	90.87 μs	175.95 μs	1.94	0.03	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: Compiled)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 158.32569140625 > 66.32821333165323.
IsChangePoint: Marked as a change because one of 1/25/2022 5:52:36 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsRegressionStdDev: Marked as regression because -180.02721611924915 (T) = (0 -159639.8898316333) / Math.Sqrt((23295.467043067907 / (13)) + (9111673.397389187 / (32))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (32) - 2, .025) and -1.523004173357953 = (63273.73197292936 - 159639.8898316333) / 63273.73197292936 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 156.69533106435642 > 66.35020388104839.
IsChangePoint: Marked as a change because one of 1/25/2022 5:52:36 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsRegressionStdDev: Marked as regression because -92.88673664090769 (T) = (0 -150947.3073534807) / Math.Sqrt((17588.9631269246 / (13)) + (28465065.763339143 / (32))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (32) - 2, .025) and -1.3855941172095874 = (63274.51357485853 - 150947.3073534807) / 63274.51357485853 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 193.19492002206286 > 105.63983411798321.
IsChangePoint: Marked as a change because one of 1/25/2022 5:52:36 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsRegressionStdDev: Marked as regression because -255.26249100827877 (T) = (0 -193721.44177344316) / Math.Sqrt((1479261.8967892947 / (13)) + (555016.1673884661 / (32))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (32) - 2, .025) and -0.9126403613681782 = (101284.82368471377 - 193721.44177344316) / 101284.82368471377 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 261.4589593596059 > 170.94476514175258.
IsChangePoint: Marked as a change because one of 1/25/2022 5:52:36 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsRegressionStdDev: Marked as regression because -106.4360890025573 (T) = (0 -261319.00088245538) / Math.Sqrt((7982267.671461924 / (13)) + (6981050.610757933 / (32))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (32) - 2, .025) and -0.5912357177412606 = (164223.9411602666 - 261319.00088245538) / 164223.9411602666 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 306.9529166666667 > 219.75764273119987.
IsChangePoint: Marked as a change because one of 1/25/2022 5:52:36 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsRegressionStdDev: Marked as regression because -137.07791441616314 (T) = (0 -307151.47487401805) / Math.Sqrt((4997706.498953922 / (13)) + (3861514.9971027854 / (32))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (32) - 2, .025) and -0.4645189302714733 = (209728.57948451533 - 307151.47487401805) / 209728.57948451533 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 175.94525983146067 > 99.90353115530304.
IsChangePoint: Marked as a change because one of 1/25/2022 5:52:36 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsRegressionStdDev: Marked as regression because -95.51776731457086 (T) = (0 -175591.9957599587) / Math.Sqrt((8283791.171685471 / (13)) + (3317048.622719166 / (32))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (32) - 2, .025) and -0.8804815282051011 = (93376.08114000424 - 175591.9957599587) / 93376.08114000424 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	4d16d0934259b2b1b056823986bb0ad08578c5c0
Compare	82ff4f89e5997ade30ac1a9de8f343c321dd10a9
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Count - Duration of single invocation	3.22 ms	5.00 ms	1.55	0.00	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Twain", Options: NonBacktracking)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.000114577259476 > 3.416715201255981.
IsChangePoint: Marked as a change because one of 1/25/2022 5:52:36 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsRegressionStdDev: Marked as regression because -221.896102895238 (T) = (0 -4978287.906884609) / Math.Sqrt((716405745.323191 / (13)) + (154396783.72417286 / (32))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (32) - 2, .025) and -0.5268725711788088 = (3260447.532331506 - 4978287.906884609) / 3260447.532331506 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository