Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Windows/x64: 23 Regressions on 10/10/2023 7:44:23 AM #23287

Closed
performanceautofiler bot opened this issue Oct 17, 2023 · 2 comments
Closed

[Perf] Windows/x64: 23 Regressions on 10/10/2023 7:44:23 AM #23287

performanceautofiler bot opened this issue Oct 17, 2023 · 2 comments
Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-coreclr untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Oct 17, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d3a782e3bb6ad0c1cb590b41c3f03e733f7d0d61
Compare 715293eeeddf76593679fba7ab8cfd451980225b
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
333.84 ns 397.51 ns 1.19 0.05 False
334.16 ns 397.22 ns 1.19 0.04 False
334.08 ns 396.96 ns 1.19 0.05 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreNonSpace, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 397.50647994202154 > 350.8214789702524.
IsChangePoint: Marked as a change because one of 7/23/2023 4:13:29 AM, 8/14/2023 10:44:26 PM, 8/27/2023 2:42:29 PM, 9/19/2023 8:08:41 AM, 10/9/2023 8:09:24 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -541.6462419762796 (T) = (0 -397.14839198810546) / Math.Sqrt((0.03434102539005903 / (14)) + (0.1773053449286112 / (16))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (16) - 2, .025) and -0.18858880117623114 = (334.1343882721141 - 397.14839198810546) / 334.1343882721141 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, None, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 397.2185831499582 > 350.8857379593751.
IsChangePoint: Marked as a change because one of 7/23/2023 4:13:29 AM, 8/14/2023 10:44:26 PM, 8/27/2023 2:42:29 PM, 9/19/2023 8:08:41 AM, 10/9/2023 8:09:24 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -489.39326403272764 (T) = (0 -396.97976374482215) / Math.Sqrt((0.06370073701159094 / (14)) + (0.19051381186001842 / (16))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (16) - 2, .025) and -0.18785901451521414 = (334.1977110868132 - 396.97976374482215) / 334.1977110868132 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, None, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 396.9550094936439 > 351.05437173670015.
IsChangePoint: Marked as a change because one of 7/23/2023 4:13:29 AM, 8/14/2023 10:44:26 PM, 8/27/2023 2:42:29 PM, 9/19/2023 8:08:41 AM, 10/9/2023 8:09:24 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -395.02576986883645 (T) = (0 -397.1566900977657) / Math.Sqrt((0.06423492975119141 / (14)) + (0.3312878610676318 / (16))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (16) - 2, .025) and -0.18791173619773943 = (334.3318177569172 - 397.1566900977657) / 334.3318177569172 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d3a782e3bb6ad0c1cb590b41c3f03e733f7d0d61
Compare 715293eeeddf76593679fba7ab8cfd451980225b
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
15.03 ns 16.67 ns 1.11 0.05 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

System.IO.Tests.Perf_Path.GetExtension

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.673752449691815 > 15.78408303235025.
IsChangePoint: Marked as a change because one of 7/14/2023 2:26:40 PM, 10/9/2023 8:09:24 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -25.59701437445162 (T) = (0 -16.63588610476346) / Math.Sqrt((0.006024592865638347 / (14)) + (0.058003730345461114 / (16))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (16) - 2, .025) and -0.10863158096803588 = (15.005784058791942 - 16.63588610476346) / 15.005784058791942 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d3a782e3bb6ad0c1cb590b41c3f03e733f7d0d61
Compare 715293eeeddf76593679fba7ab8cfd451980225b
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
13.25 μs 17.31 μs 1.31 0.11 False
12.89 μs 17.21 μs 1.34 0.12 False
14.16 μs 17.60 μs 1.24 0.06 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Collections.ContainsTrue<Int32>.List(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.310901624215578 > 13.842786757691204.
IsChangePoint: Marked as a change because one of 7/1/2023 1:53:48 AM, 7/17/2023 6:26:12 PM, 8/28/2023 1:23:07 PM, 9/1/2023 9:23:50 AM, 9/19/2023 8:08:41 AM, 10/9/2023 8:09:24 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -109.57706107264259 (T) = (0 -17270.290833119605) / Math.Sqrt((10731.78430711505 / (14)) + (9628.105239749102 / (16))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (16) - 2, .025) and -0.3066767236837228 = (13216.957584146749 - 17270.290833119605) / 13216.957584146749 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<Int32>.ICollection(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.214496145374447 > 13.58557260192467.
IsChangePoint: Marked as a change because one of 6/29/2023 1:56:51 PM, 7/14/2023 2:57:02 AM, 9/1/2023 9:23:50 AM, 9/19/2023 8:08:41 AM, 10/9/2023 8:09:24 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -139.77694455037582 (T) = (0 -17363.889397855422) / Math.Sqrt((6185.246982045102 / (14)) + (8936.239576782158 / (16))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (16) - 2, .025) and -0.3415607820924905 = (12943.050832756315 - 17363.889397855422) / 12943.050832756315 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<Int32>.ImmutableArray(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.604890078917695 > 14.864255859375.
IsChangePoint: Marked as a change because one of 6/26/2023 10:42:28 PM, 7/14/2023 2:26:40 PM, 7/23/2023 4:13:29 AM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -86.34771466049271 (T) = (0 -17718.02872232679) / Math.Sqrt((9240.85475844212 / (19)) + (14222.216880865666 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.258763416856169 = (14075.741704170705 - 17718.02872232679) / 14075.741704170705 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d3a782e3bb6ad0c1cb590b41c3f03e733f7d0d61
Compare 715293eeeddf76593679fba7ab8cfd451980225b
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
42.45 μs 56.33 μs 1.33 0.07 False
23.21 μs 41.92 μs 1.81 0.10 False
32.40 μs 48.68 μs 1.50 0.37 False
22.90 μs 31.33 μs 1.37 0.11 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: None)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 56.33410332994408 > 44.246818098192776.
IsChangePoint: Marked as a change because one of 6/22/2023 7:55:14 AM, 7/14/2023 8:09:31 AM, 8/10/2023 8:47:51 AM, 9/27/2023 11:11:59 PM, 10/9/2023 8:09:24 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -228.7936034376518 (T) = (0 -56114.19142734675) / Math.Sqrt((29985.331833617583 / (14)) + (25336.01237679139 / (16))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (16) - 2, .025) and -0.3313069926748838 = (42149.70080987947 - 56114.19142734675) / 42149.70080987947 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: None)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.92154602323504 > 24.376041240999815.
IsChangePoint: Marked as a change because one of 6/23/2023 6:04:39 AM, 7/17/2023 6:26:12 PM, 8/10/2023 8:47:51 AM, 9/27/2023 11:11:59 PM, 10/9/2023 8:09:24 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -13.004051841498825 (T) = (0 -41904.91586964944) / Math.Sqrt((24922428.761207674 / (14)) + (295.29123542096215 / (16))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (16) - 2, .025) and -0.7066145562915888 = (24554.411372599152 - 41904.91586964944) / 24554.411372599152 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: None)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.676973001038434 > 32.9863863054991.
IsChangePoint: Marked as a change because one of 7/12/2023 2:12:47 AM, 9/29/2023 6:10:44 PM, 10/8/2023 11:25:37 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -5.192937492767034 (T) = (0 -46972.83468676948) / Math.Sqrt((48550245.63127606 / (12)) + (31546361.488506336 / (18))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (18) - 2, .025) and -0.3627844723964241 = (34468.278468251745 - 46972.83468676948) / 34468.278468251745 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: None)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.33385521042084 > 24.067408930535187.
IsChangePoint: Marked as a change because one of 6/22/2023 7:55:14 AM, 7/17/2023 6:26:12 PM, 8/10/2023 8:47:51 AM, 9/27/2023 11:11:59 PM, 10/9/2023 8:09:24 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -6.0056590966253784 (T) = (0 -29756.065097747334) / Math.Sqrt((5034477.073377246 / (14)) + (11531225.969430635 / (16))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (16) - 2, .025) and -0.26546628802954697 = (23513.913708503762 - 29756.065097747334) / 23513.913708503762 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d3a782e3bb6ad0c1cb590b41c3f03e733f7d0d61
Compare 715293eeeddf76593679fba7ab8cfd451980225b
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Byte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.74 ns 4.92 ns 1.80 0.12 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Byte*'

Payloads

Baseline
Compare

System.Tests.Perf_Byte.ToString(value: 255)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.918311416144362 > 2.8776614966726304.
IsChangePoint: Marked as a change because one of 6/26/2023 10:42:28 PM, 7/12/2023 2:12:47 AM, 7/23/2023 4:13:29 AM, 10/9/2023 8:09:24 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -5.058392860694486 (T) = (0 -3.8087021484005468) / Math.Sqrt((1.2033913326730199E-05 / (14)) + (0.7130261197003657 / (16))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (16) - 2, .025) and -0.38960413729862875 = (2.7408540649602635 - 3.8087021484005468) / 2.7408540649602635 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d3a782e3bb6ad0c1cb590b41c3f03e733f7d0d61
Compare 715293eeeddf76593679fba7ab8cfd451980225b
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Tests.SearchValuesByteTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
7.72 ns 9.42 ns 1.22 0.15 True
7.26 ns 9.08 ns 1.25 0.03 True
7.67 ns 9.44 ns 1.23 0.06 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.SearchValuesByteTests*'

Payloads

Baseline
Compare

System.Buffers.Tests.SearchValuesByteTests.IndexOfAny(Values: "abcdefABCDEF0123456789")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.42215063962752 > 8.059662098176476.
IsChangePoint: Marked as a change because one of 10/9/2023 8:09:24 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -12.46955237865788 (T) = (0 -8.964677980065522) / Math.Sqrt((0.0068711309527646905 / (14)) + (0.16163738279423648 / (16))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (16) - 2, .025) and -0.16708202572145708 = (7.6812749939523846 - 8.964677980065522) / 7.6812749939523846 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.SearchValuesByteTests.IndexOfAnyExcept(Values: "abcdefABCDEF0123456789")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.083495568292552 > 7.642417663452558.
IsChangePoint: Marked as a change because one of 10/9/2023 8:09:24 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -57.30100871851927 (T) = (0 -9.087827700030042) / Math.Sqrt((0.0006954368878014451 / (14)) + (0.01497497134672144 / (16))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (16) - 2, .025) and -0.2468044790545231 = (7.288895614909504 - 9.087827700030042) / 7.288895614909504 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.SearchValuesByteTests.ContainsAny(Values: "abcdefABCDEF0123456789")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.443285731641634 > 8.037869038118231.
IsChangePoint: Marked as a change because one of 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -24.687888406918255 (T) = (0 -9.218643920516852) / Math.Sqrt((0.015399279275363078 / (19)) + (0.03389384688276221 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.20057594840967388 = (7.678517908615611 - 9.218643920516852) / 7.678517908615611 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler performanceautofiler bot added arch-x64 branch-refs/heads/main kind-micro os-windows perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-coreclr untriaged labels Oct 17, 2023
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 17, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d3a782e3bb6ad0c1cb590b41c3f03e733f7d0d61
Compare 715293eeeddf76593679fba7ab8cfd451980225b
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
69.61 μs 83.58 μs 1.20 0.04 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

PerfLabTests.LowLevelPerf.ObjectStringIsString

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 83.58270721925135 > 73.115086875.
IsChangePoint: Marked as a change because one of 6/26/2023 10:42:28 PM, 7/14/2023 2:26:40 PM, 7/23/2023 4:13:29 AM, 8/10/2023 8:47:51 AM, 9/19/2023 8:08:41 AM, 10/9/2023 8:09:24 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -1619.6478950781348 (T) = (0 -83552.09573570818) / Math.Sqrt((473.61637430790074 / (14)) + (639.8961360724392 / (16))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (16) - 2, .025) and -0.19984041813523984 = (69636.00698296414 - 83552.09573570818) / 69636.00698296414 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d3a782e3bb6ad0c1cb590b41c3f03e733f7d0d61
Compare 715293eeeddf76593679fba7ab8cfd451980225b
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
407.24 ns 434.04 ns 1.07 0.06 False
35.56 ns 39.86 ns 1.12 0.07 False
37.61 ns 40.35 ns 1.07 0.08 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 434.04092711442973 > 430.2751623470009.
IsChangePoint: Marked as a change because one of 7/13/2023 8:36:41 PM, 10/9/2023 8:09:24 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -11.788014556840874 (T) = (0 -435.8243103535664) / Math.Sqrt((26.87270787617013 / (14)) + (40.59696689099194 / (16))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (16) - 2, .025) and -0.060558469801953126 = (410.9385034046746 - 435.8243103535664) / 410.9385034046746 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.ToArray(input: ICollection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 39.859729465193816 > 37.80683249408236.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 10/10/2023 4:37:26 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -8.11361665327233 (T) = (0 -39.443024919080756) / Math.Sqrt((0.6875187704602364 / (16)) + (2.750821263345837 / (14))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (14) - 2, .025) and -0.1119268835007019 = (35.47267855859504 - 39.443024919080756) / 35.47267855859504 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.ToList(input: ICollection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 40.346377713769584 > 39.47803064095586.
IsChangePoint: Marked as a change because one of 8/29/2023 4:22:52 PM, 9/2/2023 3:43:46 AM, 9/12/2023 2:43:35 PM, 9/26/2023 11:02:05 PM, 10/9/2023 8:09:24 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -9.23843721837649 (T) = (0 -39.889235839861485) / Math.Sqrt((0.30466102107902554 / (14)) + (0.697797508325717 / (16))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (16) - 2, .025) and -0.06294413231684129 = (37.52712360612697 - 39.889235839861485) / 37.52712360612697 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d3a782e3bb6ad0c1cb590b41c3f03e733f7d0d61
Compare 715293eeeddf76593679fba7ab8cfd451980225b
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
16.67 ns 18.47 ns 1.11 0.09 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark*'

Payloads

Baseline
Compare

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.GetService_1Injected

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.46633207548504 > 17.31856311771301.
IsChangePoint: Marked as a change because one of 9/2/2023 3:43:46 AM, 10/9/2023 8:09:24 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -8.786854629533307 (T) = (0 -18.480158166067653) / Math.Sqrt((0.04333480917568641 / (14)) + (0.7230695626376193 / (16))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (16) - 2, .025) and -0.11667281396349684 = (16.549304267983867 - 18.480158166067653) / 16.549304267983867 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d3a782e3bb6ad0c1cb590b41c3f03e733f7d0d61
Compare 715293eeeddf76593679fba7ab8cfd451980225b
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
10.45 ns 12.13 ns 1.16 0.07 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*'

Payloads

Baseline
Compare

System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Span(address: [16, 32, 48, 64, 80, ...])

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.131488533356135 > 10.932896529930293.
IsChangePoint: Marked as a change because one of 7/1/2023 6:36:20 PM, 8/8/2023 9:37:01 AM, 9/1/2023 9:23:50 AM, 9/2/2023 3:43:46 AM, 10/9/2023 8:09:24 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -35.712195814286034 (T) = (0 -12.23059822682987) / Math.Sqrt((0.02560006239971707 / (14)) + (0.015498438233210744 / (16))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (16) - 2, .025) and -0.1826347652123598 = (10.341821994919693 - 12.23059822682987) / 10.341821994919693 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d3a782e3bb6ad0c1cb590b41c3f03e733f7d0d61
Compare 715293eeeddf76593679fba7ab8cfd451980225b
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
13.25 ns 14.78 ns 1.12 0.16 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

System.Collections.Tests.Perf_BitArray.BitArrayCopyToIntArray(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.782944634439328 > 12.800133929746732.
IsChangePoint: Marked as a change because one of 6/14/2023 10:30:12 PM, 10/9/2023 8:09:24 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -13.536476012133182 (T) = (0 -14.941701790670443) / Math.Sqrt((0.12568392440889858 / (14)) + (0.45639948413430076 / (16))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (16) - 2, .025) and -0.21277183222129945 = (12.32029091845198 - 14.941701790670443) / 12.32029091845198 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d3a782e3bb6ad0c1cb590b41c3f03e733f7d0d61
Compare 715293eeeddf76593679fba7ab8cfd451980225b
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
45.08 ns 50.16 ns 1.11 0.08 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;*'

Payloads

Baseline
Compare

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 50.164628789533246 > 47.32166204473928.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 6/30/2023 8:34:57 AM, 8/14/2023 10:44:26 PM, 8/28/2023 1:23:07 PM, 9/19/2023 8:08:41 AM, 10/9/2023 8:09:24 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM.
IsRegressionStdDev: Marked as regression because -10.738981570406734 (T) = (0 -49.28618725298321) / Math.Sqrt((0.7532751228466782 / (14)) + (1.3357879801555361 / (16))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (16) - 2, .025) and -0.08782534064338553 = (45.30707771877542 - 49.28618725298321) / 45.30707771877542 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@EgorBo
Copy link
Member

EgorBo commented Oct 19, 2023

Either Static PGO or dotnet/runtime#93214

@EgorBo EgorBo closed this as completed Oct 19, 2023
@DrewScoggins DrewScoggins mentioned this issue Sep 11, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-coreclr untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@EgorBo