Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 37 Regressions on 10/2/2022 4:42:49 PM #8239

Open
performanceautofiler bot opened this issue Oct 6, 2022 · 2 comments
Open

[Perf] Linux/x64: 37 Regressions on 10/2/2022 4:42:49 PM #8239

performanceautofiler bot opened this issue Oct 6, 2022 · 2 comments
Labels
CoreClr refs/heads/main Regression RunKind=micro ubuntu 18.04 untriaged x64

Comments

@performanceautofiler
Copy link

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 2.52 ms 2.68 ms 1.06 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: NonBacktracking)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 2.681041714285714 > 2.6410438062626267.
IsChangePoint: Marked as a change because one of 8/17/2022 1:34:50 AM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -12.768222873666161 (T) = (0 -2687466.386280502) / Math.Sqrt((5143613702.273555 / (42)) + (20663869.287225593 / (24))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (24) - 2, .025) and -0.055700482353756825 = (2545671.268699822 - 2687466.386280502) / 2545671.268699822 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 2.51 μs 2.95 μs 1.18 0.34 False
Dictionary - Duration of single invocation 2.25 μs 2.48 μs 1.10 0.28 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IterateForEach<String>.Queue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 2.948223095415349 > 2.636845496857219.
IsChangePoint: Marked as a change because one of 9/1/2022 10:02:47 PM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -12.782968493147893 (T) = (0 -2949.8437259833654) / Math.Sqrt((32828.06630798105 / (42)) + (1269.8355918578052 / (23))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (23) - 2, .025) and -0.143324737825566 = (2580.0576410106637 - 2949.8437259833654) / 2580.0576410106637 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.Dictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 2.4773692422324207 > 2.3653366508202853.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 9/2/2022 1:41:42 PM, 9/13/2022 12:23:36 PM, 9/30/2022 2:28:26 AM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -11.58641202523526 (T) = (0 -2550.1955163420525) / Math.Sqrt((2894.1167280611544 / (30)) + (15200.291183224612 / (34))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (34) - 2, .025) and -0.11847194538398294 = (2280.0710620117916 - 2550.1955163420525) / 2280.0710620117916 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FixedTimeEquals_256Bit_VersusZero - Duration of single invocation 89.37 ns 102.07 ns 1.14 0.04 False
FixedTimeEquals_256Bit_Equal - Duration of single invocation 89.39 ns 101.80 ns 1.14 0.05 False
FixedTimeEquals_256Bit_FirstBitDifferent - Duration of single invocation 89.27 ns 101.95 ns 1.14 0.07 False
FixedTimeEquals_256Bit_LastBitDifferent - Duration of single invocation 89.39 ns 101.91 ns 1.14 0.06 False
FixedTimeEquals_256Bit_AllBitsDifferent - Duration of single invocation 89.39 ns 102.22 ns 1.14 0.04 False
FixedTimeEquals_256Bit_SameReference - Duration of single invocation 89.24 ns 102.62 ns 1.15 0.06 False
FixedTimeEquals_256Bit_CascadingErrors - Duration of single invocation 89.39 ns 101.81 ns 1.14 0.06 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_VersusZero

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 102.06522401816012 > 93.80414491929413.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -168.16396555388832 (T) = (0 -102.00533913997702) / Math.Sqrt((0.20714264372095556 / (42)) + (0.0167992817342732 / (24))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (24) - 2, .025) and -0.14118739530799668 = (89.38526622303488 - 102.00533913997702) / 89.38526622303488 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_Equal

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 101.79978824310189 > 93.87687488792025.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -40.11338430730051 (T) = (0 -101.8526034804667) / Math.Sqrt((3.8520852237097656 / (42)) + (0.015419071932968641 / (24))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (24) - 2, .025) and -0.1359628939690853 = (89.66191063212543 - 101.8526034804667) / 89.66191063212543 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_FirstBitDifferent

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 101.949916279839 > 93.82493706468742.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -77.21774953615183 (T) = (0 -102.0228441475986) / Math.Sqrt((1.020465737908509 / (42)) + (0.0463595102278814 / (23))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (23) - 2, .025) and -0.13995482229567893 = (89.49726967437324 - 102.0228441475986) / 89.49726967437324 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_LastBitDifferent

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 101.90982352281982 > 93.88650795339994.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -48.15351131670541 (T) = (0 -101.997583601123) / Math.Sqrt((2.7046747956186574 / (42)) + (0.014325866982145692 / (23))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (23) - 2, .025) and -0.13685699760647635 = (89.71892139105213 - 101.997583601123) / 89.71892139105213 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_AllBitsDifferent

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 102.22453710190373 > 93.86257297344697.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -34.95855888790131 (T) = (0 -101.90202643941676) / Math.Sqrt((5.141818411847929 / (43)) + (0.05617643687558926 / (23))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (23) - 2, .025) and -0.13615127257541948 = (89.69054464765593 - 101.90202643941676) / 89.69054464765593 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_SameReference

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 102.61696071121274 > 93.71013811723479.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -48.08834909899224 (T) = (0 -102.71597991750437) / Math.Sqrt((2.489022601782945 / (40)) + (0.3012033550558351 / (23))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (23) - 2, .025) and -0.14743049640990769 = (89.51825861251133 - 102.71597991750437) / 89.51825861251133 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_CascadingErrors

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 101.81383297646913 > 93.85075927902294.
IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -24.793020502639717 (T) = (0 -101.86981590223299) / Math.Sqrt((8.508462644160605 / (39)) + (0.011833203976618805 / (24))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (24) - 2, .025) and -0.12842182421483572 = (90.27636094606267 - 101.86981590223299) / 90.27636094606267 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadOnlySpan - Duration of single invocation 175.29 ns 229.58 ns 1.31 0.03 False
Span - Duration of single invocation 181.38 ns 233.42 ns 1.29 0.03 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IterateFor<String>.ReadOnlySpan(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 229.57599683119093 > 184.1444162791468.
IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 8/4/2022 6:25:19 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -13.63183171316812 (T) = (0 -229.49695275782577) / Math.Sqrt((432.183431100789 / (40)) + (0.01409486487813824 / (23))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (23) - 2, .025) and -0.24262375586138493 = (184.68740169765934 - 229.49695275782577) / 184.68740169765934 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateFor&lt;String&gt;.Span(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 233.4200016300474 > 191.4093273194621.
IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 8/4/2022 6:25:19 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -13.34198052039143 (T) = (0 -233.83778211719618) / Math.Sqrt((426.2067684862456 / (42)) + (0.07319659956389271 / (23))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (23) - 2, .025) and -0.22217340015212217 = (191.3294644508633 - 233.83778211719618) / 191.3294644508633 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Collections.IterateForEachNonGeneric<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 3.48 μs 3.74 μs 1.08 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEachNonGeneric&lt;String&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IterateForEachNonGeneric<String>.Queue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 3.7398652122264813 > 3.650750636743593.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/17/2022 1:34:50 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -12.520967028654143 (T) = (0 -3741.399347561804) / Math.Sqrt((14179.865887537519 / (43)) + (5.406798717800655 / (23))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (23) - 2, .025) and -0.06472908999425993 = (3513.9448923875784 - 3741.399347561804) / 3513.9448923875784 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GenericClassGenericStaticMethod - Duration of single invocation 108.73 μs 152.20 μs 1.40 0.03 False
EmptyStaticFunction5Arg - Duration of single invocation 1.03 ms 1.24 ms 1.21 0.01 False
GenericClassGenericInstanceMethod - Duration of single invocation 108.78 μs 130.57 μs 1.20 0.03 False
GenericGenericMethod - Duration of single invocation 108.88 μs 152.45 μs 1.40 0.04 False
StructWithInterfaceInterfaceMethod - Duration of single invocation 108.80 μs 130.53 μs 1.20 0.04 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

PerfLabTests.LowLevelPerf.GenericClassGenericStaticMethod

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 152.1973545582707 > 114.22614886067706.
IsChangePoint: Marked as a change because one of 8/4/2022 6:25:19 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -2093.984283182512 (T) = (0 -152259.19853015535) / Math.Sqrt((8492.12374934872 / (41)) + (4913.85234418511 / (22))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (22) - 2, .025) and -0.39927278715200754 = (108813.09200620861 - 152259.19853015535) / 108813.09200620861 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.EmptyStaticFunction5Arg

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.239955132051282 > 1.0737858039453125.
IsChangePoint: Marked as a change because one of 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -114.05025829189512 (T) = (0 -1240437.4480591596) / Math.Sqrt((126358691.49288963 / (43)) + (9235196.381101305 / (23))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (23) - 2, .025) and -0.20197458047422978 = (1031999.7346114878 - 1240437.4480591596) / 1031999.7346114878 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.GenericClassGenericInstanceMethod

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 130.56582291666663 > 114.2213323079427.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/17/2022 6:09:21 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -864.8857174819942 (T) = (0 -130549.04871697913) / Math.Sqrt((8780.272793508519 / (25)) + (10974.300600371562 / (39))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (39) - 2, .025) and -0.19994583425800883 = (108795.78476782216 - 130549.04871697913) / 108795.78476782216 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.GenericGenericMethod

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 152.4473213996764 > 114.23529543945313.
IsChangePoint: Marked as a change because one of 8/4/2022 6:25:19 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -460.39318335674164 (T) = (0 -152387.3945753766) / Math.Sqrt((46983.02064574239 / (42)) + (179789.04878820028 / (23))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (23) - 2, .025) and -0.3997534739033202 = (108867.30943444821 - 152387.3945753766) / 108867.30943444821 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.StructWithInterfaceInterfaceMethod

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 130.52645694444445 > 114.25878535373266.
IsChangePoint: Marked as a change because one of 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -7.501443818663109 (T) = (0 -130522.14636095078) / Math.Sqrt((104853686.90676622 / (26)) + (8141.51606040868 / (40))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (40) - 2, .025) and -0.13047854892521996 = (115457.42861290213 - 130522.14636095078) / 115457.42861290213 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetUInt32 - Duration of single invocation 451.34 ns 498.75 ns 1.11 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Get.GetUInt32

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 498.7513437657641 > 473.83853933758365.
IsChangePoint: Marked as a change because one of 8/30/2022 6:17:03 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -136.2009407560189 (T) = (0 -499.3483521777175) / Math.Sqrt((0.383180687491248 / (40)) + (2.5798670613084145 / (23))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (23) - 2, .025) and -0.10518177483695741 = (451.82463513876183 - 499.3483521777175) / 451.82463513876183 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableSortedDictionary - Duration of single invocation 25.06 μs 26.57 μs 1.06 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsKeyTrue<Int32, Int32>.ImmutableSortedDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 26.57017710813492 > 26.373182925660963.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 9/13/2022 12:23:36 PM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -29.566751987271978 (T) = (0 -26582.203199255186) / Math.Sqrt((66920.44134873642 / (43)) + (13666.761247410579 / (23))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (23) - 2, .025) and -0.05438527532225898 = (25211.09107022638 - 26582.203199255186) / 25211.09107022638 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitArrayCopyToByteArray - Duration of single invocation 95.48 ns 122.98 ns 1.29 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.Tests.Perf_BitArray.BitArrayCopyToByteArray(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 122.98424693842964 > 100.30431121211656.
IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 8/15/2022 9:13:40 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -94.8249644954591 (T) = (0 -123.6646693615966) / Math.Sqrt((1.071821193590283 / (40)) + (1.3357792774907706 / (23))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (23) - 2, .025) and -0.2876457327534532 = (96.03935788856825 - 123.6646693615966) / 96.03935788856825 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in Benchstone.BenchI.Fib

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 102.42 μs 111.22 μs 1.09 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.Fib*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Benchstone.BenchI.Fib.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 111.22295719562648 > 107.50533815870101.
IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 8/4/2022 6:25:19 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -48.150868149087145 (T) = (0 -111241.12686684255) / Math.Sqrt((1342233.3117296437 / (44)) + (11520.545451244878 / (24))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (24) - 2, .025) and -0.082477533198163 = (102765.29854451794 - 111241.12686684255) / 102765.29854451794 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf8 - Duration of single invocation 68.38 ns 75.46 ns 1.10 0.03 False
EncodeUtf8 - Duration of single invocation 54.56 ns 66.76 ns 1.22 0.03 False
EncodeUtf8 - Duration of single invocation 24.55 ns 27.97 ns 1.14 0.03 False
EncodeUtf8 - Duration of single invocation 34.65 ns 37.15 ns 1.07 0.01 False
EncodeUtf8 - Duration of single invocation 85.64 ns 98.58 ns 1.15 0.03 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,&Hello+<World>!,512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.45942750994763 > 71.77986331434076.
IsChangePoint: Marked as a change because one of 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -108.13611451906125 (T) = (0 -75.28673793634144) / Math.Sqrt((0.07377968650592988 / (40)) + (0.04965537742487352 / (23))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (23) - 2, .025) and -0.09996470669242388 = (68.44468506878502 - 75.28673793634144) / 68.44468506878502 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,&lorem ipsum=dolor sit amet,16)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 66.75644551897595 > 57.35513104412514.
IsChangePoint: Marked as a change because one of 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -416.1543838216037 (T) = (0 -66.76368613722173) / Math.Sqrt((0.012248819134870173 / (39)) + (0.012953130269026747 / (24))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (24) - 2, .025) and -0.22269272007061594 = (54.603814221913275 - 66.76368613722173) / 54.603814221913275 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,hello "there",16)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 27.973661927720535 > 25.80364339169584.
IsChangePoint: Marked as a change because one of 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -94.56335710104665 (T) = (0 -27.942096965474114) / Math.Sqrt((0.04545010984085537 / (42)) + (0.0023136043867208345 / (23))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (23) - 2, .025) and -0.1317183664356057 = (24.689973931835105 - 27.942096965474114) / 24.689973931835105 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,�2020,16)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 37.147362150815205 > 36.578214068334695.
IsChangePoint: Marked as a change because one of 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -45.15416548991732 (T) = (0 -37.04900558243237) / Math.Sqrt((0.09529286188483425 / (43)) + (0.0023518483481626967 / (23))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (23) - 2, .025) and -0.0623413916736609 = (34.87485837680078 - 37.04900558243237) / 34.87485837680078 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,&lorem ipsum=dolor sit amet,512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 98.57628933547694 > 90.02366994362218.
IsChangePoint: Marked as a change because one of 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -164.99851867157003 (T) = (0 -98.58589619275148) / Math.Sqrt((0.11714357872039023 / (41)) + (0.07134448537168243 / (24))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (24) - 2, .025) and -0.1465117121525273 = (85.98769218646761 - 98.58589619275148) / 85.98769218646761 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 1.24 μs 1.36 μs 1.10 0.08 False
Stack - Duration of single invocation 1.36 μs 1.59 μs 1.17 0.29 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IterateForEach<Int32>.Queue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.3625641800197394 > 1.299166101073397.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -26.11117314868406 (T) = (0 -1451.5898923242964) / Math.Sqrt((134.56662261760798 / (43)) + (1437.4977814595625 / (24))) is less than -1.997137908391408 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (24) - 2, .025) and -0.1665939724542022 = (1244.2974390400277 - 1451.5898923242964) / 1244.2974390400277 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;Int32&gt;.Stack(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.592150411402079 > 1.4304184867127765.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -13.963652704811208 (T) = (0 -1600.167691408334) / Math.Sqrt((5507.692217151551 / (44)) + (625.6956088940345 / (24))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (24) - 2, .025) and -0.12022041328875002 = (1428.4400395012904 - 1600.167691408334) / 1428.4400395012904 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Collections.IterateFor<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IList - Duration of single invocation 1.46 μs 1.57 μs 1.08 0.02 False
ImmutableSortedSet - Duration of single invocation 11.13 μs 12.07 μs 1.08 0.03 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateFor&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IterateFor<Int32>.IList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.5654895199641787 > 1.5319539413809211.
IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 8/5/2022 10:47:59 PM, 8/17/2022 6:09:21 PM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -13.170778512474403 (T) = (0 -1567.0638895614184) / Math.Sqrt((1936.4590648288442 / (42)) + (2.1046169251587816 / (23))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (23) - 2, .025) and -0.060587243509526406 = (1477.5435959195022 - 1567.0638895614184) / 1477.5435959195022 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateFor&lt;Int32&gt;.ImmutableSortedSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 12.065987168341064 > 11.944635750066254.
IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 8/1/2022 8:26:50 AM, 9/7/2022 1:05:52 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -14.894951655015877 (T) = (0 -12085.316427251904) / Math.Sqrt((235948.2940801943 / (40)) + (837.4155688035534 / (24))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (24) - 2, .025) and -0.1048967116363983 = (10937.960354097755 - 12085.316427251904) / 10937.960354097755 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Globalization.Tests.StringEquality

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Compare_DifferentFirstChar - Duration of single invocation 11.59 ns 12.68 ns 1.09 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringEquality*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Globalization.Tests.StringEquality.Compare_DifferentFirstChar(Count: 1024, Options: (en-US, Ordinal))

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 12.679168272711198 > 12.385542235253725.
IsChangePoint: Marked as a change because one of 7/27/2022 9:27:06 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -16.454939612807518 (T) = (0 -12.79311305577415) / Math.Sqrt((0.0649167835252539 / (42)) + (0.0174763694029138 / (23))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (23) - 2, .025) and -0.06582431433228612 = (12.003022340308247 - 12.79311305577415) / 12.003022340308247 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableDictionary - Duration of single invocation 15.98 μs 17.23 μs 1.08 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.TryGetValueFalse<Int32, Int32>.ImmutableDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 17.23268517495396 > 16.832199044459756.
IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 8/18/2022 3:00:51 AM, 8/23/2022 11:46:46 PM, 9/3/2022 11:44:47 AM, 9/13/2022 12:23:36 PM, 9/24/2022 1:05:07 PM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -42.418799310245106 (T) = (0 -17209.814473334438) / Math.Sqrt((24212.652443744137 / (44)) + (5477.505214317151 / (24))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (24) - 2, .025) and -0.07385170517692229 = (16026.248680677038 - 17209.814473334438) / 16026.248680677038 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Backtracking - Duration of single invocation 1.22 μs 1.34 μs 1.10 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: None)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.3393622201127497 > 1.2842308101644955.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/27/2022 6:12:29 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -12.398492371769676 (T) = (0 -1333.3184510229314) / Math.Sqrt((1836.844096017339 / (40)) + (327.18073341318524 / (23))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (23) - 2, .025) and -0.07772246586892533 = (1237.1630853477004 - 1333.3184510229314) / 1237.1630853477004 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Collections.IndexerSetReverse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IList - Duration of single invocation 1.01 μs 1.12 μs 1.11 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSetReverse&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IndexerSetReverse<Int32>.IList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.1191184165325023 > 1.0588976333674747.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/23/2022 11:46:46 PM, 9/23/2022 1:16:35 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -21.788225115213248 (T) = (0 -1119.357770080755) / Math.Sqrt((1438.633236856536 / (44)) + (1.1183188047989947 / (24))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (24) - 2, .025) and -0.12534160839729938 = (994.6826472318335 - 1119.357770080755) / 994.6826472318335 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Collections.IterateForEachNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 3.50 μs 3.87 μs 1.11 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEachNonGeneric&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IterateForEachNonGeneric<Int32>.Queue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 3.868074950124688 > 3.676747304881558.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/17/2022 1:34:50 AM, 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -22.736724313883556 (T) = (0 -3833.449432405787) / Math.Sqrt((4813.710557004849 / (41)) + (1577.095172281156 / (22))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (22) - 2, .025) and -0.08880263927649903 = (3520.793662801079 - 3833.449432405787) / 3520.793662801079 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline fb5f07f9c580bdcd5d0726d1391c2a52a01030f8
Compare ae4dc9cef942258098164beae987589c9d569215
Diff Diff

Regressions in System.Tests.Perf_SByte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 10.58 ns 12.38 ns 1.17 0.21 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_SByte*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_SByte.Parse(value: "-128")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 12.382713367486751 > 11.517200726963729.
IsChangePoint: Marked as a change because one of 10/2/2022 4:08:26 PM, 10/6/2022 2:13:23 PM falls between 9/28/2022 12:34:15 AM and 10/6/2022 2:13:23 PM.
IsRegressionStdDev: Marked as regression because -6.660258161541821 (T) = (0 -11.606423515916257) / Math.Sqrt((0.2878157444381608 / (42)) + (0.07638011432364711 / (23))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (23) - 2, .025) and -0.06143622644775711 = (10.934640467999435 - 11.606423515916257) / 10.934640467999435 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

This was referenced Oct 11, 2022
Closed
Closed
Open
Closed
Open
@DrewScoggins DrewScoggins mentioned this issue Oct 11, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
CoreClr refs/heads/main Regression RunKind=micro ubuntu 18.04 untriaged x64
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
0 participants