[Perf] Windows/arm64: 13 Regressions on 2/9/2023 7:28:55 PM

[performanceautofiler]

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	46cb4eddbb1f4bebdea970964d7449bacde8df1a
Diff	Diff

Regressions in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
MultipleSerial - Duration of single invocation	12.65 μs	13.90 μs	1.10	0.08	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests<Byte>*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.MultipleSerial(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: False)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.903131133333334 > 13.28656064346154.
IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/13/2023 9:23:19 AM, 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -8.47746395354439 (T) = (0 -14086.452661076604) / Math.Sqrt((201431.54204090036 / (15)) + (246586.0123967142 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.10317288185797452 = (12769.034566324784 - 14086.452661076604) / 12769.034566324784 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	46cb4eddbb1f4bebdea970964d7449bacde8df1a
Diff	Diff

Regressions in System.Collections.TryAddDefaultSize<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentDictionary - Duration of single invocation	54.85 μs	79.78 μs	1.45	0.16	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddDefaultSize<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryAddDefaultSize<Int32>.ConcurrentDictionary(Count: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 79.78088329081633 > 51.851510886955.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -36.02268248256153 (T) = (0 -81885.08173008244) / Math.Sqrt((7979602.18205169 / (15)) + (5263894.918123484 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.6226461805012524 = (50463.916726927666 - 81885.08173008244) / 50463.916726927666 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	46cb4eddbb1f4bebdea970964d7449bacde8df1a
Diff	Diff

Regressions in System.MathBenchmarks.Single

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Min - Duration of single invocation	3.20 μs	3.85 μs	1.20	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Single.Min

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.85335250102501 > 3.363071201700767.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -72.16253197026325 (T) = (0 -3790.2917219222486) / Math.Sqrt((9.384504131651406 / (15)) + (1503.306974448837 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.1829508251283307 = (3204.0991403941616 - 3790.2917219222486) / 3204.0991403941616 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	46cb4eddbb1f4bebdea970964d7449bacde8df1a
Diff	Diff

Regressions in Microsoft.Extensions.DependencyInjection.GetService

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Transient - Duration of single invocation	34.07 ns	38.67 ns	1.14	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetService*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.GetService.Transient

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.674621151813895 > 36.37125237793037.
IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -5.8445217760361325 (T) = (0 -36.86058371962873) / Math.Sqrt((0.5798666611888657 / (15)) + (2.553031864730718 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.0653476189677739 = (34.59958333162965 - 36.86058371962873) / 34.59958333162965 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	46cb4eddbb1f4bebdea970964d7449bacde8df1a
Diff	Diff

Regressions in System.Collections.CtorDefaultSize<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentDictionary - Duration of single invocation	129.41 ns	146.60 ns	1.13	0.16	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorDefaultSize<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorDefaultSize<String>.ConcurrentDictionary

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 146.60192297941384 > 136.34057449634733.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -16.91327165355278 (T) = (0 -145.59783764067862) / Math.Sqrt((6.363754118412196 / (15)) + (9.069585224152616 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.11744346072528512 = (130.2954849690357 - 145.59783764067862) / 130.2954849690357 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	46cb4eddbb1f4bebdea970964d7449bacde8df1a
Diff	Diff

Regressions in System.Tests.Perf_String

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Substring_Int - Duration of single invocation	11.43 ns	12.56 ns	1.10	0.43	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Substring_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 10)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.557327310527013 > 11.955239775965355.
IsChangePoint: Marked as a change because one of 1/5/2023 9:23:01 PM, 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -5.529651190333107 (T) = (0 -12.506627675131508) / Math.Sqrt((0.32943428442269146 / (15)) + (0.21087154377118342 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.08461069007213323 = (11.530983227078226 - 12.506627675131508) / 11.530983227078226 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	46cb4eddbb1f4bebdea970964d7449bacde8df1a
Diff	Diff

Regressions in IfStatements.IfStatements

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Or - Duration of single invocation	67.99 μs	72.66 μs	1.07	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'IfStatements.IfStatements*'

Payloads

Baseline
Compare

Histogram

IfStatements.IfStatements.Or

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 72.65589669051879 > 71.42322012959868.
IsChangePoint: Marked as a change because one of 11/18/2022 7:31:41 AM, 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -143.00316770472034 (T) = (0 -72862.00285237939) / Math.Sqrt((1165.0564972977688 / (15)) + (24610.159540116543 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.07122547494528153 = (68017.42915617386 - 72862.00285237939) / 68017.42915617386 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	46cb4eddbb1f4bebdea970964d7449bacde8df1a
Diff	Diff

Regressions in System.Collections.CreateAddAndClear<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentDictionary - Duration of single invocation	51.02 μs	81.56 μs	1.60	0.13	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<Int32>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 81.5626402393418 > 53.58591798155739.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -221.11291489710928 (T) = (0 -82609.4611889647) / Math.Sqrt((100966.57032097099 / (15)) + (317070.2721605625 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.6217660531301649 = (50937.96422086927 - 82609.4611889647) / 50937.96422086927 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	46cb4eddbb1f4bebdea970964d7449bacde8df1a
Diff	Diff

Regressions in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedDictionary - Duration of single invocation	39.46 μs	51.31 μs	1.30	0.17	False
ConcurrentDictionary - Duration of single invocation	5.65 μs	6.10 μs	1.08	0.11	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue<Int32, Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueTrue<Int32, Int32>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.31199669966997 > 41.368352444700854.
IsChangePoint: Marked as a change because one of 11/30/2022 2:05:04 AM, 1/13/2023 8:11:05 PM, 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -6.676804284881521 (T) = (0 -45275.963792007744) / Math.Sqrt((200855.31815783793 / (15)) + (15221451.44989292 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.13788360020391527 = (39789.62679828941 - 45275.963792007744) / 39789.62679828941 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.TryGetValueTrue<Int32, Int32>.ConcurrentDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.09959033203125 > 5.810919315717753.
IsChangePoint: Marked as a change because one of 12/8/2022 3:49:24 AM, 12/8/2022 9:50:09 PM, 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -6.294417388317767 (T) = (0 -6197.963101804502) / Math.Sqrt((1172.3821030629772 / (15)) + (251904.92125774568 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.11939570563804007 = (5536.883043759533 - 6197.963101804502) / 5536.883043759533 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	46cb4eddbb1f4bebdea970964d7449bacde8df1a
Diff	Diff

Regressions in System.Collections.CtorGivenSize<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentDictionary - Duration of single invocation	313.30 ns	343.91 ns	1.10	0.15	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorGivenSize<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorGivenSize<Int32>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 343.9138301648655 > 333.1327777428095.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -7.300883956938807 (T) = (0 -348.0233671101925) / Math.Sqrt((116.67708584069996 / (15)) + (89.3220234621484 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.07716809138919987 = (323.09104762038993 - 348.0233671101925) / 323.09104762038993 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	46cb4eddbb1f4bebdea970964d7449bacde8df1a
Diff	Diff

Regressions in System.Collections.CtorDefaultSize<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentDictionary - Duration of single invocation	116.69 ns	132.17 ns	1.13	0.25	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorDefaultSize<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorDefaultSize<Int32>.ConcurrentDictionary

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 132.17113586592689 > 123.19949209542536.
IsChangePoint: Marked as a change because one of 11/4/2022 6:59:29 PM, 11/6/2022 4:58:03 PM, 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -6.706073118612964 (T) = (0 -130.54204983236764) / Math.Sqrt((8.370513864298212 / (15)) + (50.723413996048755 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.09336989054337658 = (119.39422418838662 - 130.54204983236764) / 119.39422418838662 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	46cb4eddbb1f4bebdea970964d7449bacde8df1a
Diff	Diff

Regressions in Span.IndexerBench

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
WriteViaIndexer2 - Duration of single invocation	829.27 ns	884.96 ns	1.07	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Span.IndexerBench*'

Payloads

Baseline
Compare

Histogram

Span.IndexerBench.WriteViaIndexer2(length: 1024)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 884.9602466705624 > 869.8033201657225.
IsChangePoint: Marked as a change because one of 11/18/2022 7:31:41 AM, 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -12.664032337961238 (T) = (0 -879.2029005585366) / Math.Sqrt((2.241603467683853 / (15)) + (349.47773767698874 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.05979659797103794 = (829.5958887221899 - 879.2029005585366) / 829.5958887221899 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository