Regressions in System.Collections.Tests.Perf_BitArray #85472

performanceautofiler · 2023-04-25T10:57:38Z

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
BitArraySetAll - Duration of single invocation	6.34 ns	20.21 ns	3.19	0.08	True
BitArrayOr - Duration of single invocation	10.23 ns	13.27 ns	1.30	0.17	False
BitArrayNot - Duration of single invocation	7.68 ns	12.34 ns	1.61	0.09	True
BitArrayXor - Duration of single invocation	11.10 ns	13.27 ns	1.20	0.16	False
BitArrayAnd - Duration of single invocation	10.21 ns	13.27 ns	1.30	0.14	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.212927410149806 > 6.74401509131666.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -22.782032921087676 (T) = (0 -19.388217620025838) / Math.Sqrt((0.764127380030929 / (12)) + (9.154316463812194 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -1.9018141528755539 = (6.681412591779207 - 19.388217620025838) / 6.681412591779207 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayOr(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.270547236239794 > 10.721935688096798.
IsChangePoint: Marked as a change because one of 3/3/2023 11:43:56 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -21.55662134780645 (T) = (0 -13.090932598882455) / Math.Sqrt((0.08404562154238614 / (12)) + (0.3654561295071419 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.2721811755099 = (10.29014801577732 - 13.090932598882455) / 10.29014801577732 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.34367551357869 > 8.45729261356312.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -25.173315248612905 (T) = (0 -12.225031627889667) / Math.Sqrt((0.13336744721792546 / (12)) + (0.5985314200161592 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.515541865454447 = (8.066442707093344 - 12.225031627889667) / 8.066442707093344 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayXor(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.27152853578668 > 11.195543888775145.
IsChangePoint: Marked as a change because one of 3/3/2023 11:43:56 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -18.26129989179763 (T) = (0 -13.129635689054156) / Math.Sqrt((0.14109544984139194 / (12)) + (0.37006676689544954 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.25812483042935563 = (10.435876767946352 - 13.129635689054156) / 10.435876767946352 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayAnd(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.274841288890011 > 10.724252428267.
IsChangePoint: Marked as a change because one of 3/3/2023 11:43:56 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -26.332988000240494 (T) = (0 -13.155049980522081) / Math.Sqrt((0.06340106453117673 / (12)) + (0.24686904533672896 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.2802018694796814 = (10.275762201369657 - 13.155049980522081) / 10.275762201369657 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Double

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ToStringWithFormat - Duration of single invocation	173.10 ns	189.76 ns	1.10	0.04	False
ToStringWithFormat - Duration of single invocation	187.03 ns	199.07 ns	1.06	0.05	False
ToStringWithFormat - Duration of single invocation	189.85 ns	201.99 ns	1.06	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G17")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 189.76186173085713 > 182.12087727615054.
IsChangePoint: Marked as a change because one of 2/23/2023 8:54:44 PM, 3/6/2023 11:14:52 AM, 3/16/2023 8:14:39 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -16.280367616050867 (T) = (0 -193.74048682864122) / Math.Sqrt((2.921473629852635 / (12)) + (50.95241369335574 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.1197871718227939 = (173.01545481474815 - 193.74048682864122) / 173.01545481474815 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 199.0659946216465 > 195.76746533251998.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -16.593279560293134 (T) = (0 -202.28571375792214) / Math.Sqrt((7.2385776634357315 / (11)) + (18.670439273809272 / (37))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (37) - 2, .025) and -0.0970313887972808 = (184.39373369224742 - 202.28571375792214) / 184.39373369224742 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 201.9929318081847 > 199.571868700985.
IsChangePoint: Marked as a change because one of 2/26/2023 1:18:12 AM, 3/7/2023 7:00:18 AM, 3/10/2023 8:23:08 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -16.05519271839466 (T) = (0 -204.15310026562997) / Math.Sqrt((9.116719032127556 / (12)) + (13.46376130056293 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.09094311799718856 = (187.13450490473335 - 204.15310026562997) / 187.13450490473335 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Enum

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString_Flags - Duration of single invocation	21.12 ns	23.11 ns	1.09	0.09	False
ToString_Flags - Duration of single invocation	20.57 ns	23.82 ns	1.16	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.ToString_Flags(value: 36)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.109282559966346 > 22.207627074377847.
IsChangePoint: Marked as a change because one of 3/5/2023 11:54:18 PM, 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -24.416003491257246 (T) = (0 -23.52948663947281) / Math.Sqrt((0.030306352843840333 / (12)) + (0.2097208620111538 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.10366664268316057 = (21.31937827011741 - 23.52948663947281) / 21.31937827011741 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_Flags(value: 32)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.819586554127373 > 22.06371874051919.
IsChangePoint: Marked as a change because one of 2/17/2023 9:53:52 PM, 2/23/2023 8:54:44 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -14.446217882630442 (T) = (0 -23.327030207622716) / Math.Sqrt((0.10704885295620135 / (12)) + (0.689660119866922 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.11459126981740116 = (20.928775273328927 - 23.327030207622716) / 20.928775273328927 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_BitOperations

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
LeadingZeroCount_ulong - Duration of single invocation	426.99 ns	453.99 ns	1.06	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BitOperations*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BitOperations.LeadingZeroCount_ulong

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 453.9888571400803 > 437.4870198443056.
IsChangePoint: Marked as a change because one of 3/6/2023 11:36:59 AM, 3/20/2023 8:31:51 AM, 4/11/2023 3:40:00 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -6.899473543431359 (T) = (0 -449.63921276123074) / Math.Sqrt((127.73613522789564 / (12)) + (24.48687560196992 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.054402916989556856 = (426.43965178416147 - 449.63921276123074) / 426.43965178416147 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Deep

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
WriteDeepUtf16 - Duration of single invocation	4.69 ms	5.52 ms	1.18	0.08	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Deep*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: True)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.5245486956521725 > 4.963877521952104.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -24.485068782269277 (T) = (0 -5330890.328456869) / Math.Sqrt((5260312330.949635 / (11)) + (3955702275.6601996 / (37))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (37) - 2, .025) and -0.12498930332550309 = (4738614.236329708 - 5330890.328456869) / 4738614.236329708 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
EndsWith - Duration of single invocation	17.70 ns	19.94 ns	1.13	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.EndsWith(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.940756126078455 > 18.415689949805408.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/24/2023 9:01:57 AM, 3/3/2023 11:43:56 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/29/2023 12:02:44 PM, 4/9/2023 11:04:55 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -35.91000189497899 (T) = (0 -20.492056053295332) / Math.Sqrt((0.038686993493706835 / (11)) + (0.09548884472250842 / (37))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (37) - 2, .025) and -0.1585348877302128 = (17.687905880368536 - 20.492056053295332) / 17.687905880368536 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Subtract - Duration of single invocation	9.94 ns	12.09 ns	1.22	0.06	False
ToStringX - Duration of single invocation	57.93 ns	62.57 ns	1.08	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.Subtract(arguments: 16,16 bits)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.093060336348236 > 10.521474691525725.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -77.16242824139755 (T) = (0 -12.085444119274811) / Math.Sqrt((0.006091211191040216 / (12)) + (0.007867817031197065 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.2067816437762143 = (10.0146071839952 - 12.085444119274811) / 10.0146071839952 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString: 123)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 62.56798798274013 > 60.847873618535985.
IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 2/10/2023 8:30:00 AM, 2/24/2023 8:06:06 PM, 3/22/2023 7:51:01 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -14.40322671948516 (T) = (0 -62.92725860755048) / Math.Sqrt((0.9228104554270762 / (12)) + (0.5065039595240335 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.07398793721799111 = (58.59214654733865 - 62.92725860755048) / 58.59214654733865 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetBytes - Duration of single invocation	17.05 ns	18.85 ns	1.11	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetBytes(size: 16, encName: "utf-8")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.846580134218538 > 17.9414329037105.
IsChangePoint: Marked as a change because one of 2/23/2023 8:54:44 PM, 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/25/2023 12:18:49 AM, 4/5/2023 6:15:03 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -11.462439215091996 (T) = (0 -19.55628530717003) / Math.Sqrt((0.37676268568474464 / (12)) + (0.08885202522279244 / (36))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (36) - 2, .025) and -0.12090212313203894 = (17.44691610764873 - 19.55628530717003) / 17.44691610764873 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Single

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToStringWithFormat - Duration of single invocation	164.68 ns	181.63 ns	1.10	0.04	False
ToStringWithFormat - Duration of single invocation	168.46 ns	190.61 ns	1.13	0.06	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G17")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 181.63333167990325 > 173.21449436062247.
IsChangePoint: Marked as a change because one of 2/17/2023 9:53:52 PM, 2/23/2023 8:54:44 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -10.615428136554076 (T) = (0 -192.92588894924864) / Math.Sqrt((26.942965029812935 / (12)) + (80.16547356032582 / (36))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (36) - 2, .025) and -0.13168168811846295 = (170.47716771843128 - 192.92588894924864) / 170.47716771843128 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G17")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 190.60926047735387 > 176.61243956619884.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -11.918662808225005 (T) = (0 -195.35381811169196) / Math.Sqrt((22.693571317953644 / (11)) + (59.186693849490936 / (35))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (35) - 2, .025) and -0.13405838436949685 = (172.26081196895603 - 195.35381811169196) / 172.26081196895603 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

performanceautofiler · 2023-04-25T10:57:42Z

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsTrue<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ICollection - Duration of single invocation	418.24 μs	450.05 μs	1.08	0.08	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<String>.ICollection(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 450.04744047619045 > 439.81782319078945.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -6.915511891108824 (T) = (0 -456612.5545593939) / Math.Sqrt((177884375.83685806 / (12)) + (196994040.0356963 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.07294011480620924 = (425571.33269443066 - 456612.5545593939) / 425571.33269443066 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.CastingPerf

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ScalarValueTypeObj - Duration of single invocation	278.52 μs	306.35 μs	1.10	0.03	False
ObjrefValueTypeObj - Duration of single invocation	278.41 μs	306.31 μs	1.10	0.04	False
IntObj - Duration of single invocation	278.44 μs	306.46 μs	1.10	0.04	False
FooObjIsFoo - Duration of single invocation	278.60 μs	306.49 μs	1.10	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf.ScalarValueTypeObj

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 306.3475773001508 > 292.4505637770433.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -656.046552854986 (T) = (0 -306386.778021153) / Math.Sqrt((18672.635718482783 / (12)) + (9129.638899069914 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.10000787621105743 = (278531.4402261306 - 306386.778021153) / 278531.4402261306 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

PerfLabTests.CastingPerf.ObjrefValueTypeObj

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 306.31208858543414 > 292.3966451322115.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -838.0905066332441 (T) = (0 -306354.00740025734) / Math.Sqrt((11261.843632552698 / (12)) + (6152.321257190443 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.10002410963038778 = (278497.53902502503 - 306354.00740025734) / 278497.53902502503 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

PerfLabTests.CastingPerf.IntObj

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 306.46116946778716 > 292.441201171875.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -807.9901910210531 (T) = (0 -306357.76509096357) / Math.Sqrt((11614.127680945023 / (12)) + (8004.306362643904 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.09981733987841093 = (278553.31424837565 - 306357.76509096357) / 278553.31424837565 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

PerfLabTests.CastingPerf.FooObjIsFoo

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 306.4854901960784 > 292.4477490234375.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -1257.1749801102367 (T) = (0 -306346.8586710498) / Math.Sqrt((3167.0400773902365 / (12)) + (8388.488591019073 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.09998851443242886 = (278500.0521838343 - 306346.8586710498) / 278500.0521838343 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.MathBenchmarks.Single

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Hypot - Duration of single invocation	15.32 μs	16.72 μs	1.09	0.02	False
Exp - Duration of single invocation	19.50 μs	20.89 μs	1.07	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Single.Hypot

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.719442513368985 > 16.089785437773.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/22/2023 7:51:01 PM, 4/6/2023 1:55:21 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -783.3359869112949 (T) = (0 -16718.61600943784) / Math.Sqrt((28.311748540684718 / (12)) + (28.667436583432703 / (36))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (36) - 2, .025) and -0.09078861578740795 = (15327.090664004745 - 16718.61600943784) / 15327.090664004745 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.MathBenchmarks.Single.Exp

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.893140040106957 > 20.48001608696249.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 4/6/2023 1:55:21 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -603.6473618458264 (T) = (0 -20895.809779150302) / Math.Sqrt((54.82220569687902 / (12)) + (27.985913790487007 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.07142342787260947 = (19502.8494202712 - 20895.809779150302) / 19502.8494202712 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
FrozenDictionaryOptimized - Duration of single invocation	2.02 μs	2.30 μs	1.14	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<Int32, Int32>.FrozenDictionaryOptimized(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3003796309944966 > 2.121923287750782.
IsChangePoint: Marked as a change because one of 4/5/2023 11:58:06 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -20.180275088602635 (T) = (0 -2299.50903272552) / Math.Sqrt((2088.2312870667547 / (12)) + (0.7936887967574187 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.13093476004065316 = (2033.2817718352387 - 2299.50903272552) / 2033.2817718352387 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsFalse<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Span - Duration of single invocation	712.71 μs	814.97 μs	1.14	0.19	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<String>.Span(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 814.9746428571428 > 748.3568026200772.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -6.787256941935116 (T) = (0 -818622.7801391255) / Math.Sqrt((1582264920.3870263 / (12)) + (1448672606.3195305 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.12160754453806688 = (729865.6148717996 - 818622.7801391255) / 729865.6148717996 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
EmptyTakeSelectToArray - Duration of single invocation	17.54 ns	19.10 ns	1.09	0.12	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.EmptyTakeSelectToArray

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.097756083601173 > 17.840828734250543.
IsChangePoint: Marked as a change because one of 2/13/2023 6:54:14 PM, 2/20/2023 9:09:49 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -18.39626057261227 (T) = (0 -18.91344323686354) / Math.Sqrt((0.10282578512782935 / (12)) + (0.05821642135092264 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.10859189897705936 = (17.060780666280987 - 18.91344323686354) / 17.060780666280987 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.CastingPerf2.CastingPerf

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IntObj - Duration of single invocation	111.38 μs	167.04 μs	1.50	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf2.CastingPerf.IntObj

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 167.0387172667758 > 116.9559088243317.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -4011.392829358727 (T) = (0 -167102.0992753773) / Math.Sqrt((1532.2055413495834 / (12)) + (2403.8234130497426 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.49969152043087367 = (111424.31426655495 - 167102.0992753773) / 111424.31426655495 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateForEach<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
FrozenDictionary - Duration of single invocation	2.44 μs	2.59 μs	1.06	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<String>.FrozenDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.5893050945901095 > 2.565480535160109.
IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -21.879440030515823 (T) = (0 -2602.0185011971757) / Math.Sqrt((1.0390532795249057 / (12)) + (1941.5098888042512 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.0649186091188095 = (2443.3965928628795 - 2602.0185011971757) / 2443.3965928628795 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_Path

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetFileNameWithoutExtension - Duration of single invocation	27.57 ns	30.38 ns	1.10	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Path.GetFileNameWithoutExtension

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.384713429173438 > 28.857825995642738.
IsChangePoint: Marked as a change because one of 2/9/2023 4:10:34 PM, 2/14/2023 11:21:41 AM, 3/17/2023 9:25:32 PM, 3/20/2023 8:31:51 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -26.520003044153448 (T) = (0 -30.48910070212556) / Math.Sqrt((0.03620090670624483 / (11)) + (0.3421020278843808 / (36))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (36) - 2, .025) and -0.10912093296671935 = (27.489428605925 - 30.48910070212556) / 27.489428605925 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryParseSingle - Duration of single invocation	31.07 ns	32.98 ns	1.06	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: 12345)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.97781688421849 > 32.62557112859975.
IsChangePoint: Marked as a change because one of 2/9/2023 4:10:34 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -32.028029721462815 (T) = (0 -33.40991687844801) / Math.Sqrt((0.034736520337854834 / (12)) + (0.07747592802288111 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.07262648704158192 = (31.147764186390845 - 33.40991687844801) / 31.147764186390845 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.BinaryWriterExtendedTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
WriteAsciiCharArray - Duration of single invocation	81.54 μs	88.34 μs	1.08	0.00	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 2000000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 88.34201586787565 > 84.78663289304123.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -33.90762173483388 (T) = (0 -87747.52615838259) / Math.Sqrt((131260.25173041897 / (12)) + (1032703.3432310184 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.08244403724199545 = (81064.26118985162 - 87747.52615838259) / 81064.26118985162 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-25T10:57:43Z

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in SciMark2.kernel

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
benchMonteCarlo - Duration of single invocation	732.58 ms	776.09 ms	1.06	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'SciMark2.kernel*'

Payloads

Baseline
Compare

Histogram

SciMark2.kernel.benchMonteCarlo

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 776.0866733333334 > 769.085247.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -244.50389085779807 (T) = (0 -776852159.0733589) / Math.Sqrt((314764066031.21893 / (12)) + (234489354084.88895 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.060219624096299135 = (732727579.661172 - 776852159.0733589) / 732727579.661172 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Char>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IterateTryGetArray - Duration of single invocation	6.89 ns	7.97 ns	1.16	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateTryGetArray

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.968825114812262 > 7.235449130888102.
IsChangePoint: Marked as a change because one of 2/24/2023 8:06:06 PM, 3/25/2023 12:18:49 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -954.1864951779512 (T) = (0 -7.971466174293845) / Math.Sqrt((1.0790463871684801E-05 / (12)) + (1.395053752640102E-05 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.1563724083317204 = (6.893511222560342 - 7.971466174293845) / 6.893511222560342 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

kunalspathak · 2023-04-25T16:42:55Z

@cincuranet - can you check the commit range between 4/15 and 4/17 of when it regressed?
@AndyAyersMS - most likely from #84875.

cincuranet · 2023-04-26T07:31:54Z

@kunalspathak The regression first occurred between 9c02b25...e92289a or possibly 2d833f4...e92289a (taking System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size: 512) and System.Collections.Tests.Perf_BitArray.BitArrayOr(Size: 512)).

kunalspathak · 2023-04-26T15:16:29Z

Thanks @cincuranet for finding out. @AndyAyersMS - could this be from #84817?

ghost · 2023-04-29T14:48:57Z

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
BitArraySetAll - Duration of single invocation	6.34 ns	20.21 ns	3.19	0.08	True
BitArrayOr - Duration of single invocation	10.23 ns	13.27 ns	1.30	0.17	False
BitArrayNot - Duration of single invocation	7.68 ns	12.34 ns	1.61	0.09	True
BitArrayXor - Duration of single invocation	11.10 ns	13.27 ns	1.20	0.16	False
BitArrayAnd - Duration of single invocation	10.21 ns	13.27 ns	1.30	0.14	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.212927410149806 > 6.74401509131666.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -22.782032921087676 (T) = (0 -19.388217620025838) / Math.Sqrt((0.764127380030929 / (12)) + (9.154316463812194 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -1.9018141528755539 = (6.681412591779207 - 19.388217620025838) / 6.681412591779207 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayOr(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.270547236239794 > 10.721935688096798.
IsChangePoint: Marked as a change because one of 3/3/2023 11:43:56 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -21.55662134780645 (T) = (0 -13.090932598882455) / Math.Sqrt((0.08404562154238614 / (12)) + (0.3654561295071419 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.2721811755099 = (10.29014801577732 - 13.090932598882455) / 10.29014801577732 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.34367551357869 > 8.45729261356312.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -25.173315248612905 (T) = (0 -12.225031627889667) / Math.Sqrt((0.13336744721792546 / (12)) + (0.5985314200161592 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.515541865454447 = (8.066442707093344 - 12.225031627889667) / 8.066442707093344 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayXor(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.27152853578668 > 11.195543888775145.
IsChangePoint: Marked as a change because one of 3/3/2023 11:43:56 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -18.26129989179763 (T) = (0 -13.129635689054156) / Math.Sqrt((0.14109544984139194 / (12)) + (0.37006676689544954 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.25812483042935563 = (10.435876767946352 - 13.129635689054156) / 10.435876767946352 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayAnd(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.274841288890011 > 10.724252428267.
IsChangePoint: Marked as a change because one of 3/3/2023 11:43:56 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -26.332988000240494 (T) = (0 -13.155049980522081) / Math.Sqrt((0.06340106453117673 / (12)) + (0.24686904533672896 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.2802018694796814 = (10.275762201369657 - 13.155049980522081) / 10.275762201369657 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Double

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ToStringWithFormat - Duration of single invocation	173.10 ns	189.76 ns	1.10	0.04	False
ToStringWithFormat - Duration of single invocation	187.03 ns	199.07 ns	1.06	0.05	False
ToStringWithFormat - Duration of single invocation	189.85 ns	201.99 ns	1.06	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G17")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 189.76186173085713 > 182.12087727615054.
IsChangePoint: Marked as a change because one of 2/23/2023 8:54:44 PM, 3/6/2023 11:14:52 AM, 3/16/2023 8:14:39 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -16.280367616050867 (T) = (0 -193.74048682864122) / Math.Sqrt((2.921473629852635 / (12)) + (50.95241369335574 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.1197871718227939 = (173.01545481474815 - 193.74048682864122) / 173.01545481474815 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 199.0659946216465 > 195.76746533251998.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -16.593279560293134 (T) = (0 -202.28571375792214) / Math.Sqrt((7.2385776634357315 / (11)) + (18.670439273809272 / (37))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (37) - 2, .025) and -0.0970313887972808 = (184.39373369224742 - 202.28571375792214) / 184.39373369224742 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 201.9929318081847 > 199.571868700985.
IsChangePoint: Marked as a change because one of 2/26/2023 1:18:12 AM, 3/7/2023 7:00:18 AM, 3/10/2023 8:23:08 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -16.05519271839466 (T) = (0 -204.15310026562997) / Math.Sqrt((9.116719032127556 / (12)) + (13.46376130056293 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.09094311799718856 = (187.13450490473335 - 204.15310026562997) / 187.13450490473335 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Enum

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString_Flags - Duration of single invocation	21.12 ns	23.11 ns	1.09	0.09	False
ToString_Flags - Duration of single invocation	20.57 ns	23.82 ns	1.16	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.ToString_Flags(value: 36)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.109282559966346 > 22.207627074377847.
IsChangePoint: Marked as a change because one of 3/5/2023 11:54:18 PM, 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -24.416003491257246 (T) = (0 -23.52948663947281) / Math.Sqrt((0.030306352843840333 / (12)) + (0.2097208620111538 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.10366664268316057 = (21.31937827011741 - 23.52948663947281) / 21.31937827011741 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_Flags(value: 32)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.819586554127373 > 22.06371874051919.
IsChangePoint: Marked as a change because one of 2/17/2023 9:53:52 PM, 2/23/2023 8:54:44 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -14.446217882630442 (T) = (0 -23.327030207622716) / Math.Sqrt((0.10704885295620135 / (12)) + (0.689660119866922 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.11459126981740116 = (20.928775273328927 - 23.327030207622716) / 20.928775273328927 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_BitOperations

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
LeadingZeroCount_ulong - Duration of single invocation	426.99 ns	453.99 ns	1.06	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BitOperations*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BitOperations.LeadingZeroCount_ulong

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 453.9888571400803 > 437.4870198443056.
IsChangePoint: Marked as a change because one of 3/6/2023 11:36:59 AM, 3/20/2023 8:31:51 AM, 4/11/2023 3:40:00 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -6.899473543431359 (T) = (0 -449.63921276123074) / Math.Sqrt((127.73613522789564 / (12)) + (24.48687560196992 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.054402916989556856 = (426.43965178416147 - 449.63921276123074) / 426.43965178416147 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Deep

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
WriteDeepUtf16 - Duration of single invocation	4.69 ms	5.52 ms	1.18	0.08	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Deep*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: True)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.5245486956521725 > 4.963877521952104.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -24.485068782269277 (T) = (0 -5330890.328456869) / Math.Sqrt((5260312330.949635 / (11)) + (3955702275.6601996 / (37))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (37) - 2, .025) and -0.12498930332550309 = (4738614.236329708 - 5330890.328456869) / 4738614.236329708 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
EndsWith - Duration of single invocation	17.70 ns	19.94 ns	1.13	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.EndsWith(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.940756126078455 > 18.415689949805408.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/24/2023 9:01:57 AM, 3/3/2023 11:43:56 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/29/2023 12:02:44 PM, 4/9/2023 11:04:55 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -35.91000189497899 (T) = (0 -20.492056053295332) / Math.Sqrt((0.038686993493706835 / (11)) + (0.09548884472250842 / (37))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (37) - 2, .025) and -0.1585348877302128 = (17.687905880368536 - 20.492056053295332) / 17.687905880368536 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Subtract - Duration of single invocation	9.94 ns	12.09 ns	1.22	0.06	False
ToStringX - Duration of single invocation	57.93 ns	62.57 ns	1.08	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.Subtract(arguments: 16,16 bits)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.093060336348236 > 10.521474691525725.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -77.16242824139755 (T) = (0 -12.085444119274811) / Math.Sqrt((0.006091211191040216 / (12)) + (0.007867817031197065 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.2067816437762143 = (10.0146071839952 - 12.085444119274811) / 10.0146071839952 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString: 123)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 62.56798798274013 > 60.847873618535985.
IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 2/10/2023 8:30:00 AM, 2/24/2023 8:06:06 PM, 3/22/2023 7:51:01 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -14.40322671948516 (T) = (0 -62.92725860755048) / Math.Sqrt((0.9228104554270762 / (12)) + (0.5065039595240335 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.07398793721799111 = (58.59214654733865 - 62.92725860755048) / 58.59214654733865 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetBytes - Duration of single invocation	17.05 ns	18.85 ns	1.11	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetBytes(size: 16, encName: "utf-8")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.846580134218538 > 17.9414329037105.
IsChangePoint: Marked as a change because one of 2/23/2023 8:54:44 PM, 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/25/2023 12:18:49 AM, 4/5/2023 6:15:03 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -11.462439215091996 (T) = (0 -19.55628530717003) / Math.Sqrt((0.37676268568474464 / (12)) + (0.08885202522279244 / (36))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (36) - 2, .025) and -0.12090212313203894 = (17.44691610764873 - 19.55628530717003) / 17.44691610764873 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Single

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToStringWithFormat - Duration of single invocation	164.68 ns	181.63 ns	1.10	0.04	False
ToStringWithFormat - Duration of single invocation	168.46 ns	190.61 ns	1.13	0.06	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G17")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 181.63333167990325 > 173.21449436062247.
IsChangePoint: Marked as a change because one of 2/17/2023 9:53:52 PM, 2/23/2023 8:54:44 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -10.615428136554076 (T) = (0 -192.92588894924864) / Math.Sqrt((26.942965029812935 / (12)) + (80.16547356032582 / (36))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (36) - 2, .025) and -0.13168168811846295 = (170.47716771843128 - 192.92588894924864) / 170.47716771843128 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G17")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 190.60926047735387 > 176.61243956619884.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -11.918662808225005 (T) = (0 -195.35381811169196) / Math.Sqrt((22.693571317953644 / (11)) + (59.186693849490936 / (35))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (35) - 2, .025) and -0.13405838436949685 = (172.26081196895603 - 195.35381811169196) / 172.26081196895603 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	AndyAyersMS
Labels:	`os-windows`, `arch-x64`, `area-CodeGen-coreclr`, `untriaged`, `runtime-coreclr`, `needs-area-label`
Milestone:	-

AndyAyersMS · 2023-04-29T15:41:45Z

Note regression in System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size: 512) is amd64 (orange) only, not seeng this on intel (blue)

Similarly for System.Collections.Tests.Perf_BitArray.BitArrayOr(Size: 512)

AndyAyersMS · 2023-05-12T22:36:46Z

Seems like this has also fixed itself (somewhat) at least the worst cases:

Will have to dig through some the others to see if there's anything that is not back to "normal" here.

AndyAyersMS · 2023-05-15T16:07:53Z

Here's the full set, sorted from worst to least worst.

Note we are just evaluating the regression between 4/12 and 4/17 here.

Ratio	Test	Notes
3.19	System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size: 512)	Fixed itself
1.61	System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 512)	Fixed itself
1.50	PerfLabTests.CastingPerf2.CastingPerf.IntObj	Bistable
1.30	System.Collections.Tests.Perf_BitArray.BitArrayOr(Size: 512)	Fixed itself
1.30	System.Collections.Tests.Perf_BitArray.BitArrayAnd(Size: 512)	Fixed itself
1.22	System.Numerics.Tests.Perf_BigInteger.Subtract(arguments: 16,16 bits)	Still regressed Analysis
1.20	System.Collections.Tests.Perf_BitArray.BitArrayXor(Size: 512)	Fixed itself
1.18	System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: True)	Fixed itself
1.16	System.Tests.Perf_Enum.ToString_Flags(value: 32)	Still regressed
1.16	System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateTryGetArray	Fixed itself
1.14	System.Collections.TryGetValueFalse<Int32, Int32>.FrozenDictionaryOptimized(Size: 512)	Bistable
1.14	System.Collections.ContainsFalse<String>.Span(Size: 512)	Still regressed
1.13	System.Memory.Span<Int32>.EndsWith(Size: 512)	Multi-stable
1.13	System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G17")	Fixed itself
1.11	System.Text.Tests.Perf_Encoding.GetBytes(size: 16, encName: "utf-8")	Still regressed
1.10	PerfLabTests.CastingPerf.ScalarValueTypeObj	bistable
1.10	PerfLabTests.CastingPerf.ObjrefValueTypeObj	bistable
1.10	PerfLabTests.CastingPerf.IntObj	bistable
1.10	PerfLabTests.CastingPerf.FooObjIsFoo	bistable
1.10	System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G17")	Fixed itself
1.10	System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G17")	Fixed itself
1.10	System.IO.Tests.Perf_Path.GetFileNameWithoutExtension	Still regressed
1.09	System.Tests.Perf_Enum.ToString_Flags(value: 36)	Still regressed
1.09	System.MathBenchmarks.Single.Hypot	bistable
1.09	System.Linq.Tests.Perf_Enumerable.EmptyTakeSelectToArray	Fixed itself
1.08	System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString: 123)	Fixed itself
1.08	System.Collections.ContainsTrue<String>.ICollection(Size: 512)	Still regressed
1.08	System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 2000000)	Fixed itself
1.07	System.MathBenchmarks.Single.Exp	bistable
1.06	System.Collections.IterateForEach<String>.FrozenDictionary(Size: 512)	[Fixed itself]
1.06	System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: 12345)	Still regressed
1.06	SciMark2.kernel.benchMonteCarlo	bistable
1.06	System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G")	Fixed itself
1.06	System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G")	Fixed itself
1.06	System.Numerics.Tests.Perf_BitOperations.LeadingZeroCount_ulong	noisy

AndyAyersMS · 2023-05-15T18:49:11Z

8 benchmarks that are still regressed as of 5/15

AndyAyersMS · 2023-05-15T18:53:42Z

More that are no longer regressed

AndyAyersMS · 2023-05-16T01:31:43Z

More that are no longer regressed

AndyAyersMS · 2023-05-16T01:58:04Z

System.Numerics.Tests.Perf_BigInteger.Subtract(arguments: 16,16 bits)

The regression on 4/14 is amd64 only, about 2ns slowdown.

Since then things have gotten even worse, the regression on 5/2 looks like it may be from #85620 (see dotnet/perf-autofiling-issues#17605) which was (hopefully) fixed by #86246.

So we should check back in a day or two and see where this test has ended up.

00.70%   2.6E+05     ?        Unknown
43.63%   1.624E+07   Tier-1   [System.Runtime.Numerics]BigInteger.Subtract(value class System.ReadOnlySpan`1<unsigned int32>,int32,value class System.ReadOnlySpan`1<unsigned int32>,int32)
28.10%   1.046E+07   Tier-1   [MicroBenchmarks]Perf_BigInteger.Subtract(class BigIntegers)
13.41%   4.99E+06    native   coreclr.dll (mostly checked write barrier)
08.92%   3.32E+06    Tier-1   [System.Runtime.Numerics]BigInteger..ctor(int64)
04.67%   1.74E+06    Tier-1   [fbbfaa49-80f2-4fb3-b341-975678a5e8cd]Runnable_0.WorkloadActionUnroll(int64)
00.32%   1.2E+05     native   clrjit.dll
00.11%   4E+04       native   ntoskrnl.exe
00.05%   2E+04       native   ntdll.dll

This seems to be largely fixed, indeed improved by PGO, but has recently back slid some.

AndyAyersMS · 2023-05-16T02:40:12Z

System.Tests.Perf_Enum.ToString_Flags(value: 32)

The regression on 4/14 is amd64 only, about 4ns slowdown.

AndyAyersMS · 2023-05-16T02:42:48Z

System.Collections.ContainsFalse<String>.Span(Size: 512)

AndyAyersMS · 2023-06-09T01:58:34Z

Collated and updated. First few are bimodal, remainder are small.

Recent Score	Orig Score	x64-win-amd	Benchmark
1.50	1.50	1.50 1.50	PerfLabTests.CastingPerf2.CastingPerf.IntObj
1.48	3.19	1.48 3.19	System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size: 512)
1.48	1.61	1.48 1.61	System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 512)
1.25	1.13	1.25 1.13	System.Memory.Span(Int32).EndsWith(Size: 512)
1.25	1.22	1.25 1.22	System.Numerics.Tests.Perf_BigInteger.Subtract(arguments: 16,16 bits)
1.10	1.10	1.10 1.10	PerfLabTests.CastingPerf.FooObjIsFoo
1.10	1.10	1.10 1.10	PerfLabTests.CastingPerf.IntObj
1.10	1.10	1.10 1.10	PerfLabTests.CastingPerf.ObjrefValueTypeObj
1.10	1.10	1.10 1.10	PerfLabTests.CastingPerf.ScalarValueTypeObj
1.07	1.14	1.07 1.14	System.Collections.ContainsFalse(String).Span(Size: 512)
1.07	1.07	1.07 1.07	System.MathBenchmarks.Single.Exp
1.06	1.06	1.06 1.06	System.Collections.IterateForEach(String).FrozenDictionary(Size: 512)
1.04	1.30	1.04 1.30	System.Collections.Tests.Perf_BitArray.BitArrayOr(Size: 512)
1.03	1.06	1.03 1.06	System.Numerics.Tests.Perf_BitOperations.LeadingZeroCount_ulong
1.01	1.06	1.01 1.06	System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: 12345)
0.99	1.08	0.99 1.08	System.Collections.ContainsTrue(String).ICollection(Size: 512)
0.97	1.08	0.97 1.08	System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 2000000)
0.94	1.30	0.94 1.30	System.Collections.Tests.Perf_BitArray.BitArrayAnd(Size: 512)
0.94	1.08	0.94 1.08	System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString: 123)
0.93	1.16	0.93 1.16	System.Tests.Perf_Enum.ToString_Flags(value: 32)
0.93	1.09	0.93 1.09	System.Tests.Perf_Enum.ToString_Flags(value: 36)
0.93	1.11	0.93 1.11	System.Text.Tests.Perf_Encoding.GetBytes(size: 16, encName: "utf-8")
0.92	1.16	0.92 1.16	System.Buffers.Tests.ReadOnlySequenceTests(Char).IterateTryGetArray
0.88	1.06	0.88 1.06	SciMark2.kernel.benchMonteCarlo
0.87	1.06	0.87 1.06	System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G")
0.87	1.20	0.87 1.20	System.Collections.Tests.Perf_BitArray.BitArrayXor(Size: 512)
0.86	1.06	0.86 1.06	System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G")
0.86	1.18	0.86 1.18	System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: True)
0.83	1.10	0.83 1.10	System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G17")
0.82	1.09	0.82 1.09	System.MathBenchmarks.Single.Hypot
0.80	1.10	0.80 1.10	System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G17")
0.78	1.09	0.78 1.09	System.Linq.Tests.Perf_Enumerable.EmptyTakeSelectToArray
0.76	1.10	0.76 1.10	System.IO.Tests.Perf_Path.GetFileNameWithoutExtension
0.74	1.13	0.74 1.13	System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G17")
0.66	1.14	0.66 1.14	System.Collections.TryGetValueFalse(Int32, Int32).FrozenDictionaryOptimized(Size: 512)

performanceautofiler bot assigned kunalspathak Apr 25, 2023

performanceautofiler bot added arch-x64 os-windows runtime-coreclr specific to the CoreCLR runtime untriaged New issue has not been triaged by the area owner labels Apr 25, 2023

cincuranet removed the untriaged New issue has not been triaged by the area owner label Apr 27, 2023

cincuranet changed the title ~~[Perf] Windows/x64: 35 Regressions on 4/17/2023 1:30:24 AM~~ Regressions in System.Collections.Tests.Perf_BitArray Apr 27, 2023

dotnet-issue-labeler bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Apr 27, 2023

ghost added the untriaged New issue has not been triaged by the area owner label Apr 27, 2023

cincuranet unassigned kunalspathak Apr 27, 2023

cincuranet transferred this issue from dotnet/perf-autofiling-issues Apr 27, 2023

cincuranet assigned AndyAyersMS Apr 27, 2023

jeffschwMSFT added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Apr 29, 2023

teo-tsirpanis removed the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Apr 30, 2023

JulieLeeMSFT removed the untriaged New issue has not been triaged by the area owner label May 1, 2023

JulieLeeMSFT added this to the 8.0.0 milestone May 1, 2023

AndyAyersMS closed this as completed Jun 9, 2023

ghost locked as resolved and limited conversation to collaborators Jul 9, 2023

Regressions in System.Collections.Tests.Perf_BitArray #85472

Regressions in System.Collections.Tests.Perf_BitArray #85472

Comments

performanceautofiler bot commented Apr 25, 2023

Run Information

Regressions in System.Collections.Tests.Perf_BitArray

Repro

Payloads

Payloads

Histogram

System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size: 512)

Description of detection logic

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayOr(Size: 512)

Description of detection logic

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 512)

Description of detection logic

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayXor(Size: 512)

Description of detection logic

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayAnd(Size: 512)

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Tests.Perf_Double

Repro

Payloads

Payloads

Histogram

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G17")

Description of detection logic

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G")

Description of detection logic

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G")

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Tests.Perf_Enum

Repro

Payloads

Payloads

Histogram

System.Tests.Perf_Enum.ToString_Flags(value: 36)

Description of detection logic

JIT Disasms

System.Tests.Perf_Enum.ToString_Flags(value: 32)

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Numerics.Tests.Perf_BitOperations

Repro

Payloads

Payloads

Histogram

System.Numerics.Tests.Perf_BitOperations.LeadingZeroCount_ulong

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Text.Json.Tests.Perf_Deep

Repro

Payloads

Payloads

Histogram

System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: True)

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Memory.Span<Int32>

Repro

Payloads

Payloads