[Perf] Windows/x64: 3 Regressions on 10/14/2022 1:19:37 PM

[performanceautofiler]

Run Information

Architecture	x64
OS	Windows 10.0.19042
Baseline	8bc9a0748ea2e0b28041ac48100d266aacd27108
Compare	a07188777ffa92adc8a00ca974bd1c63156d19f1
Diff	Diff

Regressions in System.Memory.Span<Char>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Fill - Duration of single invocation	8.36 ns	17.49 ns	2.09	0.07	False
SequenceCompareTo - Duration of single invocation	22.59 ns	26.17 ns	1.16	0.09	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span<Char>*'

Related Issues

Regressions

• [Perf] Alpine/x64: 1 Regression on 10/14/2022 1:19:37 PM
• [Perf] Windows/x64: 2 Regressions on 10/14/2022 1:19:37 PM

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.Span<Char>.Fill(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 17.48751313839686 > 8.741671533105968.
IsChangePoint: Marked as a change because one of 9/13/2022 12:23:36 PM, 9/22/2022 3:11:11 PM, 9/27/2022 11:22:50 AM, 10/6/2022 2:13:23 PM, 10/14/2022 9:06:25 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -5.446731940724723 (T) = (0 -17.468213665459224) / Math.Sqrt((28.642907577484504 / (42)) + (0.0003115714533808055 / (12))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (12) - 2, .025) and -0.34680397750401776 = (12.97012331210398 - 17.468213665459224) / 12.97012331210398 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span<Char>.SequenceCompareTo(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 26.166458490996888 > 23.79136111398909.
IsChangePoint: Marked as a change because one of 8/11/2022 4:03:37 AM, 9/3/2022 11:44:47 AM, 9/9/2022 3:12:19 AM, 9/13/2022 12:23:36 PM, 9/29/2022 7:42:19 AM, 10/11/2022 8:42:28 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -20.762879787024907 (T) = (0 -25.52598969557747) / Math.Sqrt((0.16709065066244252 / (27)) + (0.22462907983743918 / (27))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (27) - 2, .025) and -0.1086154949877465 = (23.025106370049073 - 25.52598969557747) / 23.025106370049073 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	Windows 10.0.19042
Baseline	8bc9a0748ea2e0b28041ac48100d266aacd27108
Compare	a07188777ffa92adc8a00ca974bd1c63156d19f1
Diff	Diff

Regressions in System.Collections.Concurrent.AddRemoveFromDifferentThreads<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentBag - Duration of single invocation	160.36 ms	251.89 ms	1.57	0.28	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Concurrent.AddRemoveFromDifferentThreads<Int32>*'

Related Issues

Regressions

• [Perf] Alpine/x64: 1 Regression on 10/14/2022 1:19:37 PM
• [Perf] Windows/x64: 2 Regressions on 10/14/2022 1:19:37 PM

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.Concurrent.AddRemoveFromDifferentThreads<Int32>.ConcurrentBag(Size: 2000000)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 251.89416 > 166.45408561403508.
IsChangePoint: Marked as a change because one of 10/14/2022 9:06:25 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -6.251600959264772 (T) = (0 -204209888.00925925) / Math.Sqrt((38965408326384.766 / (40)) + (602181085961338.1 / (12))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (12) - 2, .025) and -0.28034211516548907 = (159496345.22712263 - 204209888.00925925) / 159496345.22712263 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[EgorBo]

Span regressions caused by BDN update dotnet/performance@4a35773, ConcurrentBag is likely dotnet/runtime#76451