[Perf] Changes at 5/23/2022 7:50:08 PM

[performanceautofiler]

Run Information

Architecture	arm64
OS	ubuntu 18.04
Baseline	4aeec6397c7dd6198129219ed806f93d6ed675c0
Compare	110b4cfaa627deed4b06b864b06128267d0a64c2
Diff	Diff

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
BitArrayNot - Duration of single invocation	5.34 ns	16.94 ns	3.17	0.78	False
BitArrayOr - Duration of single invocation	8.86 ns	19.64 ns	2.22	0.79	False
BitArrayIntArrayCtor - Duration of single invocation	70.77 ns	79.70 ns	1.13	0.39	False
BitArrayAnd - Duration of single invocation	12.90 ns	16.32 ns	1.26	0.75	False
BitArrayXor - Duration of single invocation	15.20 ns	18.48 ns	1.22	0.74	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 4)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.94369200015564 > 6.1507962296720855.
IsChangePoint: Marked as a change because one of 3/11/2022 4:59:21 AM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -10.596836227523005 (T) = (0 -15.259266653274173) / Math.Sqrt((0.4601779737990884 / (32)) + (7.155475112321412 / (10))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (10) - 2, .025) and -1.4588829532171754 = (6.205771866167568 - 15.259266653274173) / 6.205771866167568 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayOr(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.64236254439722 > 11.473004338071622.
IsChangePoint: Marked as a change because one of 3/11/2022 5:51:22 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -12.921752022334722 (T) = (0 -17.844041081240157) / Math.Sqrt((3.6781236623136353 / (32)) + (1.9246075751118603 / (10))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (10) - 2, .025) and -0.6708336302806992 = (10.679723437361245 - 17.844041081240157) / 10.679723437361245 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayIntArrayCtor(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 79.70119284532507 > 73.15963880598702.
IsChangePoint: Marked as a change because one of 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -7.124511195550236 (T) = (0 -80.59644230864944) / Math.Sqrt((13.892972612580785 / (32)) + (13.59818013685014 / (10))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (10) - 2, .025) and -0.13429969191521626 = (71.05392241847991 - 80.59644230864944) / 71.05392241847991 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayAnd(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.31679594639665 > 10.512813746364365.
IsChangePoint: Marked as a change because one of 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -13.56168665194812 (T) = (0 -18.463610453177495) / Math.Sqrt((3.8427227044786973 / (32)) + (2.245062137578813 / (10))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (10) - 2, .025) and -0.757996609751622 = (10.502642809866465 - 18.463610453177495) / 10.502642809866465 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayXor(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.481772690955946 > 12.355400464330613.
IsChangePoint: Marked as a change because one of 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -7.2190787285585385 (T) = (0 -17.478556396833188) / Math.Sqrt((4.387410449428178 / (32)) + (7.1927647142465 / (10))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (10) - 2, .025) and -0.6186919279923477 = (10.797951169443168 - 17.478556396833188) / 10.797951169443168 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	arm64
OS	ubuntu 18.04
Baseline	4aeec6397c7dd6198129219ed806f93d6ed675c0
Compare	110b4cfaa627deed4b06b864b06128267d0a64c2
Diff	Diff

Regressions in System.Tests.Perf_Random

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Next_long_unseeded - Duration of single invocation	7.33 ns	11.53 ns	1.57	0.75	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Random*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Random.Next_long_unseeded

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.528628267095508 > 7.683869357273554.
IsChangePoint: Marked as a change because one of 2/8/2022 11:58:21 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -5.273941882346851 (T) = (0 -14.377762172330666) / Math.Sqrt((8.566421319518579 / (32)) + (7.422701539949364 / (10))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (10) - 2, .025) and -0.58387389484387 = (9.077592742159533 - 14.377762172330666) / 9.077592742159533 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository