[Perf] Changes at 6/2/2022 9:03:53 PM

[performanceautofiler]

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare	fe2c1ed058facc3c29e75c9892888fa521be586b
Diff	Diff

Regressions in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Utf8JsonReaderCommentParsing - Duration of single invocation	9.23 μs	14.88 μs	1.61	0.01	False
Utf8JsonReaderCommentParsing - Duration of single invocation	9.49 μs	14.87 μs	1.57	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongMultiLine)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.883785569105692 > 9.699163294232083.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -282.2660900046652 (T) = (0 -14688.917905944649) / Math.Sqrt((2561.1792573839366 / (41)) + (9249.453085919857 / (30))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (30) - 2, .025) and -0.5873595947513897 = (9253.680107842993 - 14688.917905944649) / 9253.680107842993 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongMultiLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.871529360557142 > 10.00784566519428.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -81.75241346738288 (T) = (0 -14733.622294964356) / Math.Sqrt((186673.2365412168 / (41)) + (6480.362748494429 / (30))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (30) - 2, .025) and -0.6212247500295154 = (9087.95791249555 - 14733.622294964356) / 9087.95791249555 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare	fe2c1ed058facc3c29e75c9892888fa521be586b
Diff	Diff

Regressions in Span.IndexerBench

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
KnownSizeCtor2 - Duration of single invocation	563.55 ns	652.32 ns	1.16	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Span.IndexerBench*'

Payloads

Baseline
Compare

Histogram

Span.IndexerBench.KnownSizeCtor2(length: 1024)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 652.3151675688298 > 591.9468472584205.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -103.12788399334563 (T) = (0 -645.3930375004572) / Math.Sqrt((14.450933658091463 / (40)) + (6.301587453208653 / (28))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (28) - 2, .025) and -0.13941329246904083 = (566.4257576826482 - 645.3930375004572) / 566.4257576826482 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	f873951b78d0a8123f4cd2e28cba08f08d67fbc4
Compare	fe2c1ed058facc3c29e75c9892888fa521be586b
Diff	Diff

Regressions in System.Collections.CtorFromCollection<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedSet - Duration of single invocation	9.88 μs	11.39 μs	1.15	0.29	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollection<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<Int32>.SortedSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.393505949279136 > 11.318675689845474.
IsChangePoint: Marked as a change because one of 4/29/2022 11:03:27 PM, 6/2/2022 5:58:32 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -9.622352773228338 (T) = (0 -16785.977365476952) / Math.Sqrt((4933788.381153581 / (39)) + (5838276.363393015 / (30))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (30) - 2, .025) and -0.48112469304068267 = (11333.264136604254 - 16785.977365476952) / 11333.264136604254 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare	fe2c1ed058facc3c29e75c9892888fa521be586b
Diff	Diff

Regressions in System.Memory.Span<Char>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IndexOfAnyThreeValues - Duration of single invocation	20.54 ns	23.27 ns	1.13	0.11	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span<Char>*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.IndexOfAnyThreeValues(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.268375721209694 > 21.991525205470694.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/25/2022 5:27:54 PM, 5/20/2022 10:18:01 PM, 5/24/2022 7:55:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -16.48597773436918 (T) = (0 -23.427842435046234) / Math.Sqrt((0.2433845403497706 / (41)) + (0.4560451100054248 / (27))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (27) - 2, .025) and -0.11896558844822472 = (20.937053540257512 - 23.427842435046234) / 20.937053540257512 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare	fe2c1ed058facc3c29e75c9892888fa521be586b
Diff	Diff

Regressions in PerfLabTests.CastingPerf2.CastingPerf

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ObjrefValueTypeObj - Duration of single invocation	810.79 μs	1.02 ms	1.26	0.47	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf2.CastingPerf.ObjrefValueTypeObj

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.02430203125 > 851.326065234375.
IsChangePoint: Marked as a change because one of 4/22/2022 11:35:13 AM, 4/28/2022 4:25:37 PM, 5/28/2022 3:04:49 PM, 6/2/2022 5:58:32 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -10.398200730366728 (T) = (0 -1037509.3127000432) / Math.Sqrt((5208346517.561143 / (38)) + (1536700570.4434202 / (30))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (30) - 2, .025) and -0.15945071284051648 = (894828.3020657848 - 1037509.3127000432) / 894828.3020657848 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[AndyAyersMS]

These regressed with dotnet/runtime#70144, which undid improvements we saw with dotnet/runtime#68869.

[AndyAyersMS]

@kunalspathak you might find this interesting to investigate -- alignment made perf worse.

[AndyAyersMS]

Also look at related:

• [Perf] Changes at 6/2/2022 9:03:53 PM #5835
• [Perf] Changes at 5/30/2022 12:57:05 PM #5867

[kunalspathak]

From https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu%2018.04/Benchstone.BenchI.LogicArray.Test.html , this one is interesting. Will take a look.

[AndyAyersMS]

@kunalspathak to follow up

[kunalspathak]

For LogicArray2, there are 2 loops that are alignment candidate. We end up aligning the first of these 2 loops by adding padding of 7 bytes.

For the 2nd loop, we do not add any alignment because it doesn't match the heuristics, but then we end up having a JCC erratum in the middle of hot loop and that might be adding to the regression after loop alignment.