[Perf] Windows/x64: 8 Improvements on 7/26/2023 8:42:30 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	8f5364695b107f37af899aba3ad1b9f98d577c33
Compare	bc23bf4ee16cbd1999105b1916e1f2f76baef6a7
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsFalse<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
LinkedList - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	1.30 ms	847.22 μs	0.65	0.07	False	14431490.384615384	4170515.046296296	0.28898713404834836)

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse<String>*'

Payloads

Baseline
Compare

System.Collections.ContainsFalse<String>.LinkedList(Size: 512)

ETL Files

Baseline
Compare

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 14.102772859803416 (T) = (0 -909230.6425438597) / Math.Sqrt((5703088272.316532 / (299)) + (3056614491.7718887 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.33320424313185676 = (1363581.9262174116 - 909230.6425438597) / 1363581.9262174116 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	8f5364695b107f37af899aba3ad1b9f98d577c33
Compare	bc23bf4ee16cbd1999105b1916e1f2f76baef6a7
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsTrue<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
LinkedList - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	646.86 μs	341.85 μs	0.53	0.06	False
ICollection - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	465.07 μs	380.63 μs	0.82	0.31	False	5064828.431372549	3989939.024390244	0.7877737772272349)
List - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	469.96 μs	379.35 μs	0.81	0.29	False	4930269.607843137	3985365.8536585364	0.8083464334929198)

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue<String>*'

Payloads

Baseline
Compare

System.Collections.ContainsTrue<String>.LinkedList(Size: 512)

ETL Files

Baseline
Compare

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 6.682852975219135 (T) = (0 -430099.9712998102) / Math.Sqrt((1510628508.4353876 / (299)) + (5611587026.255397 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.3682973840922415 = (680858.3033675669 - 430099.9712998102) / 680858.3033675669 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Collections.ContainsTrue<String>.ICollection(Size: 512)

ETL Files

Baseline
Compare

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/2/2023 3:18:41 PM, 5/8/2023 11:50:12 PM, 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 46.73002853406124 (T) = (0 -381656.41193621757) / Math.Sqrt((1483882675.4461195 / (299)) + (645110.2745741954 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.2170133551528324 = (487436.6816444407 - 381656.41193621757) / 487436.6816444407 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Collections.ContainsTrue<String>.List(Size: 512)

ETL Files

Baseline
Compare

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/20/2023 3:57:43 PM, 7/26/2023 12:02:55 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 49.02263383896598 (T) = (0 -378862.0748299319) / Math.Sqrt((1393587890.2252169 / (299)) + (542302.597142735 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.22161703129505506 = (486729.6563030838 - 378862.0748299319) / 486729.6563030838 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	8f5364695b107f37af899aba3ad1b9f98d577c33
Compare	bc23bf4ee16cbd1999105b1916e1f2f76baef6a7
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SerializeToString - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	27.53 μs	17.07 μs	0.62	0.10	False
SerializeToUtf8Bytes - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	21.83 μs	16.11 μs	0.74	0.08	False	186179.92992992993	151509.0811965812	0.8137777323989898)
SerializeToStream - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	22.22 μs	16.74 μs	0.75	0.09	False	186002.21893491125	152183.1735889244	0.8181793446355534)

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToString(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 61.6026785778903 (T) = (0 -17163.8784852896) / Math.Sqrt((2706088.9831080446 / (299)) + (7159.892059686274 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.27203194138639775 = (23577.790649191113 - 17163.8784852896) / 23577.790649191113 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToUtf8Bytes(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 43.69757712420155 (T) = (0 -16334.760052486203) / Math.Sqrt((2768246.9959000223 / (299)) + (54818.39297125186 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.28844683942321453 = (22956.485836202646 - 16334.760052486203) / 22956.485836202646 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToStream(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 64.1860646458521 (T) = (0 -16702.012425888588) / Math.Sqrt((2591709.309662356 / (299)) + (1250.2878402458773 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2680918030972151 = (22819.81879225629 - 16702.012425888588) / 22819.81879225629 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	8f5364695b107f37af899aba3ad1b9f98d577c33
Compare	bc23bf4ee16cbd1999105b1916e1f2f76baef6a7
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CreateAddAndRemove<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
List - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	45.40 μs	43.06 μs	0.95	0.02	False	486214.14441147377	468651.9607843137	0.963879735238024)

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove<String>*'

Payloads

Baseline
Compare

System.Collections.CreateAddAndRemove<String>.List(Size: 512)

ETL Files

Baseline
Compare

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 6/27/2023 3:20:01 PM, 6/28/2023 2:04:56 PM, 7/3/2023 3:46:39 PM, 7/5/2023 3:16:33 PM, 7/26/2023 12:02:55 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 22.616290104077788 (T) = (0 -42714.386861755374) / Math.Sqrt((1680233.3537578683 / (299)) + (71451.11042241787 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.07504759396304289 = (46180.09162738336 - 42714.386861755374) / 46180.09162738336 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository