[Perf] Linux/arm64: 5 Improvements on 7/13/2023 7:10:54 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	d3d910ea88b00c7af2f69df07cbfbbe86da56596
Compare	c88dbd3f943b8206b4821aaf10ddf3c216100942
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsFalse<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ImmutableList - Duration of single invocation 📝 - Benchmark Source	1.36 ms	1.06 ms	0.78	0.23	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse<Int32>*'

Payloads

Baseline
Compare

System.Collections.ContainsFalse<Int32>.ImmutableList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0642278241666665 < 1.2755322944357639.
IsChangePoint: Marked as a change because one of 4/11/2023 5:52:15 AM, 5/19/2023 4:22:12 AM, 7/12/2023 7:14:29 PM, 7/18/2023 4:12:30 PM falls between 7/10/2023 3:14:43 AM and 7/18/2023 4:12:30 PM.
IsImprovementStdDev: Marked as improvement because 69.26217880729591 (T) = (0 -1064004.881440476) / Math.Sqrt((8545362964.125846 / (299)) + (210502.49598972528 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.25830251691175027 = (1434553.7172516966 - 1064004.881440476) / 1434553.7172516966 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	d3d910ea88b00c7af2f69df07cbfbbe86da56596
Compare	c88dbd3f943b8206b4821aaf10ddf3c216100942
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsTrue<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ImmutableList - Duration of single invocation 📝 - Benchmark Source	682.49 μs	524.53 μs	0.77	0.27	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue<Int32>*'

Payloads

Baseline
Compare

System.Collections.ContainsTrue<Int32>.ImmutableList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 524.5341230555555 < 638.2135721842918.
IsChangePoint: Marked as a change because one of 4/11/2023 5:52:15 AM, 5/19/2023 4:22:12 AM, 7/12/2023 7:14:29 PM, 7/18/2023 4:12:30 PM falls between 7/10/2023 3:14:43 AM and 7/18/2023 4:12:30 PM.
IsImprovementStdDev: Marked as improvement because 52.46308943564502 (T) = (0 -525164.9617413003) / Math.Sqrt((4315839688.018377 / (299)) + (1103046.285013192 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.2766346577873876 = (726002.3823299834 - 525164.9617413003) / 726002.3823299834 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	d3d910ea88b00c7af2f69df07cbfbbe86da56596
Compare	c88dbd3f943b8206b4821aaf10ddf3c216100942
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Net.Primitives.Tests.CredentialCacheTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
GetCredential_HostPort - Duration of single invocation 📝 - Benchmark Source	28.15 ns	19.44 ns	0.69	0.57	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Primitives.Tests.CredentialCacheTests*'

Payloads

Baseline
Compare

System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_HostPort(host: "notfound", hostPortCount: 0)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.444516008780745 < 21.791835942566852.
IsChangePoint: Marked as a change because one of 5/9/2023 11:02:55 PM, 7/12/2023 7:14:29 PM, 7/18/2023 4:12:30 PM falls between 7/10/2023 3:14:43 AM and 7/18/2023 4:12:30 PM.
IsImprovementStdDev: Marked as improvement because 7.198673239606639 (T) = (0 -17.861274013867522) / Math.Sqrt((4.307016913460749 / (299)) + (2.2338693921213486 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.21488569265426632 = (22.74990258966471 - 17.861274013867522) / 22.74990258966471 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	d3d910ea88b00c7af2f69df07cbfbbe86da56596
Compare	c88dbd3f943b8206b4821aaf10ddf3c216100942
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_TimeSpan

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
FromSeconds - Duration of single invocation 📝 - Benchmark Source	1.47 ns	0.01 ns	0.01	0.42	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_TimeSpan*'

Payloads

Baseline
Compare

System.Tests.Perf_TimeSpan.FromSeconds

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.01221329277590462 < 1.426116986596517.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/12/2023 7:14:29 PM, 7/18/2023 4:12:30 PM falls between 7/10/2023 3:14:43 AM and 7/18/2023 4:12:30 PM.
IsImprovementStdDev: Marked as improvement because 6.09708372630424 (T) = (0 -0.02446903194751749) / Math.Sqrt((0.2756808968659237 / (299)) + (0.0008366898945599469 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.8915883931831897 = (0.2257049098890702 - 0.02446903194751749) / 0.2257049098890702 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	d3d910ea88b00c7af2f69df07cbfbbe86da56596
Compare	c88dbd3f943b8206b4821aaf10ddf3c216100942
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.EightQueens

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Test - Duration of single invocation 📝 - Benchmark Source	2.46 μs	2.31 μs	0.94	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.EightQueens*'

Payloads

Baseline
Compare

Benchstone.BenchI.EightQueens.Test

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.308366332431887 < 2.3382929331715054.
IsChangePoint: Marked as a change because one of 4/4/2023 10:11:10 PM, 5/19/2023 4:22:12 AM, 7/12/2023 7:14:29 PM, 7/18/2023 4:12:30 PM falls between 7/10/2023 3:14:43 AM and 7/18/2023 4:12:30 PM.
IsImprovementStdDev: Marked as improvement because 44.386905673190284 (T) = (0 -2308.007175342055) / Math.Sqrt((2856.7006294682847 / (299)) + (57.069991013900456 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.08093636159067812 = (2511.259371915377 - 2308.007175342055) / 2511.259371915377 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[cincuranet]

Improvements in Benchstone.BenchI.EightQueens and System.Collections.ContainsFalse<Int32> and System.Collections.ContainsTrue<Int32> is dotnet/runtime#88749.