[Perf] Linux/arm64: 7 Improvements on 10/7/2023 2:03:18 PM #23004

performanceautofiler · 2023-10-12T07:50:17Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f55549222e4dc1469190caa56e03361ee46208ca
Compare	551a4db7917df8c321867d47ce028c726106a3c2
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Array

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Reverse - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	52.39 ns	33.24 ns	0.63	0.08	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Array*'

Payloads

Baseline
Compare

System.Tests.Perf_Array.Reverse

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 33.23789109788687 < 49.81174770570195.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/28/2023 1:33:49 PM, 5/3/2023 7:05:50 AM, 7/1/2023 8:07:46 AM, 8/4/2023 10:42:53 AM, 8/22/2023 6:29:15 PM, 10/6/2023 9:14:38 AM, 10/11/2023 11:44:03 AM falls between 10/2/2023 7:07:56 PM and 10/11/2023 11:44:03 AM.
IsImprovementStdDev: Marked as improvement because 16.28841598256251 (T) = (0 -33.211229341923705) / Math.Sqrt((96.1287269536155 / (299)) + (0.004414910489345148 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.21777695650926732 = (42.45749293413289 - 33.211229341923705) / 42.45749293413289 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f55549222e4dc1469190caa56e03361ee46208ca
Compare	551a4db7917df8c321867d47ce028c726106a3c2
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Perf_Ascii

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ToUpperInPlace_Chars - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	29.47 ns	26.51 ns	0.90	0.10	False
ToLower_Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	18.41 ns	16.59 ns	0.90	0.27	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Ascii*'

Payloads

Baseline
Compare

System.Text.Perf_Ascii.ToUpperInPlace_Chars(Size: 128)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 26.5093775547953 < 27.962108138980597.
IsChangePoint: Marked as a change because one of 7/1/2023 8:07:46 AM, 8/4/2023 10:42:53 AM, 8/22/2023 6:29:15 PM, 10/6/2023 9:14:38 AM, 10/11/2023 11:44:03 AM falls between 10/2/2023 7:07:56 PM and 10/11/2023 11:44:03 AM.
IsImprovementStdDev: Marked as improvement because 14.553201367800575 (T) = (0 -26.4562348223384) / Math.Sqrt((2.5122232129851945 / (288)) + (0.005882642783073473 / (6))) is greater than 1.9681214070095314 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (288) + (6) - 2, .975) and 0.051401494873097586 = (27.88981289697385 - 26.4562348223384) / 27.88981289697385 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.ToLower_Bytes(Size: 128)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.587212093370603 < 17.49654481053279.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/1/2023 8:07:46 AM, 8/4/2023 10:42:53 AM, 8/22/2023 6:29:15 PM, 10/6/2023 9:14:38 AM, 10/11/2023 11:44:03 AM falls between 10/2/2023 7:07:56 PM and 10/11/2023 11:44:03 AM.
IsImprovementStdDev: Marked as improvement because 15.139438584730751 (T) = (0 -16.648771140848805) / Math.Sqrt((1.1771864996634787 / (289)) + (0.00852384369324682 / (6))) is greater than 1.9680934526838334 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (289) + (6) - 2, .975) and 0.06314545793117637 = (17.7709242932034 - 16.648771140848805) / 17.7709242932034 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f55549222e4dc1469190caa56e03361ee46208ca
Compare	551a4db7917df8c321867d47ce028c726106a3c2
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Perf_Frozen<Int16>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Contains_True - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	19.68 ns	17.96 ns	0.91	0.02	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Perf_Frozen&lt;Int16&gt;*'

Payloads

Baseline
Compare

System.Collections.Perf_Frozen<Int16>.Contains_True(Count: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 17.958181584746878 < 18.66087806450823.
IsChangePoint: Marked as a change because one of 10/6/2023 9:14:38 AM, 10/11/2023 11:44:03 AM falls between 10/2/2023 7:07:56 PM and 10/11/2023 11:44:03 AM.
IsImprovementStdDev: Marked as improvement because 63.9359750631175 (T) = (0 -17.98846508027604) / Math.Sqrt((0.008342918982363652 / (52)) + (0.003109698685216539 / (6))) is greater than 2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (6) - 2, .975) and 0.08474960272098868 = (19.65414615907816 - 17.98846508027604) / 19.65414615907816 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f55549222e4dc1469190caa56e03361ee46208ca
Compare	551a4db7917df8c321867d47ce028c726106a3c2
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in ByteMark

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
BenchNeural - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	851.72 ms	780.34 ms	0.92	0.01	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'ByteMark*'

Payloads

Baseline
Compare

ByteMark.BenchNeural

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 9/28/2023 11:34:37 PM, 10/6/2023 9:14:38 AM, 10/11/2023 11:44:03 AM falls between 10/2/2023 7:07:56 PM and 10/11/2023 11:44:03 AM.
IsImprovementStdDev: Marked as improvement because 155.88125116996326 (T) = (0 -781046880.9928573) / Math.Sqrt((100700414826102.16 / (299)) + (551796739352.4241 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.11557956693067338 = (883117182.4946222 - 781046880.9928573) / 883117182.4946222 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f55549222e4dc1469190caa56e03361ee46208ca
Compare	551a4db7917df8c321867d47ce028c726106a3c2
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Memory.Span<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Reverse - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	25.05 ns	17.17 ns	0.69	0.30	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

System.Memory.Span<Byte>.Reverse(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 17.165486728008027 < 26.227219641447764.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 7/1/2023 8:07:46 AM, 8/4/2023 10:42:53 AM, 8/22/2023 6:29:15 PM, 10/6/2023 9:14:38 AM, 10/11/2023 11:44:03 AM falls between 10/2/2023 7:07:56 PM and 10/11/2023 11:44:03 AM.
IsImprovementStdDev: Marked as improvement because 15.330826529535166 (T) = (0 -17.181996290548142) / Math.Sqrt((31.931597433424134 / (299)) + (0.004726592000372342 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.22640105365573762 = (22.210470130219015 - 17.181996290548142) / 22.210470130219015 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f55549222e4dc1469190caa56e03361ee46208ca
Compare	551a4db7917df8c321867d47ce028c726106a3c2
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Count - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	326.35 μs	307.89 μs	0.94	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: Compiled)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 307.893476388889 < 310.2073483658854.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 10/6/2023 9:14:38 AM, 10/11/2023 11:44:03 AM falls between 10/2/2023 7:07:56 PM and 10/11/2023 11:44:03 AM.
IsImprovementStdDev: Marked as improvement because 49.55729001229812 (T) = (0 -308188.94811784755) / Math.Sqrt((19328961.307514444 / (299)) + (326626.35812799324 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.05257283012916275 = (325290.3842306559 - 308188.94811784755) / 325290.3842306559 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

performanceautofiler bot added ampere arch-arm64 branch-refs/heads/main kind-micro os-linux perf-improvement runtime-coreclr untriaged labels Oct 12, 2023

performanceautofiler bot mentioned this issue Oct 12, 2023

[SENTINEL] Autofile run complete at 10/12/2023 7:50:22 AM. 6 issues filed. #23006

Closed

EgorBo removed the untriaged label Oct 12, 2023

EgorBo closed this as completed Oct 12, 2023

EgorBo mentioned this issue Oct 12, 2023

[Perf] Linux/arm64: 8 Regressions on 10/7/2023 2:03:18 PM dotnet/runtime#93401

Closed

[Perf] Linux/arm64: 7 Improvements on 10/7/2023 2:03:18 PM #23004

[Perf] Linux/arm64: 7 Improvements on 10/7/2023 2:03:18 PM #23004

Comments

performanceautofiler bot commented Oct 12, 2023 • edited Loading

Run Information

Improvements in System.Tests.Perf_Array

Repro

Payloads

System.Tests.Perf_Array.Reverse

ETL Files

Histogram

Description of detection logic

JIT Disasms

Docs

Run Information

Improvements in System.Text.Perf_Ascii

Repro

Payloads

System.Text.Perf_Ascii.ToUpperInPlace_Chars(Size: 128)

ETL Files

Histogram

Description of detection logic

JIT Disasms

System.Text.Perf_Ascii.ToLower_Bytes(Size: 128)

ETL Files

Histogram

Description of detection logic

JIT Disasms

Docs

Run Information

Improvements in System.Collections.Perf_Frozen<Int16>

Repro

Payloads

System.Collections.Perf_Frozen<Int16>.Contains_True(Count: 4)

ETL Files

Histogram

Description of detection logic

JIT Disasms

Docs

Run Information

Improvements in ByteMark

Repro

Payloads

ByteMark.BenchNeural

ETL Files

Histogram

Description of detection logic

JIT Disasms

Docs

Run Information

Improvements in System.Memory.Span<Byte>

Repro

Payloads

System.Memory.Span<Byte>.Reverse(Size: 512)

ETL Files

Histogram

Description of detection logic

JIT Disasms

Docs

Run Information

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Repro

Payloads

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: Compiled)

ETL Files

Histogram

Description of detection logic

JIT Disasms

Docs

performanceautofiler bot commented Oct 12, 2023 •

edited

Loading