Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/arm64: 49 Improvements on 6/29/2023 8:27:03 AM #19587

Closed
performanceautofiler bot opened this issue Jul 6, 2023 · 2 comments
Closed

[Perf] Linux/arm64: 49 Improvements on 6/29/2023 8:27:03 AM #19587

performanceautofiler bot opened this issue Jul 6, 2023 · 2 comments
Labels
ampere arch-arm64 branch-refs/heads/main kind-micro os-linux perf-improvement runtime-coreclr

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Jul 6, 2023 •
edited
Loading

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Guid

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
GuidToString - Duration of single invocation 38.38 ns 21.16 ns 0.55 0.33 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Guid*'

Payloads

Baseline
Compare

System.Tests.Perf_Guid.GuidToString

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.164160526024023 < 36.499539982447914.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 20.371981144127183 (T) = (0 -21.16773824845084) / Math.Sqrt((27.383449548939417 / (299)) + (0.8970607255522256 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2783272901129322 = (29.33149329113929 - 21.16773824845084) / 29.33149329113929 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
Enumerate - Duration of single invocation 294.89 ns 162.35 ns 0.55 0.02 True
Enumerate - Duration of single invocation 43.16 ns 17.98 ns 0.42 0.08 True

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Guid, Guid&gt;*'

Payloads

Baseline
Compare

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Enumerate(Size: 100)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 162.3528974848099 < 278.40317526176784.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 64.79431440176215 (T) = (0 -162.66068161221958) / Math.Sqrt((1067.2417325608753 / (299)) + (4.854128653082194 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.44164036723489536 = (291.31884195620034 - 162.66068161221958) / 291.31884195620034 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Enumerate(Size: 10)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 17.98057055402939 < 41.03266236551283.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 8.135325198940713 (T) = (0 -21.810067506858754) / Math.Sqrt((11.970350089206132 / (299)) + (79.2324209603121 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4802192621998911 = (41.96013034104817 - 21.810067506858754) / 41.96013034104817 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
IterateForEachArray - Duration of single invocation 17.33 ns 4.73 ns 0.27 0.12 True
IterateForEachTenSegments - Duration of single invocation 69.18 ns 47.39 ns 0.69 0.02 True
IterateForEachSingleSegment - Duration of single invocation 24.19 ns 10.39 ns 0.43 0.04 True
IterateForEachMemory - Duration of single invocation 29.22 ns 19.33 ns 0.66 0.39 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;*'

Payloads

Baseline
Compare

System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateForEachArray

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.726460174061214 < 16.617630919893617.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 422.26601069419496 (T) = (0 -4.656651352996352) / Math.Sqrt((0.22908110386055186 / (299)) + (0.0027382636180881856 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7391819528257508 = (17.854022769694705 - 4.656651352996352) / 17.854022769694705 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateForEachTenSegments

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 47.39114875395558 < 64.11722901336069.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 36.64644226629069 (T) = (0 -46.592283499362495) / Math.Sqrt((1.5565882804277735 / (299)) + (4.79036987311246 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.32469570424869726 = (68.99450187493142 - 46.592283499362495) / 68.99450187493142 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateForEachSingleSegment

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.387676395875422 < 22.81533878340542.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 242.26623925917195 (T) = (0 -10.59233048852666) / Math.Sqrt((0.41497481226572697 / (299)) + (0.01828393576157915 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5473145138775712 = (23.398873640189926 - 10.59233048852666) / 23.398873640189926 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateForEachMemory

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.332638454163806 < 30.520985374539993.
IsChangePoint: Marked as a change because one of 4/9/2023 11:24:15 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 48.346302211034185 (T) = (0 -15.262344012389443) / Math.Sqrt((7.072165910474404 / (299)) + (1.5444840572586356 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5445444096011574 = (33.51005967239133 - 15.262344012389443) / 33.51005967239133 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Binary.Tests.BinaryReadAndWriteTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
ReadStructAndReverseBE - Duration of single invocation 11.12 ns 1.82 ns 0.16 0.23 True
ReadStructFieldByFieldUsingBitConverterLE - Duration of single invocation 8.76 ns 1.93 ns 0.22 0.35 True
ReadStructFieldByFieldUsingBitConverterBE - Duration of single invocation 18.09 ns 2.13 ns 0.12 0.14 True
ReadStructFieldByFieldBE - Duration of single invocation 9.01 ns 5.89 ns 0.65 0.41 False
ReadStructFieldByFieldLE - Duration of single invocation 9.15 ns 5.32 ns 0.58 0.31 True

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Binary.Tests.BinaryReadAndWriteTests*'

Payloads

Baseline
Compare

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructAndReverseBE

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.8238390012589043 < 11.011082214402222.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 113.53978440526505 (T) = (0 -2.429658571280879) / Math.Sqrt((0.17864075078957542 / (299)) + (0.07809191505732772 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7915679916695139 = (11.656839996611533 - 2.429658571280879) / 11.656839996611533 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldUsingBitConverterLE

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.9267575116476343 < 8.537915763616553.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 92.28637060645275 (T) = (0 -1.9041677289769827) / Math.Sqrt((0.21444198876151996 / (299)) + (0.06559029581381513 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7862837592095383 = (8.909794229648302 - 1.9041677289769827) / 8.909794229648302 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldUsingBitConverterBE

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.1332528244143134 < 17.355999833040364.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 203.57637293376578 (T) = (0 -2.0383873453740136) / Math.Sqrt((1.0158377971978847 / (299)) + (0.04066272283584183 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.8897168490990426 = (18.483216418114832 - 2.0383873453740136) / 18.483216418114832 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldBE

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.886265079270616 < 8.387061174979525.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 39.26949896593799 (T) = (0 -5.936906531550207) / Math.Sqrt((0.3103882325095821 / (299)) + (0.05926678461907569 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.33103757279805013 = (8.87479818019427 - 5.936906531550207) / 8.87479818019427 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldLE

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.323512341990691 < 8.355354323001189.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 30.065627011269456 (T) = (0 -5.715554141206766) / Math.Sqrt((0.19323592754286922 / (299)) + (0.1443437890171988 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.36313687413548057 = (8.97454085357525 - 5.715554141206766) / 8.97454085357525 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Document.Tests.Perf_EnumerateObject

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
EnumerateProperties - Duration of single invocation 1.30 μs 706.82 ns 0.54 0.00 True
EnumerateProperties - Duration of single invocation 2.30 μs 1.30 μs 0.57 0.01 True
EnumerateProperties - Duration of single invocation 1.30 μs 706.53 ns 0.54 0.00 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateObject*'

Payloads

Baseline
Compare

System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: NumericProperties)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 706.8178703297953 < 1.2345553313757465.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 638.9221938280333 (T) = (0 -707.7282333462473) / Math.Sqrt((242.9481094183197 / (299)) + (0.321182534187444 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4523701985498161 = (1292.3479172829987 - 707.7282333462473) / 1292.3479172829987 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: ObjectProperties)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3018600067422916 < 2.183620923128144.
IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 196.38829341563343 (T) = (0 -1301.4255267230278) / Math.Sqrt((6725.969558477173 / (299)) + (3.062820674735137 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.41841867533016597 = (2237.736102447671 - 1301.4255267230278) / 2237.736102447671 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: StringProperties)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 706.5322187043392 < 1.235752596686803.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 664.9773687861947 (T) = (0 -707.5979909515504) / Math.Sqrt((222.9150124373356 / (299)) + (0.3685640137651414 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4525709510697263 = (1292.583929066719 - 707.5979909515504) / 1292.583929066719 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
IterateForEachTenSegments - Duration of single invocation 69.70 ns 45.37 ns 0.65 0.03 True
IterateGetPositionMemory - Duration of single invocation 31.76 ns 27.37 ns 0.86 0.37 False
IterateForEachMemory - Duration of single invocation 30.30 ns 14.49 ns 0.48 0.47 False
IterateForEachArray - Duration of single invocation 16.74 ns 5.24 ns 0.31 0.18 True
IterateForEachSingleSegment - Duration of single invocation 24.08 ns 10.54 ns 0.44 0.07 True

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachTenSegments

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 45.37023310536621 < 65.69228500037413.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 208.2610712548734 (T) = (0 -45.23438574658516) / Math.Sqrt((1.6874736898605143 / (299)) + (0.10442247392954851 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.34998313010646953 = (69.58955658180737 - 45.23438574658516) / 69.58955658180737 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateGetPositionMemory

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 27.36634231105157 < 30.9423074121565.
IsChangePoint: Marked as a change because one of 4/28/2023 10:02:43 PM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 31.371216242831682 (T) = (0 -27.379791326786307) / Math.Sqrt((9.814479308297178 / (299)) + (0.16595962997405925 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19655875189821187 = (34.078149947458954 - 27.379791326786307) / 34.078149947458954 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachMemory

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.486798119937893 < 28.250639234407423.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 80.58752189103146 (T) = (0 -14.50773606208521) / Math.Sqrt((9.067828762921291 / (299)) + (0.2596793318718425 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5547327342911166 = (32.58208536616387 - 14.50773606208521) / 32.58208536616387 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachArray

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.242900143811187 < 16.22287694875414.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 149.01794226159242 (T) = (0 -5.079773965874079) / Math.Sqrt((1.0295783476527356 / (299)) + (0.04604912599931054 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7103019668335125 = (17.53472024076452 - 5.079773965874079) / 17.53472024076452 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachSingleSegment

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.54251827455374 < 22.689202804370403.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 235.79854133271184 (T) = (0 -10.452394308402774) / Math.Sqrt((0.4908363954267743 / (299)) + (0.016714398926590168 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5496637299419125 = (23.210198696752872 - 10.452394308402774) / 23.210198696752872 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
Enumerate - Duration of single invocation 241.55 ns 118.64 ns 0.49 0.00 True
Enumerate - Duration of single invocation 33.89 ns 14.55 ns 0.43 0.08 True

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Enumerate(Size: 100)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 118.64196054480946 < 230.90995927842906.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 510.05271694517245 (T) = (0 -119.41618338327262) / Math.Sqrt((8.985698362899361 / (299)) + (0.33632521896532913 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5025038818284261 = (240.03440232289205 - 119.41618338327262) / 240.03440232289205 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Enumerate(Size: 10)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.548790462287723 < 33.56658911351127.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 208.18700848051893 (T) = (0 -14.772665400491018) / Math.Sqrt((0.6840387646109801 / (299)) + (0.09096530645538403 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5759050223180624 = (34.83338916493892 - 14.772665400491018) / 34.83338916493892 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 6, 2023 •
edited
Loading

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
Reverse - Duration of single invocation 120.04 ns 66.05 ns 0.55 0.34 False
ToArray - Duration of single invocation 179.56 ns 160.59 ns 0.89 0.04 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Memory.Span<Int32>.Reverse(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 66.04626596081187 < 107.93860801329409.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 15.219717426939692 (T) = (0 -65.9684569567826) / Math.Sqrt((597.699638705291 / (299)) + (0.09164982311274177 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24628920693997786 = (87.52489358544872 - 65.9684569567826) / 87.52489358544872 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.ToArray(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 160.58827169240698 < 169.8645033552437.
IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 4/28/2023 1:33:49 PM, 5/3/2023 7:05:50 AM, 7/1/2023 8:07:46 AM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 7.571324239845559 (T) = (0 -164.73288265345977) / Math.Sqrt((21.257042128190268 / (299)) + (15.270636100175494 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.06453042553733868 = (176.09646230138827 - 164.73288265345977) / 176.09646230138827 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_Matrix3x2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
CreateFromScalars - Duration of single invocation 3.01 ns 0.97 ns 0.32 0.49 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_Matrix3x2.CreateFromScalars

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9679722659503709 < 2.745578043936771.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 30.408099683019504 (T) = (0 -1.1842351803903939) / Math.Sqrt((0.11224105695068372 / (299)) + (0.04012912799777457 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.601733364130143 = (2.9734732305755345 - 1.1842351803903939) / 2.9734732305755345 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
Reverse - Duration of single invocation 53.90 ns 38.96 ns 0.72 0.23 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

System.Memory.Span<Char>.Reverse(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 38.95684813082677 < 51.150559372128434.
IsChangePoint: Marked as a change because one of 5/6/2023 11:12:53 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 5.9625035201188314 (T) = (0 -39.6788047990728) / Math.Sqrt((138.1455577014525 / (299)) + (18.648801761046965 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17146088402345389 = (47.890080303940664 - 39.6788047990728) / 47.890080303940664 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_DateTimeOffset

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
Parse - Duration of single invocation 498.38 ns 437.36 ns 0.88 0.01 False
TryParse - Duration of single invocation 504.80 ns 441.89 ns 0.88 0.01 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTimeOffset*'

Payloads

Baseline
Compare

System.Tests.Perf_DateTimeOffset.Parse(value: "12/30/2017 3:45:22 AM -08:00")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 437.35699336270767 < 473.95771571361695.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 63.356134615879846 (T) = (0 -434.76245544905976) / Math.Sqrt((687.8133585479715 / (299)) + (4.005844358218437 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19051977230746295 = (537.0884186861134 - 434.76245544905976) / 537.0884186861134 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_DateTimeOffset.TryParse(value: "12/30/2017 3:45:22 AM -08:00")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 441.8940547621104 < 477.1286205960245.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 34.13715277142889 (T) = (0 -440.85316859475125) / Math.Sqrt((603.5923683298038 / (299)) + (73.29728367830184 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17645994400263876 = (535.3147857038344 - 440.85316859475125) / 535.3147857038344 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
MatchesBoundary - Duration of single invocation 143.46 μs 130.39 μs 0.91 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesBoundary(Options: None)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 130.3858507595346 < 136.67205647685486.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 61.27516867815474 (T) = (0 -129975.33125090749) / Math.Sqrt((44377907.41633725 / (299)) + (128930.72880882815 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1579608925077997 = (154357.8321890607 - 129975.33125090749) / 154357.8321890607 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Sort<BigStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
Array - Duration of single invocation 27.40 μs 8.63 μs 0.31 0.00 True
Array_Comparison - Duration of single invocation 35.79 μs 21.28 μs 0.59 0.01 False
LinqQuery - Duration of single invocation 42.46 μs 27.24 μs 0.64 0.09 False
LinqOrderByExtension - Duration of single invocation 45.49 μs 27.05 μs 0.59 0.09 False
List - Duration of single invocation 27.34 μs 8.50 μs 0.31 0.00 True
Array_ComparerClass - Duration of single invocation 34.82 μs 18.21 μs 0.52 0.04 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;BigStruct&gt;*'

Payloads

Baseline
Compare

System.Collections.Sort<BigStruct>.Array(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.625029466666668 < 26.138813626999998.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 544.9575060153081 (T) = (0 -8679.078872161172) / Math.Sqrt((153628.8712516976 / (299)) + (9665.661470990326 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6900614402007392 = (28002.57856841816 - 8679.078872161172) / 28002.57856841816 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<BigStruct>.Array_Comparison(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.27912165333333 < 34.013610603928576.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 113.08959650630682 (T) = (0 -21106.775483076923) / Math.Sqrt((8411755.62934494 / (299)) + (8181.12687435024 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4760785382014208 = (40286.14405414717 - 21106.775483076923) / 40286.14405414717 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<BigStruct>.LinqQuery(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 27.244050186666662 < 40.595918482857144.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 68.15102525916065 (T) = (0 -27592.01967896309) / Math.Sqrt((21240349.569379274 / (299)) + (974868.1436959282 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4855591509623152 = (53634.97033833304 - 27592.01967896309) / 53634.97033833304 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<BigStruct>.LinqOrderByExtension(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 27.048621906666668 < 40.934882218166656.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 60.85130671506484 (T) = (0 -27746.30211362637) / Math.Sqrt((20211244.847601064 / (299)) + (1499179.2185672226 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4839982190031076 = (53771.71772551202 - 27746.30211362637) / 53771.71772551202 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<BigStruct>.List(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.503188199999999 < 25.955122386999996.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 553.2372712884852 (T) = (0 -8532.717847218935) / Math.Sqrt((192134.38942853018 / (299)) + (7572.334950043846 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6941309037808981 = (27896.63275137391 - 8532.717847218935) / 27896.63275137391 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<BigStruct>.Array_ComparerClass(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.207122226666666 < 33.11791430035714.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 106.59567024202377 (T) = (0 -18211.433119487177) / Math.Sqrt((12740131.225147607 / (299)) + (13672.61159564521 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5501661045820453 = (40484.79517659817 - 18211.433119487177) / 40484.79517659817 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Document.Tests.Perf_EnumerateArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
EnumerateArray - Duration of single invocation 847.91 ns 707.86 ns 0.83 0.02 False
EnumerateArray - Duration of single invocation 1.74 μs 1.19 μs 0.69 0.01 True
EnumerateArray - Duration of single invocation 848.84 ns 707.98 ns 0.83 0.01 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*'

Payloads

Baseline
Compare

System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: ArrayOfStrings)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 707.8602545601914 < 808.4244421835035.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 88.83962304187058 (T) = (0 -707.7219556045516) / Math.Sqrt((1061.5650552640955 / (299)) + (0.3463756184128148 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19186265503072222 = (875.7446491121585 - 707.7219556045516) / 875.7446491121585 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: Json400KB)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1926060502633327 < 1.650471951814064.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 213.42856315876122 (T) = (0 -1193.676008742984) / Math.Sqrt((1022.3091019419078 / (299)) + (31.216922235526493 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.30136493075875215 = (1708.583008922491 - 1193.676008742984) / 1708.583008922491 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: ArrayOfNumbers)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 707.9807310366767 < 804.4995108460847.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 84.39353210139299 (T) = (0 -707.7113354616597) / Math.Sqrt((1176.45353747953 / (299)) + (0.3538199829710981 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19182568177565126 = (875.6914436684679 - 707.7113354616597) / 875.6914436684679 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Struct.SpanWrapper

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
WrapperSum - Duration of single invocation 19.38 μs 10.02 μs 0.52 0.01 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Struct.SpanWrapper*'

Payloads

Baseline
Compare

Struct.SpanWrapper.WrapperSum

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.015190384615385 < 18.413267126296088.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 1196.5806365074404 (T) = (0 -10015.922750753232) / Math.Sqrt((9222.921162656421 / (153)) + (1.3779040764310846 / (13))) is greater than 1.9745345758582833 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (153) + (13) - 2, .975) and 0.4814272727646064 = (19314.40321620837 - 10015.922750753232) / 19314.40321620837 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.MDBenchI.MDLogicArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
Test - Duration of single invocation 609.62 ms 542.86 ms 0.89 0.00 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchI.MDLogicArray*'

Payloads

Baseline
Compare

Benchstone.MDBenchI.MDLogicArray.Test

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 542.8618043571428 < 576.8739861482142.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 147.32731841735384 (T) = (0 -542905629.4041983) / Math.Sqrt((34961895256548.504 / (299)) + (1512997720356.4805 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11588753333644697 = (614068514.8949492 - 542905629.4041983) / 614068514.8949492 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Struct.FilteredSpanEnumerator

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
Sum - Duration of single invocation 22.87 μs 10.03 μs 0.44 0.01 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Struct.FilteredSpanEnumerator*'

Payloads

Baseline
Compare

Struct.FilteredSpanEnumerator.Sum

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.03174746183489 < 21.700902740407983.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 497.82891139000907 (T) = (0 -10030.201801592424) / Math.Sqrt((103059.43617631592 / (153)) + (0.9828873899740459 / (13))) is greater than 1.9745345758582833 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (153) + (13) - 2, .975) and 0.5629807521051884 = (22951.3959623276 - 10030.201801592424) / 22951.3959623276 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 6, 2023 •
edited
Loading

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
LinkedList - Duration of single invocation 1.81 μs 693.57 ns 0.38 0.38 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Collections.IterateForEach<Int32>.LinkedList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 693.5672298670175 < 1.682888876641811.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 68.82162463888159 (T) = (0 -709.2524112341222) / Math.Sqrt((169930.5017064201 / (299)) + (47.74441801005995 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6988605658269123 = (2355.2292750422744 - 709.2524112341222) / 2355.2292750422744 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in LinqBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
Where00ForX - Duration of single invocation 290.22 ms 217.65 ms 0.75 0.07 False
Count00ForX - Duration of single invocation 198.81 ms 134.18 ms 0.67 0.01 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'

Payloads

Baseline
Compare

LinqBenchmarks.Where00ForX

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 217.65215769230767 < 279.9690017822619.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 54.86872135871174 (T) = (0 -224234730.59150463) / Math.Sqrt((792730168940908.8 / (299)) + (19477544229424.598 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.33264401425610696 = (336004674.2392714 - 224234730.59150463) / 336004674.2392714 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

LinqBenchmarks.Count00ForX

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 134.18419142857144 < 188.86680500833336.
IsChangePoint: Marked as a change because one of 5/22/2023 10:35:01 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 93.24952566934134 (T) = (0 -134198799.51648353) / Math.Sqrt((306089509594884.94 / (299)) + (57707423.34367737 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.41281893720225804 = (228547560.5719749 - 134198799.51648353) / 228547560.5719749 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CopyTo<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
List - Duration of single invocation 420.16 ns 367.18 ns 0.87 0.31 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CopyTo&lt;String&gt;*'

Payloads

Baseline
Compare

System.Collections.CopyTo<String>.List(Size: 2048)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 367.17651036595504 < 398.27618796513235.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 4/12/2023 8:39:35 PM, 5/5/2023 3:18:41 PM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 4.789439287408874 (T) = (0 -375.8045922829359) / Math.Sqrt((663.7847911944696 / (299)) + (377.95338178952966 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06654880419356798 = (402.59693701315456 - 375.8045922829359) / 402.59693701315456 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Guids

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
WriteGuids - Duration of single invocation 3.23 ms 2.14 ms 0.66 0.04 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Guids*'

Payloads

Baseline
Compare

System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: True, SkipValidation: False)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.138732199435028 < 3.0859208409384653.
IsChangePoint: Marked as a change because one of 2/18/2023 9:46:56 PM, 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 98.32805591986717 (T) = (0 -2137342.3226787755) / Math.Sqrt((60677043207.00659 / (299)) + (16318213.750893528 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3966394568540537 = (3542396.5769033986 - 2137342.3226787755) / 3542396.5769033986 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in FractalPerf.Launch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
Test - Duration of single invocation 276.33 ms 145.50 ms 0.53 0.01 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'FractalPerf.Launch*'

Payloads

Baseline
Compare

FractalPerf.Launch.Test

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 145.50375573076923 < 260.1987167067858.
IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 235.32199752185315 (T) = (0 -145727765.10432512) / Math.Sqrt((76102818035825.14 / (299)) + (205422749304.93246 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.45639892818747463 = (268078509.5188021 - 145727765.10432512) / 268078509.5188021 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Struct.GSeq

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
FilterSkipMapSum - Duration of single invocation 22.90 μs 16.69 μs 0.73 0.01 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Struct.GSeq*'

Payloads

Baseline
Compare

Struct.GSeq.FilterSkipMapSum

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.693182874821936 < 21.741513971390273.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 81.8085197020604 (T) = (0 -16692.65838921875) / Math.Sqrt((1164506.5277749402 / (153)) + (8.043269326327056 / (13))) is greater than 1.9745345758582833 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (153) + (13) - 2, .975) and 0.299513159788649 = (23830.0813533945 - 16692.65838921875) / 23830.0813533945 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.LogicArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
Test - Duration of single invocation 461.15 ms 394.08 ms 0.85 0.00 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.LogicArray*'

Payloads

Baseline
Compare

Benchstone.BenchI.LogicArray.Test

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 394.07970471428575 < 438.19504388997245.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 224.47239806544286 (T) = (0 -394154831.5773458) / Math.Sqrt((32164726411584.297 / (299)) + (98042129753.36023 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16193530838034528 = (470315520.4111953 - 394154831.5773458) / 470315520.4111953 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@EgorBo EgorBo mentioned this issue Jul 6, 2023
Merged
@EgorBo EgorBo closed this as completed Jul 6, 2023
@EgorBo EgorBo removed the untriaged label Jul 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
ampere arch-arm64 branch-refs/heads/main kind-micro os-linux perf-improvement runtime-coreclr
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@EgorBo