Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Windows/x64: 22 Improvements on 1/6/2023 6:58:01 PM #11466

Closed
performanceautofiler bot opened this issue Jan 10, 2023 · 1 comment
Closed

[Perf] Windows/x64: 22 Improvements on 1/6/2023 6:58:01 PM #11466

performanceautofiler bot opened this issue Jan 10, 2023 · 1 comment
Assignees
@dakersnar
Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-improvement runtime-coreclr

Comments

@performanceautofiler
Copy link

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 75f9609c5bb962d86afae1952536813a95b4a6d7
Compare f8218f9c4aaf4d6725524dc5d9e14c985e0f1b51
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OnesComplementOperatorBenchmark - Duration of single invocation 6.76 ns 0.33 ns 0.05 0.02 False
OnesComplementBenchmark - Duration of single invocation 6.76 ns 0.00 ns 0.00 0.04 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<SByte>.OnesComplementOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.3325736222880565 < 6.424300845063999.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/6/2023 2:04:15 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 866.7281339222446 (T) = (0 -0.3317449611172191) / Math.Sqrt((0.01676714139955456 / (299)) + (3.760077674936595E-07 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.9513839822861195 = (6.823779007767264 - 0.3317449611172191) / 6.823779007767264 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0 < 6.42440706002045.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/6/2023 2:04:15 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 821.6082224578455 (T) = (0 -0.036158289861438535) / Math.Sqrt((0.01826224898573099 / (299)) + (0.00010831668131615823 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.9947227862879555 = (6.851776682629387 - 0.036158289861438535) / 6.851776682629387 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline 75f9609c5bb962d86afae1952536813a95b4a6d7
Compare f8218f9c4aaf4d6725524dc5d9e14c985e0f1b51
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OnesComplementBenchmark - Duration of single invocation 6.79 ns 0.04 ns 0.01 0.06 False
OnesComplementOperatorBenchmark - Duration of single invocation 6.79 ns 0.33 ns 0.05 0.04 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int64>.OnesComplementBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.03883091652428235 < 6.424567686264648.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/6/2023 2:04:15 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 421.17008435401186 (T) = (0 -0.05318281292555873) / Math.Sqrt((0.016539892765676537 / (299)) + (0.0028612854618282244 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.9922251448507494 = (6.840360611822388 - 0.05318281292555873) / 6.840360611822388 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.33130679773050276 < 6.575120492267739.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/6/2023 2:04:15 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 850.6172382335866 (T) = (0 -0.3315397625564839) / Math.Sqrt((0.017532007491018207 / (299)) + (2.3441110442611832E-07 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.951571540025555 = (6.845969554502302 - 0.3315397625564839) / 6.845969554502302 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 75f9609c5bb962d86afae1952536813a95b4a6d7
Compare f8218f9c4aaf4d6725524dc5d9e14c985e0f1b51
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OnesComplementOperatorBenchmark - Duration of single invocation 6.76 ns 0.33 ns 0.05 0.02 False
OnesComplementBenchmark - Duration of single invocation 6.76 ns 0.04 ns 0.01 0.06 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Byte>.OnesComplementOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.33122361363826824 < 6.424470207746038.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/6/2023 2:04:15 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 807.4648681193102 (T) = (0 -0.3316437902191397) / Math.Sqrt((0.01952036059318346 / (299)) + (3.2801046397645076E-07 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.9516348717691973 = (6.85708489464776 - 0.3316437902191397) / 6.85708489464776 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.03888896059245507 < 6.566043934423189.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/6/2023 2:04:15 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 920.3795044623467 (T) = (0 -0.03885913263816878) / Math.Sqrt((0.01630792401912059 / (299)) + (1.7029547618460946E-08 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.9943156364695998 = (6.836144878902897 - 0.03885913263816878) / 6.836144878902897 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 75f9609c5bb962d86afae1952536813a95b4a6d7
Compare f8218f9c4aaf4d6725524dc5d9e14c985e0f1b51
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OnesComplementBenchmark - Duration of single invocation 6.79 ns 0.04 ns 0.01 0.04 False
OnesComplementOperatorBenchmark - Duration of single invocation 7.10 ns 0.33 ns 0.05 0.03 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int32>.OnesComplementBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.038907903784692104 < 6.437762180291506.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/6/2023 2:04:15 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 888.8193598766338 (T) = (0 -0.03890119980082094) / Math.Sqrt((0.017543685730773986 / (299)) + (4.747554481329599E-09 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.9943186831600791 = (6.847215337027955 - 0.03890119980082094) / 6.847215337027955 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.33125996995762824 < 6.437815670989129.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/6/2023 2:04:15 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 897.9496829397943 (T) = (0 -0.33169958032273217) / Math.Sqrt((0.015667695974434626 / (299)) + (3.581312130992447E-07 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.9514588590621031 = (6.8333700838862965 - 0.33169958032273217) / 6.8333700838862965 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 75f9609c5bb962d86afae1952536813a95b4a6d7
Compare f8218f9c4aaf4d6725524dc5d9e14c985e0f1b51
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OnesComplementOperatorBenchmark - Duration of single invocation 6.76 ns 0.33 ns 0.05 0.04 False
EqualsBenchmark - Duration of single invocation 9.54 ns 1.10 ns 0.11 0.02 False
OnesComplementBenchmark - Duration of single invocation 6.76 ns 0.04 ns 0.01 0.04 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Double>.OnesComplementOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.3313104946847102 < 6.424312357420455.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/6/2023 2:04:15 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 847.12288795638 (T) = (0 -0.3317240730386309) / Math.Sqrt((0.017633026735376922 / (299)) + (4.5962128730306605E-07 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.951494799668907 = (6.83893831536219 - 0.3317240730386309) / 6.83893831536219 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.096469650685753 < 9.062418633835865.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/6/2023 2:04:15 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 1060.6050684930592 (T) = (0 -1.0923843881032251) / Math.Sqrt((0.019340967707711668 / (299)) + (4.462887986508811E-06 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.8867236899521475 = (9.643537891036154 - 1.0923843881032251) / 9.643537891036154 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.03906603903111229 < 6.43833879535862.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/6/2023 2:04:15 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 895.0188567494405 (T) = (0 -0.03897021033256288) / Math.Sqrt((0.01729027194458466 / (299)) + (4.261317344026595E-09 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.9943068251241628 = (6.845075231740905 - 0.03897021033256288) / 6.845075231740905 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 75f9609c5bb962d86afae1952536813a95b4a6d7
Compare f8218f9c4aaf4d6725524dc5d9e14c985e0f1b51
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OnesComplementBenchmark - Duration of single invocation 6.79 ns 0.04 ns 0.01 0.05 False
OnesComplementOperatorBenchmark - Duration of single invocation 6.79 ns 0.33 ns 0.05 0.02 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt16>.OnesComplementBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.03884344776254197 < 6.437826685830765.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/6/2023 2:04:15 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 911.5343772211754 (T) = (0 -0.03893411843907497) / Math.Sqrt((0.016648967505667828 / (299)) + (1.40063563652032E-08 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.9943086336232282 = (6.840908819009884 - 0.03893411843907497) / 6.840908819009884 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.3312488355864111 < 6.4376922573744775.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/6/2023 2:04:15 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 857.0388087858688 (T) = (0 -0.33145842550188226) / Math.Sqrt((0.017260607639019182 / (299)) + (1.6090429720392613E-07 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.9515679654199051 = (6.8437848703987365 - 0.33145842550188226) / 6.8437848703987365 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 75f9609c5bb962d86afae1952536813a95b4a6d7
Compare f8218f9c4aaf4d6725524dc5d9e14c985e0f1b51
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OnesComplementOperatorBenchmark - Duration of single invocation 7.07 ns 0.33 ns 0.05 0.04 False
OnesComplementBenchmark - Duration of single invocation 6.76 ns 0.04 ns 0.01 0.05 False
EqualsBenchmark - Duration of single invocation 9.54 ns 1.22 ns 0.13 0.01 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Single>.OnesComplementOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.33126868449988933 < 6.424377775396316.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/6/2023 2:04:15 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 841.2188014968742 (T) = (0 -0.3317733960589039) / Math.Sqrt((0.01792491040880924 / (299)) + (4.3614278779818597E-07 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.9515431307904877 = (6.846777381023522 - 0.3317733960589039) / 6.846777381023522 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.038787302913404646 < 6.43938752592949.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/6/2023 2:04:15 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 825.6307122575796 (T) = (0 -0.03970479182222297) / Math.Sqrt((0.020308547068198936 / (299)) + (6.71939825913191E-06 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.9942189563376772 = (6.8681010110673135 - 0.03970479182222297) / 6.8681010110673135 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.224898486430334 < 9.062418260216909.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/6/2023 2:04:15 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 79.31181934031976 (T) = (0 -1.2198394830880255) / Math.Sqrt((3.0763213396877434 / (299)) + (0.0005624645888408617 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.8685572364560218 = (9.280385242964632 - 1.2198394830880255) / 9.280385242964632 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 75f9609c5bb962d86afae1952536813a95b4a6d7
Compare f8218f9c4aaf4d6725524dc5d9e14c985e0f1b51
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OnesComplementBenchmark - Duration of single invocation 7.07 ns 0.04 ns 0.01 0.06 False
OnesComplementOperatorBenchmark - Duration of single invocation 6.76 ns 0.33 ns 0.05 0.04 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt32>.OnesComplementBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.03888734193075523 < 6.424597050505927.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/6/2023 2:04:15 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 318.5418513940841 (T) = (0 -0.06911413517149269) / Math.Sqrt((0.017549567165164134 / (299)) + (0.005513012900897358 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.9899029869935113 = (6.845008036245707 - 0.06911413517149269) / 6.845008036245707 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.331198728464941 < 6.424382067761408.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/6/2023 2:04:15 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 827.5074128865315 (T) = (0 -0.33173694205435045) / Math.Sqrt((0.018558567697034355 / (299)) + (3.6942983503062557E-07 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.9515891727024028 = (6.852536107574766 - 0.33173694205435045) / 6.852536107574766 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 75f9609c5bb962d86afae1952536813a95b4a6d7
Compare f8218f9c4aaf4d6725524dc5d9e14c985e0f1b51
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OnesComplementOperatorBenchmark - Duration of single invocation 7.07 ns 0.33 ns 0.05 0.04 False
OnesComplementBenchmark - Duration of single invocation 6.76 ns 0.04 ns 0.01 0.06 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int16>.OnesComplementOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.33176352821201777 < 6.585918424080312.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/6/2023 2:04:15 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 764.8395468554957 (T) = (0 -0.328107918917936) / Math.Sqrt((0.018066534621697064 / (299)) + (0.00017174428384193944 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.9520943104565462 = (6.849038643318544 - 0.328107918917936) / 6.849038643318544 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.03910668079922551 < 6.437938938813896.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/6/2023 2:04:15 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 911.6154385404196 (T) = (0 -0.038923572216258306) / Math.Sqrt((0.016650528388255738 / (299)) + (1.6691838436807674E-08 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.9943109444528085 = (6.841833744350401 - 0.038923572216258306) / 6.841833744350401 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 10, 2023 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 75f9609c5bb962d86afae1952536813a95b4a6d7
Compare f8218f9c4aaf4d6725524dc5d9e14c985e0f1b51
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OnesComplementOperatorBenchmark - Duration of single invocation 7.05 ns 0.33 ns 0.05 0.02 False
OnesComplementBenchmark - Duration of single invocation 6.76 ns 0.04 ns 0.01 0.04 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt64>.OnesComplementOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.3312601189426607 < 6.437705089161858.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/6/2023 2:04:15 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 847.0653080797492 (T) = (0 -0.3315720678619588) / Math.Sqrt((0.017687497652004107 / (299)) + (2.8169573190037613E-07 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.9515789272148395 = (6.847681160083166 - 0.3315720678619588) / 6.847681160083166 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.03893977294651275 < 6.424360549670543.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/6/2023 2:04:15 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 905.7227626382733 (T) = (0 -0.03890813712978357) / Math.Sqrt((0.016881126889797687 / (299)) + (1.3467604370841915E-08 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.9943153923531438 = (6.844471869804634 - 0.03890813712978357) / 6.844471869804634 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@EgorBo EgorBo mentioned this issue Jan 10, 2023
Merged
@EgorBo EgorBo removed the untriaged label Jan 10, 2023
@EgorBo EgorBo closed this as completed Jan 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dakersnar dakersnar

Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-improvement runtime-coreclr
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@EgorBo @dakersnar