Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 286 Improvements on 10/11/2022 6:23:40 AM #9041

Closed
performanceautofiler bot opened this issue Oct 18, 2022 · 14 comments
Closed

[Perf] Linux/x64: 286 Improvements on 10/11/2022 6:23:40 AM #9041

performanceautofiler bot opened this issue Oct 18, 2022 · 14 comments
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-improvement runtime-mono untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SubtractBenchmark - Duration of single invocation 260.93 ns 211.93 ns 0.81 0.11 False
AddBenchmark - Duration of single invocation 254.36 ns 204.01 ns 0.80 0.10 False
XorBenchmark - Duration of single invocation 28.73 ns 16.58 ns 0.58 0.30 False
SubtractionOperatorBenchmark - Duration of single invocation 252.01 ns 208.19 ns 0.83 0.07 False
MinBenchmark - Duration of single invocation 298.01 ns 273.05 ns 0.92 0.09 False
LessThanOrEqualAllBenchmark - Duration of single invocation 328.15 ns 282.52 ns 0.86 0.09 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 324.25 ns 282.79 ns 0.87 0.07 False
UnaryNegateOperatorBenchmark - Duration of single invocation 257.35 ns 210.16 ns 0.82 0.10 False
BitwiseOrOperatorBenchmark - Duration of single invocation 27.05 ns 16.39 ns 0.61 0.26 False
MultiplyOperatorBenchmark - Duration of single invocation 248.91 ns 203.47 ns 0.82 0.10 False
LessThanAllBenchmark - Duration of single invocation 329.87 ns 303.11 ns 0.92 0.09 False
GreaterThanAnyBenchmark - Duration of single invocation 282.40 ns 266.87 ns 0.94 0.07 False
GreaterThanAllBenchmark - Duration of single invocation 298.95 ns 263.35 ns 0.88 0.05 False
AddOperatorBenchmark - Duration of single invocation 247.40 ns 203.87 ns 0.82 0.08 False
OnesComplementBenchmark - Duration of single invocation 138.81 ns 91.61 ns 0.66 0.11 False
OneBenchmark - Duration of single invocation 112.82 ns 78.84 ns 0.70 0.05 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 27.19 ns 16.59 ns 0.61 0.25 False
OnesComplementOperatorBenchmark - Duration of single invocation 140.03 ns 92.91 ns 0.66 0.10 False
MultiplyBenchmark - Duration of single invocation 249.44 ns 203.30 ns 0.82 0.10 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 282.88 ns 254.42 ns 0.90 0.08 False
BitwiseAndOperatorBenchmark - Duration of single invocation 26.59 ns 19.72 ns 0.74 0.27 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<SByte>.SubtractBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 211.9324705118378 < 247.9884236469714.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 69.60015680477161 (T) = (0 -208.27923202480846) / Math.Sqrt((142.48376966473103 / (299)) + (6.210670786687659 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.2203607831327727 = (267.14822384348895 - 208.27923202480846) / 267.14822384348895 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 204.00935277375677 < 239.39084787509785.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 26.658723048728188 (T) = (0 -203.55209022054976) / Math.Sqrt((132.22259087961072 / (299)) + (105.79699680658167 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.22066631148271715 = (261.18733633575715 - 203.55209022054976) / 261.18733633575715 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.579365901433654 < 27.828826038224545.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 20.547052827169857 (T) = (0 -17.75069699039991) / Math.Sqrt((2.8105677020386453 / (299)) + (6.770288167208137 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.3894878459479113 = (29.075091908629595 - 17.75069699039991) / 29.075091908629595 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 208.19131796060958 < 241.15068077471292.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 31.9168931066382 (T) = (0 -210.35463979337948) / Math.Sqrt((141.22921697458722 / (299)) + (66.96647413959916 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.20942063970005967 = (266.07656404484476 - 210.35463979337948) / 266.07656404484476 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.MinBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 273.0547630378452 < 283.93376646490367.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 11.856941180506183 (T) = (0 -277.83568499324724) / Math.Sqrt((360.46416379256084 / (299)) + (359.9977769345319 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.14204921616694863 = (323.83639041853394 - 277.83568499324724) / 323.83639041853394 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 282.52461764622404 < 314.184132399676.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 65.85775841413313 (T) = (0 -285.4698920265242) / Math.Sqrt((71.51373591713606 / (299)) + (15.642714301036413 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.17085781210532303 = (344.29546125421183 - 285.4698920265242) / 344.29546125421183 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 282.7940268645026 < 309.43928143831886.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 54.93573203220229 (T) = (0 -283.0743565492198) / Math.Sqrt((210.19655872395907 / (299)) + (15.052020642320006 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1789027772794093 = (344.75132629397103 - 283.0743565492198) / 344.75132629397103 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.UnaryNegateOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 210.15675563326343 < 242.72959541403824.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 38.39508475844208 (T) = (0 -209.62574649665748) / Math.Sqrt((279.5029429149266 / (299)) + (30.55574944336313 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.21198026585291682 = (266.01585901087475 - 209.62574649665748) / 266.01585901087475 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.390747730847455 < 25.729221750245127.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 38.47510049600373 (T) = (0 -17.099760681611844) / Math.Sqrt((4.00552459127438 / (299)) + (1.3617955682281038 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.3591383397171856 = (26.682452300338365 - 17.099760681611844) / 26.682452300338365 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 203.47020883416198 < 236.88140754844713.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 58.042929584543245 (T) = (0 -202.6647994785061) / Math.Sqrt((156.3721479730587 / (299)) + (13.359166121242223 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.22416375719309944 = (261.22110349638274 - 202.6647994785061) / 261.22110349638274 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 303.1107744673057 < 315.767325425387.
IsChangePoint: Marked as a change because one of 8/3/2022 6:54:00 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 25.187589718010056 (T) = (0 -289.7607547594198) / Math.Sqrt((109.46745710178617 / (299)) + (115.59514964272988 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.15781865323588073 = (344.0598107198127 - 289.7607547594198) / 344.0598107198127 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.GreaterThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 266.8700970782242 < 267.5567325016444.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 48.4081996876223 (T) = (0 -262.8041879823822) / Math.Sqrt((184.71140918752943 / (299)) + (6.369750651037164 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.14482271413330355 = (307.3095980514006 - 262.8041879823822) / 307.3095980514006 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 263.34584272054786 < 284.896688323177.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 58.81400101357174 (T) = (0 -263.92373778745525) / Math.Sqrt((222.2845683348857 / (299)) + (6.989612127452538 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.18240598858307538 = (322.80537051643074 - 263.92373778745525) / 322.80537051643074 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 203.87067600801288 < 235.53721571708283.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 82.53101249295597 (T) = (0 -201.61207112790913) / Math.Sqrt((113.16146714516645 / (299)) + (3.160773769986721 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.22448643302081014 = (259.9723328029401 - 201.61207112790913) / 259.9723328029401 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 91.60658340069521 < 132.1171076815848.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 118.6053772393964 (T) = (0 -91.37420886851623) / Math.Sqrt((18.127137660705017 / (299)) + (2.4936823183679686 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.34490873540983225 = (139.4831740363397 - 91.37420886851623) / 139.4831740363397 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 78.84165101790677 < 107.12707412626446.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 67.8993650306017 (T) = (0 -78.54308428049872) / Math.Sqrt((9.239255965718037 / (299)) + (6.832628465863355 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.3153812017708886 = (114.72528140282506 - 78.54308428049872) / 114.72528140282506 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.594392586866185 < 25.453841057956407.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 64.40959735521677 (T) = (0 -17.068580770950195) / Math.Sqrt((3.9879396130828626 / (299)) + (0.22418909283895957 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.3569648474431841 = (26.54377556667415 - 17.068580770950195) / 26.54377556667415 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 92.91147906578384 < 132.7849642981692.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 156.44991698843018 (T) = (0 -89.73824037021798) / Math.Sqrt((15.478449093980831 / (299)) + (1.19904224573972 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.3573487626989016 = (139.6375439143103 - 89.73824037021798) / 139.6375439143103 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 203.29663982404728 < 240.42444832177821.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 92.82860509667964 (T) = (0 -201.75920573180017) / Math.Sqrt((102.75818308197886 / (299)) + (1.6748871001072814 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.22715394377413706 = (261.06001849459705 - 201.75920573180017) / 261.06001849459705 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 254.42319815025078 < 267.56748025822395.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 45.765420993829295 (T) = (0 -256.1714625004393) / Math.Sqrt((117.53193319690784 / (299)) + (13.717414137704587 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.14633484057091986 = (300.08424224758494 - 256.1714625004393) / 300.08424224758494 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.72275084161949 < 25.54446822568773.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 47.91707360269266 (T) = (0 -17.167038986793372) / Math.Sqrt((3.6391274828811326 / (299)) + (0.7092577881499553 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.35087283773992534 = (26.446342080375537 - 17.167038986793372) / 26.446342080375537 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryFormatL - Duration of single invocation 1.32 μs 1.01 μs 0.77 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Version*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Version.TryFormatL

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.01453186537773 < 1.2548162602330242.
IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 36.28813574762491 (T) = (0 -993.3363157139321) / Math.Sqrt((2179.4472909959395 / (299)) + (768.1483840554217 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1792471691615179 = (1210.2746142211151 - 993.3363157139321) / 1210.2746142211151 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in BilinearTest

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Interpol_Vector - Duration of single invocation 1.88 ms 1.60 ms 0.85 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'BilinearTest*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

BilinearTest.Interpol_Vector

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.6019444061002177 < 1.7870481556987847.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 29.649158698212137 (T) = (0 -1621164.9842704944) / Math.Sqrt((3164490857.502241 / (299)) + (870213774.2747064 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.10824773356239697 = (1817954.4311636793 - 1621164.9842704944) / 1817954.4311636793 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare 2b07c8c46523b816b1eac5c8af092e52b20a69d2
Diff Diff

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentStack - Duration of single invocation 61.79 μs 57.53 μs 0.93 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.CreateAddAndClear<Int32>.ConcurrentStack(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 57.52812200215326 < 58.28032645264406.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 20.086452203711836 (T) = (0 -57773.07762031523) / Math.Sqrt((1814233.479262524 / (299)) + (838930.3790607117 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.06472904377645634 = (61771.48690000227 - 57773.07762031523) / 61771.48690000227 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Narrow_ulong - Duration of single invocation 94.11 μs 78.73 μs 0.84 0.19 False
Convert_int_float - Duration of single invocation 97.69 μs 70.74 μs 0.72 0.26 False
Widen_int - Duration of single invocation 197.95 μs 144.03 μs 0.73 0.15 False
Convert_double_long - Duration of single invocation 70.27 μs 48.45 μs 0.69 0.24 False
Widen_short - Duration of single invocation 247.95 μs 184.80 μs 0.75 0.14 False
Widen_sbyte - Duration of single invocation 328.93 μs 238.91 μs 0.73 0.15 False
Widen_ushort - Duration of single invocation 239.02 μs 180.94 μs 0.76 0.13 False
Convert_float_uint - Duration of single invocation 119.85 μs 81.44 μs 0.68 0.18 False
Widen_byte - Duration of single invocation 330.44 μs 245.15 μs 0.74 0.13 False
Convert_double_ulong - Duration of single invocation 79.50 μs 55.77 μs 0.70 0.23 False
Narrow_long - Duration of single invocation 99.79 μs 80.42 μs 0.81 0.26 False
Widen_uint - Duration of single invocation 193.92 μs 134.57 μs 0.69 0.16 False
Convert_float_int - Duration of single invocation 97.81 μs 70.16 μs 0.72 0.18 False
Widen_float - Duration of single invocation 198.71 μs 143.55 μs 0.72 0.17 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorConvert.Narrow_ulong

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 78.72704029166668 < 88.48557779142732.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 16.20835303913514 (T) = (0 -79605.83483849873) / Math.Sqrt((19874719.61279675 / (299)) + (12259150.540181745 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.12808133723016962 = (91299.61111924626 - 79605.83483849873) / 91299.61111924626 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Convert_int_float

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 70.74105879011644 < 93.05585257180739.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 39.64887645743602 (T) = (0 -71182.1681903137) / Math.Sqrt((36190864.620619476 / (299)) + (2222220.744902538 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.20199310849976768 = (89199.94169034439 - 71182.1681903137) / 89199.94169034439 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_int

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 144.02984157282282 < 184.43244172839505.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 51.460604152653886 (T) = (0 -140060.66285155923) / Math.Sqrt((34716072.70181874 / (299)) + (22800930.44973484 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.2714296430458362 = (192240.4082388035 - 140060.66285155923) / 192240.4082388035 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Convert_double_long

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 48.44978696546739 < 67.55336080756588.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 39.38630496602971 (T) = (0 -48613.72578057914) / Math.Sqrt((13689748.640442701 / (299)) + (4634979.106639497 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.27719407408204294 = (67256.95520390227 - 48613.72578057914) / 67256.95520390227 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_short

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 184.80449767441863 < 229.8734478529695.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 45.31644348208031 (T) = (0 -176380.98445137913) / Math.Sqrt((73959813.86123046 / (299)) + (32343132.577833317 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.23598217176016953 = (230859.77569100904 - 176380.98445137913) / 230859.77569100904 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_sbyte

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 238.9055444302721 < 313.2340354806858.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 70.3450850896003 (T) = (0 -239711.85704943506) / Math.Sqrt((98996958.51979508 / (299)) + (25109041.059448797 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.24302832303097907 = (316672.1613802801 - 239711.85704943506) / 316672.1613802801 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_ushort

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 180.93526659482765 < 228.7560221727318.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 54.72300415811042 (T) = (0 -176184.27490884453) / Math.Sqrt((92413360.84999514 / (299)) + (19744232.031841297 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.2382707680071855 = (231295.14728995788 - 176184.27490884453) / 231295.14728995788 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Convert_float_uint

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 81.43829846412679 < 101.59901983708309.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 19.895916268564932 (T) = (0 -81685.79988299168) / Math.Sqrt((25797592.64752496 / (299)) + (18150404.519486416 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.17743814911878447 = (99306.57469159453 - 81685.79988299168) / 99306.57469159453 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_byte

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 245.1517392578125 < 313.0493226011654.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 66.97093856929345 (T) = (0 -240698.198683229) / Math.Sqrt((106871911.92615308 / (299)) + (22874877.57994884 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.23394085738368164 = (314203.1538990234 - 240698.198683229) / 314203.1538990234 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Convert_double_ulong

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 55.76627406791407 < 68.9562957671844.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 27.299358167266963 (T) = (0 -51431.01968507737) / Math.Sqrt((13400568.501892561 / (299)) + (7715607.712861875 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.24660475200272758 = (68265.65447790503 - 51431.01968507737) / 68265.65447790503 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_long

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 80.41692466240227 < 89.11706908854165.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 11.60501874883765 (T) = (0 -80192.51994308094) / Math.Sqrt((16970392.82304156 / (299)) + (23765597.894958243 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.12286793036289184 = (91425.82140025804 - 80192.51994308094) / 91425.82140025804 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_uint

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 134.57192067920917 < 184.78757061545667.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 58.0630443358638 (T) = (0 -130672.63651218035) / Math.Sqrt((28417493.89281183 / (299)) + (28862926.52490947 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.320409185492905 = (192281.34595514918 - 130672.63651218035) / 192281.34595514918 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Convert_float_int

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 70.16090198167882 < 89.69268535491597.
IsChangePoint: Marked as a change because one of 9/29/2022 2:35:34 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 54.86458794213201 (T) = (0 -70580.3820677871) / Math.Sqrt((22605576.875515226 / (299)) + (695394.5707080291 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.20088186924264392 = (88322.83907874204 - 70580.3820677871) / 88322.83907874204 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_float

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 143.54996831039756 < 188.6316690979491.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 51.9889682161176 (T) = (0 -144467.94620321837) / Math.Sqrt((26717741.0912911 / (299)) + (25041780.606270436 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.26626009780995835 = (196892.58519540154 - 144467.94620321837) / 196892.58519540154 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare 2b07c8c46523b816b1eac5c8af092e52b20a69d2
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
XorBenchmark - Duration of single invocation 28.80 ns 16.81 ns 0.58 0.22 False
GreaterThanAllBenchmark - Duration of single invocation 149.13 ns 117.69 ns 0.79 0.19 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 26.80 ns 16.92 ns 0.63 0.32 False
UnaryNegateOperatorBenchmark - Duration of single invocation 95.96 ns 76.56 ns 0.80 0.22 False
AndNotBenchmark - Duration of single invocation 107.46 ns 65.83 ns 0.61 0.23 False
SubtractBenchmark - Duration of single invocation 96.90 ns 76.76 ns 0.79 0.22 False
NegateBenchmark - Duration of single invocation 96.28 ns 76.10 ns 0.79 0.17 False
AddBenchmark - Duration of single invocation 98.58 ns 76.77 ns 0.78 0.21 False
DivideBenchmark - Duration of single invocation 96.50 ns 77.91 ns 0.81 0.16 False
SubtractionOperatorBenchmark - Duration of single invocation 93.74 ns 77.18 ns 0.82 0.19 False
AbsBenchmark - Duration of single invocation 95.53 ns 67.41 ns 0.71 0.18 False
MultiplyBenchmark - Duration of single invocation 95.84 ns 76.71 ns 0.80 0.21 False
ConditionalSelectBenchmark - Duration of single invocation 166.71 ns 89.69 ns 0.54 0.28 False
OnesComplementBenchmark - Duration of single invocation 80.30 ns 44.50 ns 0.55 0.23 False
OnesComplementOperatorBenchmark - Duration of single invocation 88.47 ns 42.88 ns 0.48 0.25 False
BitwiseAndBenchmark - Duration of single invocation 28.99 ns 17.50 ns 0.60 0.30 False
MultiplyOperatorBenchmark - Duration of single invocation 95.81 ns 76.76 ns 0.80 0.20 False
BitwiseOrBenchmark - Duration of single invocation 29.24 ns 17.09 ns 0.58 0.26 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Single>.XorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.814388659124454 < 27.484051977718046.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 81.13526730947466 (T) = (0 -17.123746215691657) / Math.Sqrt((1.7467933994883953 / (299)) + (0.3743434132557741 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.40603799769346555 = (28.82969979425445 - 17.123746215691657) / 28.82969979425445 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 117.69313432653641 < 141.61010883310382.
IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 27.019684571554468 (T) = (0 -117.50980394752756) / Math.Sqrt((40.736584578040684 / (299)) + (9.56877664618454 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1403633692399731 = (136.69706448366924 - 117.50980394752756) / 136.69706448366924 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.92354407455445 < 25.492667224164972.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 60.42736101910733 (T) = (0 -16.887098345407505) / Math.Sqrt((4.0531641668276945 / (299)) + (0.3463538548578455 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.36757988532370733 = (26.702342246105264 - 16.887098345407505) / 26.702342246105264 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.UnaryNegateOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 76.56046790277703 < 91.21576831864887.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 19.40493835546519 (T) = (0 -77.27370509181003) / Math.Sqrt((39.41305508774301 / (299)) + (5.87668800851536 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.128002116378719 = (88.61684935622036 - 77.27370509181003) / 88.61684935622036 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 65.83019095676488 < 101.96023392652214.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 65.63775733051153 (T) = (0 -61.27529959341606) / Math.Sqrt((30.213532695242176 / (299)) + (7.038518824708459 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.39508716769259306 = (101.29608155225402 - 61.27529959341606) / 101.29608155225402 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 76.75621168816247 < 91.85904424835415.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 19.652724624692652 (T) = (0 -78.05874261878901) / Math.Sqrt((42.85502857671029 / (299)) + (5.227565274523323 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1263596422702734 = (89.34882864344235 - 78.05874261878901) / 89.34882864344235 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.NegateBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 76.10144305492784 < 91.43610478243755.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 10:51:47 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 17.030066341162495 (T) = (0 -77.27533992462006) / Math.Sqrt((36.65789018587268 / (299)) + (8.652866054059158 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.12792859589467515 = (88.61125311624951 - 77.27533992462006) / 88.61125311624951 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 76.77133254521206 < 93.57533086093784.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 13.042995090597017 (T) = (0 -79.22351903969455) / Math.Sqrt((37.98405455721316 / (299)) + (13.877914718609999 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.11497656007336861 = (89.51572971475444 - 79.22351903969455) / 89.51572971475444 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 77.91200721606745 < 91.8595809374347.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 20.87033960280112 (T) = (0 -79.07981477387777) / Math.Sqrt((34.900212937770384 / (299)) + (4.91864343006418 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1248848000393894 = (90.36503397202699 - 79.07981477387777) / 90.36503397202699 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 77.18304504458142 < 89.31150544683585.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 8.192236186338839 (T) = (0 -79.19544895411158) / Math.Sqrt((31.66580094951389 / (299)) + (20.465933605945487 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.08905045592835598 = (86.93725077256644 - 79.19544895411158) / 86.93725077256644 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.AbsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 67.41439391847838 < 91.06464298595924.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 30.841468173890235 (T) = (0 -69.32514228436992) / Math.Sqrt((38.376396125140836 / (299)) + (6.529751651481838 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.21967428935403008 = (88.84128939821952 - 69.32514228436992) / 88.84128939821952 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 76.70855121467264 < 91.16262000370607.
IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 9/29/2022 8:01:59 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 24.58644358347414 (T) = (0 -77.80235311393506) / Math.Sqrt((57.37925994896481 / (299)) + (1.2855951682646019 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.13394272765232595 = (89.8351132172027 - 77.80235311393506) / 89.8351132172027 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 89.69218130528597 < 158.5416090093775.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 61.54825876022584 (T) = (0 -85.27572015087357) / Math.Sqrt((50.974523854209735 / (299)) + (35.887085159547 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.469174779603158 = (160.64745395315225 - 85.27572015087357) / 160.64745395315225 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 44.49800330968529 < 80.88210336386021.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 60.14495533494485 (T) = (0 -45.723951355548486) / Math.Sqrt((27.625378392191294 / (299)) + (4.786682962812528 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.4058544740057857 = (76.95749501611789 - 45.723951355548486) / 76.95749501611789 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 42.87657757416551 < 80.087251641025.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 62.4771949535458 (T) = (0 -43.989615644822706) / Math.Sqrt((26.862908728022934 / (299)) + (5.144151149060322 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.42923314711239147 = (77.07107625867131 - 43.989615644822706) / 77.07107625867131 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.497593832372015 < 27.40669204724363.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 75.82765117197326 (T) = (0 -17.051949499815017) / Math.Sqrt((2.867493855542353 / (299)) + (0.4423436534507347 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.4147081554974328 = (29.134097220007007 - 17.051949499815017) / 29.134097220007007 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 76.76060489359844 < 90.0026640010621.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 19.251442224106725 (T) = (0 -78.0107475520441) / Math.Sqrt((41.50277213794707 / (299)) + (3.066974606144215 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.10951168273934148 = (87.60445930612838 - 78.0107475520441) / 87.60445930612838 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.09019923443771 < 27.94948966186318.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 76.48154979000125 (T) = (0 -17.26315931971533) / Math.Sqrt((1.9215507308966644 / (299)) + (0.4800083700846108 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.40482012837270537 = (29.00494479511839 - 17.26315931971533) / 29.00494479511839 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare 2b07c8c46523b816b1eac5c8af092e52b20a69d2
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitwiseOrOperatorBenchmark - Duration of single invocation 26.54 ns 18.68 ns 0.70 0.26 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 29.37 ns 18.16 ns 0.62 0.24 False
AddOperatorBenchmark - Duration of single invocation 245.66 ns 200.69 ns 0.82 0.07 False
OneBenchmark - Duration of single invocation 112.87 ns 78.59 ns 0.70 0.17 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 330.49 ns 281.55 ns 0.85 0.10 False
LessThanAnyBenchmark - Duration of single invocation 276.82 ns 250.54 ns 0.91 0.06 False
NegateBenchmark - Duration of single invocation 252.75 ns 206.39 ns 0.82 0.12 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 281.09 ns 253.58 ns 0.90 0.06 False
GreaterThanAllBenchmark - Duration of single invocation 311.17 ns 264.55 ns 0.85 0.11 False
LessThanAllBenchmark - Duration of single invocation 341.61 ns 278.96 ns 0.82 0.10 False
ConditionalSelectBenchmark - Duration of single invocation 218.69 ns 130.51 ns 0.60 0.20 False
EqualsAnyBenchmark - Duration of single invocation 296.10 ns 263.22 ns 0.89 0.05 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 299.83 ns 279.61 ns 0.93 0.08 False
BitwiseAndOperatorBenchmark - Duration of single invocation 26.83 ns 16.96 ns 0.63 0.31 False
LessThanOrEqualAllBenchmark - Duration of single invocation 340.38 ns 282.92 ns 0.83 0.12 False
MultiplyBenchmark - Duration of single invocation 249.98 ns 199.79 ns 0.80 0.11 False
GreaterThanAnyBenchmark - Duration of single invocation 280.64 ns 263.84 ns 0.94 0.08 False
MultiplyOperatorBenchmark - Duration of single invocation 244.27 ns 200.98 ns 0.82 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Byte>.BitwiseOrOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.680776593670252 < 25.784801264143006.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 51.197402873889004 (T) = (0 -17.329690817874933) / Math.Sqrt((2.7655631958506035 / (299)) + (0.6124640325965619 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.345520894484098 = (26.478600572304856 - 17.329690817874933) / 26.478600572304856 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.158731751572414 < 25.69043758196243.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 28.84768581760877 (T) = (0 -17.425579760638392) / Math.Sqrt((3.460079673149747 / (299)) + (2.2790112095932282 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.3427451250590533 = (26.512667193535922 - 17.425579760638392) / 26.512667193535922 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 200.69484422834503 < 233.4632804364795.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 7.810177321072225 (T) = (0 -215.50003393245356) / Math.Sqrt((163.83071593983368 / (299)) + (840.3857990403623 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.16941261186648798 = (259.45497970625723 - 215.50003393245356) / 259.45497970625723 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 78.5874908630922 < 107.84683681809511.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 75.45960374721014 (T) = (0 -78.3283900659769) / Math.Sqrt((16.461417782273948 / (299)) + (4.621156014190392 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.3112765763770869 = (113.72981864613179 - 78.3283900659769) / 113.72981864613179 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 281.54863225439766 < 315.8685999563308.
IsChangePoint: Marked as a change because one of 7/30/2022 7:04:11 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 22.900474682919125 (T) = (0 -289.3035424733373) / Math.Sqrt((186.93926877320678 / (299)) + (119.43706223107982 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.15100471802877183 = (340.7598942147497 - 289.3035424733373) / 340.7598942147497 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 250.54458840359703 < 262.07365626890663.
IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 16.580161677137024 (T) = (0 -257.66904255094755) / Math.Sqrt((86.52318811199258 / (299)) + (115.89949964155916 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.12295848235889713 = (293.793437788414 - 257.66904255094755) / 293.793437788414 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.NegateBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 206.38819120855754 < 239.96508547840827.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 34.211117438144136 (T) = (0 -210.60049956892664) / Math.Sqrt((361.5768122332867 / (299)) + (34.188267626080915 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.20683092850454102 = (265.5177907679326 - 210.60049956892664) / 265.5177907679326 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 253.58179232329798 < 267.2114760049479.
IsChangePoint: Marked as a change because one of 9/28/2022 10:18:57 AM, 9/28/2022 6:40:13 PM, 10/11/2022 8:42:28 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 20.28270449559481 (T) = (0 -259.7635355667711) / Math.Sqrt((117.47537125931571 / (299)) + (58.61130224244935 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.12460787663056733 = (296.73963088327423 - 259.7635355667711) / 296.73963088327423 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 264.54551357243565 < 290.8676740530206.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 41.67681424819655 (T) = (0 -266.36021635326784) / Math.Sqrt((224.2234905594266 / (299)) + (22.60800136684578 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1691061841559579 = (320.5707050337024 - 266.36021635326784) / 320.5707050337024 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 278.95890451185426 < 323.58154714373217.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 37.27439786333317 (T) = (0 -284.92097124810584) / Math.Sqrt((73.67521748149925 / (299)) + (52.55633490651913 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.16229036669274918 = (340.1190101195888 - 284.92097124810584) / 340.1190101195888 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 130.50961980560143 < 207.8036939803399.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 111.47880353717902 (T) = (0 -129.21374623132195) / Math.Sqrt((44.99788703671544 / (299)) + (14.543643748026561 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.41732043019802 = (221.75781154509062 - 129.21374623132195) / 221.75781154509062 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 263.22202878204433 < 275.3341254600527.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 52.35478898823419 (T) = (0 -263.5213078292545) / Math.Sqrt((121.13749053542098 / (299)) + (4.929142788798437 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1328582948004053 = (303.89647533859346 - 263.5213078292545) / 303.89647533859346 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 279.60624263651073 < 289.24040041253215.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 32.338076913269525 (T) = (0 -285.0981614673031) / Math.Sqrt((177.05902700140564 / (299)) + (20.41063428281562 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.11637319146436989 = (322.6454411673808 - 285.0981614673031) / 322.6454411673808 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.963567394024622 < 25.46738066932298.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 36.022094646900655 (T) = (0 -17.417244220105758) / Math.Sqrt((2.9633978410879944 / (299)) + (1.502928176449648 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.34988494868465814 = (26.791018274175332 - 17.417244220105758) / 26.791018274175332 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 282.9165837425123 < 323.15602015641923.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 34.17835338494448 (T) = (0 -291.2595572674369) / Math.Sqrt((109.7593286918432 / (299)) + (50.02063373683395 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.15082601323910647 = (342.9916151558331 - 291.2595572674369) / 342.9916151558331 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 199.785942606051 < 237.49355311694612.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 3:03:11 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 19.1433523309247 (T) = (0 -208.0031067340372) / Math.Sqrt((229.27182808312799 / (299)) + (167.24428000270055 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.21026284924536182 = (263.3827046572123 - 208.0031067340372) / 263.3827046572123 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.GreaterThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 263.8381550141867 < 271.25551546892257.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 25.767140933373 (T) = (0 -265.96600216625325) / Math.Sqrt((153.38965446419917 / (299)) + (43.428279311135434 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.12366032433056588 = (303.49647465531257 - 265.96600216625325) / 303.49647465531257 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 200.98279570437904 < 233.13570237600717.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 38.98046496423925 (T) = (0 -206.25419736031566) / Math.Sqrt((167.4552407529533 / (299)) + (36.79974625314561 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.21223119254867107 = (261.8207212692396 - 206.25419736031566) / 261.8207212692396 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
List - Duration of single invocation 5.19 ms 4.36 ms 0.84 0.10 False
Queue - Duration of single invocation 5.26 ms 4.23 ms 0.80 0.10 False
ImmutableArray - Duration of single invocation 5.35 ms 4.20 ms 0.79 0.09 False
Array - Duration of single invocation 5.22 ms 4.25 ms 0.82 0.13 False
Stack - Duration of single invocation 5.30 ms 4.18 ms 0.79 0.10 False
ImmutableList - Duration of single invocation 9.28 ms 8.29 ms 0.89 0.02 False
ICollection - Duration of single invocation 5.27 ms 4.17 ms 0.79 0.12 False
LinkedList - Duration of single invocation 5.61 ms 4.45 ms 0.79 0.14 False
Span - Duration of single invocation 3.12 ms 2.13 ms 0.68 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsTrue<String>.List(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.364436194791666 < 5.032303695250497.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 42.13546875459561 (T) = (0 -4314388.012097663) / Math.Sqrt((16619021666.756557 / (299)) + (14770587786.708912 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.19338550672493215 = (5348760.836890134 - 4314388.012097663) / 5348760.836890134 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;String&gt;.Queue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.228196981026786 < 5.00412998924355.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 50.66767624379823 (T) = (0 -4297033.075137787) / Math.Sqrt((25112500916.570393 / (299)) + (9436850959.925253 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.19711177872016583 = (5351969.254559686 - 4297033.075137787) / 5351969.254559686 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;String&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.198947175 < 5.087472120555555.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 65.59493989825896 (T) = (0 -4281291.406747092) / Math.Sqrt((13820866829.81461 / (299)) + (6727792316.51588 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.2084416713770051 = (5408687.208427055 - 4281291.406747092) / 5408687.208427055 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;String&gt;.Array(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.25393398125 < 4.989280291093749.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 68.30937886275619 (T) = (0 -4274639.582535208) / Math.Sqrt((37832950191.81202 / (299)) + (4127990743.10603 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.2076280549137043 = (5394738.68180639 - 4274639.582535208) / 5394738.68180639 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;String&gt;.Stack(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.180190563616072 < 5.0345568243656516.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 62.20038367711193 (T) = (0 -4273874.8175886525) / Math.Sqrt((16276665725.68364 / (299)) + (7313161019.272436 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.20543854589472807 = (5378910.335389117 - 4273874.8175886525) / 5378910.335389117 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;String&gt;.ImmutableList(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.286229852678572 < 8.833639355572915.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 64.74062933176819 (T) = (0 -8258295.460241078) / Math.Sqrt((22720958675.877327 / (299)) + (5405384499.982317 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.11393564325932998 = (9320198.242279692 - 8258295.460241078) / 9320198.242279692 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;String&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.174415822395833 < 5.012106121730769.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 36.12910313996572 (T) = (0 -4301091.631602811) / Math.Sqrt((32250068761.737232 / (299)) + (21285056852.585197 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1983869954537551 = (5365546.226433109 - 4301091.631602811) / 5365546.226433109 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;String&gt;.LinkedList(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.446595769230769 < 5.260193931845238.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 41.957720421523135 (T) = (0 -4448114.352025439) / Math.Sqrt((56068051198.60872 / (299)) + (12490815264.527617 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.19600104098688878 = (5532487.700587808 - 4448114.352025439) / 5532487.700587808 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;String&gt;.Span(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.1271524921875 < 2.930928338293269.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 160.2391671382382 (T) = (0 -2118249.3989292923) / Math.Sqrt((6058284787.383454 / (299)) + (450303619.9280581 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.314959501991024 = (3092152.0772652733 - 2118249.3989292923) / 3092152.0772652733 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConditionalSelectBenchmark - Duration of single invocation 186.26 ns 104.03 ns 0.56 0.24 False
LessThanAllBenchmark - Duration of single invocation 228.22 ns 198.52 ns 0.87 0.13 False
GreaterThanAllBenchmark - Duration of single invocation 208.92 ns 185.22 ns 0.89 0.05 False
LessThanOrEqualAllBenchmark - Duration of single invocation 238.20 ns 206.22 ns 0.87 0.14 False
AndNotBenchmark - Duration of single invocation 139.03 ns 92.81 ns 0.67 0.21 False
EqualsAnyBenchmark - Duration of single invocation 196.39 ns 184.95 ns 0.94 0.11 False
LessThanAnyBenchmark - Duration of single invocation 185.92 ns 169.21 ns 0.91 0.10 False
DivideBenchmark - Duration of single invocation 173.50 ns 154.11 ns 0.89 0.11 False
BitwiseOrOperatorBenchmark - Duration of single invocation 27.30 ns 16.72 ns 0.61 0.24 False
DivisionOperatorBenchmark - Duration of single invocation 172.62 ns 154.55 ns 0.90 0.06 False
AddBenchmark - Duration of single invocation 170.59 ns 136.58 ns 0.80 0.11 False
OneBenchmark - Duration of single invocation 78.94 ns 51.70 ns 0.65 0.08 False
GreaterThanOrEqualBenchmark - Duration of single invocation 168.75 ns 156.03 ns 0.92 0.14 False
BitwiseAndBenchmark - Duration of single invocation 28.76 ns 16.88 ns 0.59 0.23 False
BitwiseAndOperatorBenchmark - Duration of single invocation 27.84 ns 19.10 ns 0.69 0.26 False
BitwiseOrBenchmark - Duration of single invocation 30.14 ns 16.76 ns 0.56 0.31 False
GreaterThanBenchmark - Duration of single invocation 160.94 ns 149.19 ns 0.93 0.11 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 218.75 ns 192.72 ns 0.88 0.15 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Int16>.ConditionalSelectBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 104.03038066491287 < 181.86094147096563.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 104.09332647915875 (T) = (0 -103.87332419702533) / Math.Sqrt((49.131966695139596 / (299)) + (12.045757551586208 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.43913942078280643 = (185.20346775308008 - 103.87332419702533) / 185.20346775308008 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 198.5154461242048 < 216.73364684801348.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 31.984557414245774 (T) = (0 -196.46900113768123) / Math.Sqrt((62.977559537694056 / (299)) + (26.0790925098653 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.15879250096920933 = (233.55593163879996 - 196.46900113768123) / 233.55593163879996 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 185.2227427901462 < 199.4000544654836.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 49.011490013273054 (T) = (0 -182.68546608603023) / Math.Sqrt((42.28203292436923 / (299)) + (11.165924233145002 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.16470565940861043 = (218.70789398224423 - 182.68546608603023) / 218.70789398224423 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 206.21774792838013 < 224.0956957969746.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 19.980810933278605 (T) = (0 -205.15392861613893) / Math.Sqrt((62.43947511911628 / (299)) + (60.46747252287301 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.14346039495825288 = (239.51481917306074 - 205.15392861613893) / 239.51481917306074 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 92.80713704193235 < 123.12987728645824.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 71.234142698149 (T) = (0 -79.78817409255083) / Math.Sqrt((19.42019925243088 / (299)) + (8.912074719736117 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.35943823076619197 = (124.55968795638158 - 79.78817409255083) / 124.55968795638158 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 184.95271578417464 < 186.4206341680849.
IsChangePoint: Marked as a change because one of 9/30/2022 2:28:26 AM, 10/11/2022 1:45:10 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 26.448869816995014 (T) = (0 -181.82711180505544) / Math.Sqrt((56.06693549760865 / (299)) + (12.204349753957471 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10822277150117786 = (203.8929746066011 - 181.82711180505544) / 203.8929746066011 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 169.20608923378012 < 176.24835939033315.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 31.733437585698336 (T) = (0 -167.17932934500774) / Math.Sqrt((45.738786021092814 / (299)) + (10.124188661526441 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.1241778771070163 = (190.8827431679701 - 167.17932934500774) / 190.8827431679701 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 154.10655080260915 < 165.42977874885145.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 40.54848238537436 (T) = (0 -155.2834161820032) / Math.Sqrt((137.80529367598982 / (299)) + (5.931297597346809 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1772385722199762 = (188.7344386099737 - 155.2834161820032) / 188.7344386099737 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.71954754998402 < 25.97890336226258.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 48.1269021890937 (T) = (0 -17.17769085266627) / Math.Sqrt((3.063828124691651 / (299)) + (0.6320528649723719 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.35241205603557985 = (26.525649547314686 - 17.17769085266627) / 26.525649547314686 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 154.54838702450238 < 163.6416484352115.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 51.36917046159259 (T) = (0 -154.46038218911642) / Math.Sqrt((89.81078640574061 / (299)) + (1.7084329610437718 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.167513669848555 = (185.54104325174583 - 154.46038218911642) / 185.54104325174583 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 136.5750889258671 < 159.43969548196122.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 23.38150914305613 (T) = (0 -138.12148413650243) / Math.Sqrt((13.884937729265003 / (299)) + (36.83366061223526 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.16740048726733606 = (165.89186280349327 - 138.12148413650243) / 165.89186280349327 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 51.700087727319215 < 75.08131917604997.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 42.47493705951593 (T) = (0 -52.68770165882693) / Math.Sqrt((9.890648058459485 / (299)) + (7.235225396598273 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.3102661222833786 = (76.38844974999732 - 52.68770165882693) / 76.38844974999732 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 156.03194957719174 < 160.1203549443977.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 28.39257038001559 (T) = (0 -156.48812298757514) / Math.Sqrt((114.80511193247442 / (299)) + (6.287164269569752 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.12472351693976914 = (178.78707587395175 - 156.48812298757514) / 178.78707587395175 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.878859288166563 < 27.328488396766538.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 30.50160672044095 (T) = (0 -17.49818427783328) / Math.Sqrt((1.5439720743209668 / (299)) + (3.6221186824690155 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.3941688737925166 = (28.882940345723583 - 17.49818427783328) / 28.882940345723583 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.096944087725003 < 25.67413958439466.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 56.689535590439604 (T) = (0 -16.99350963830118) / Math.Sqrt((2.433532050917049 / (299)) + (0.5036425274273048 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.356210746034638 = (26.39607532065996 - 16.99350963830118) / 26.39607532065996 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.755462768711002 < 29.108898314933636.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 51.410473911089206 (T) = (0 -17.13658081584358) / Math.Sqrt((2.9854416712581777 / (299)) + (1.313114187057233 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.41358000812590334 = (29.222368018317447 - 17.13658081584358) / 29.222368018317447 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 149.18697756770126 < 151.80617922750827.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 32.62194532554902 (T) = (0 -148.44637196783802) / Math.Sqrt((84.12898376780925 / (299)) + (1.8887793294641115 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.11524365302583646 = (167.78220633908933 - 148.44637196783802) / 167.78220633908933 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 192.71685213324773 < 207.9243228682002.
IsChangePoint: Marked as a change because one of 9/16/2022 6:13:21 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 38.770722662070135 (T) = (0 -191.55218764512964) / Math.Sqrt((151.58707562604016 / (299)) + (14.390001271954608 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1710914176957575 = (231.08964213236044 - 191.55218764512964) / 231.08964213236044 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare 2b07c8c46523b816b1eac5c8af092e52b20a69d2
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LessThanOrEqualAllBenchmark - Duration of single invocation 147.88 ns 127.09 ns 0.86 0.20 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 132.98 ns 115.43 ns 0.87 0.18 False
AddOperatorBenchmark - Duration of single invocation 81.46 ns 69.18 ns 0.85 0.19 False
ConditionalSelectBenchmark - Duration of single invocation 156.78 ns 80.59 ns 0.51 0.26 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 26.82 ns 17.06 ns 0.64 0.27 False
BitwiseOrBenchmark - Duration of single invocation 29.10 ns 16.46 ns 0.57 0.31 False
UnaryNegateOperatorBenchmark - Duration of single invocation 86.49 ns 67.15 ns 0.78 0.22 False
MultiplyBenchmark - Duration of single invocation 84.59 ns 69.92 ns 0.83 0.16 False
OnesComplementBenchmark - Duration of single invocation 70.81 ns 43.16 ns 0.61 0.22 False
NegateBenchmark - Duration of single invocation 88.75 ns 67.33 ns 0.76 0.22 False
OneBenchmark - Duration of single invocation 48.45 ns 30.39 ns 0.63 0.18 False
LessThanAllBenchmark - Duration of single invocation 145.61 ns 124.02 ns 0.85 0.21 False
SubtractBenchmark - Duration of single invocation 85.72 ns 68.16 ns 0.80 0.20 False
OnesComplementOperatorBenchmark - Duration of single invocation 70.91 ns 41.44 ns 0.58 0.21 False
XorBenchmark - Duration of single invocation 28.58 ns 19.57 ns 0.68 0.25 False
AddBenchmark - Duration of single invocation 83.82 ns 68.51 ns 0.82 0.16 False
DivisionOperatorBenchmark - Duration of single invocation 87.17 ns 77.03 ns 0.88 0.17 False
SubtractionOperatorBenchmark - Duration of single invocation 81.68 ns 68.91 ns 0.84 0.19 False
AndNotBenchmark - Duration of single invocation 96.32 ns 57.90 ns 0.60 0.24 False
BitwiseAndOperatorBenchmark - Duration of single invocation 26.78 ns 20.55 ns 0.77 0.27 False
BitwiseOrOperatorBenchmark - Duration of single invocation 29.94 ns 17.25 ns 0.58 0.27 False
BitwiseAndBenchmark - Duration of single invocation 36.96 ns 16.66 ns 0.45 0.23 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Int32>.LessThanOrEqualAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 127.09236734516878 < 140.5352274536962.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 19.93044947109984 (T) = (0 -125.92239971650122) / Math.Sqrt((46.029277868789634 / (299)) + (17.007582518530956 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.12630521160345787 = (144.12630290218587 - 125.92239971650122) / 144.12630290218587 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 115.43481129558133 < 126.37023490754159.
IsChangePoint: Marked as a change because one of 8/4/2022 5:17:15 AM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 16.349135181316303 (T) = (0 -114.51967145855845) / Math.Sqrt((40.0751631193411 / (299)) + (28.003514073511248 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.13382452337620449 = (132.21301520211196 - 114.51967145855845) / 132.21301520211196 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 69.18049686866942 < 77.39026507403076.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 8.972398097064623 (T) = (0 -69.52970312844938) / Math.Sqrt((23.528802161975896 / (299)) + (14.632677949335402 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.09228063597283032 = (76.5982371687823 - 69.52970312844938) / 76.5982371687823 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 80.58793300342596 < 148.18724046367944.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 153.73321659893975 (T) = (0 -80.5528192197703) / Math.Sqrt((41.53617854769577 / (299)) + (2.1247164101265583 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.47269894389289385 = (152.76438058832238 - 80.5528192197703) / 152.76438058832238 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.06310832286028 < 25.55876491694911.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 44.77861597760032 (T) = (0 -17.341887842182892) / Math.Sqrt((3.124470365044395 / (299)) + (0.8885028433024997 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.34654177347832826 = (26.538632675102996 - 17.341887842182892) / 26.538632675102996 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.463063612879004 < 28.047011869340587.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 65.28259491788792 (T) = (0 -16.76720372462708) / Math.Sqrt((4.97750823304047 / (299)) + (0.4520568829915581 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.42588927184224823 = (29.2055223883917 - 16.76720372462708) / 29.2055223883917 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.UnaryNegateOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 67.15301681541631 < 79.07161714527966.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 24.871580656089833 (T) = (0 -66.88428689688453) / Math.Sqrt((31.536653392581325 / (299)) + (2.2989154078416036 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.1417995932864792 = (77.93551060295808 - 66.88428689688453) / 77.93551060295808 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 69.92471541609582 < 81.85301160644963.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 26.84126978167647 (T) = (0 -68.28854026648354) / Math.Sqrt((25.40339954158782 / (299)) + (2.4400420304301806 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1402005846188967 = (79.42380402319169 - 68.28854026648354) / 79.42380402319169 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.16355733318878 < 67.45436423172714.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 70.61535444174058 (T) = (0 -43.35907774504053) / Math.Sqrt((12.594781203254705 / (299)) + (2.3437080598704023 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.3661677772643015 = (68.40781549082085 - 43.35907774504053) / 68.40781549082085 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.NegateBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 67.33436699313175 < 81.49210720263186.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 24.20133792531791 (T) = (0 -66.77682549726886) / Math.Sqrt((33.65575714530704 / (299)) + (2.95080270614105 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.14581302408202343 = (78.17588816020664 - 66.77682549726886) / 78.17588816020664 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.389020396432866 < 45.917796124881235.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 60.498052687641135 (T) = (0 -30.476070333733396) / Math.Sqrt((7.58443554576607 / (299)) + (0.82584825943003 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.31952568157664984 = (44.78651068617261 - 30.476070333733396) / 44.78651068617261 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 124.02190136842566 < 138.25087573220608.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 22.07079948784525 (T) = (0 -121.89715269282325) / Math.Sqrt((48.665518682887985 / (299)) + (14.71363639324387 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.13386726984036593 = (140.73726629676815 - 121.89715269282325) / 140.73726629676815 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 68.1623638053467 < 79.82816120219181.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 26.034171922892646 (T) = (0 -67.46372780909499) / Math.Sqrt((44.80417737818005 / (299)) + (1.454788902904056 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.14898236202081655 = (79.27418281164368 - 67.46372780909499) / 79.27418281164368 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 41.44185708233415 < 68.35189626228723.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 108.94289736149196 (T) = (0 -41.52739335514525) / Math.Sqrt((10.750922171282362 / (299)) + (0.6920748212302651 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.3925380986199623 = (68.36213639209788 - 41.52739335514525) / 68.36213639209788 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.57422411054793 < 28.710380428094968.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 67.68361697953371 (T) = (0 -17.09833800415361) / Math.Sqrt((3.2100059847719153 / (299)) + (0.5476862442395063 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.4107923272716659 = (29.01920459551982 - 17.09833800415361) / 29.01920459551982 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 68.51424783072085 < 79.62949234979897.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 13.00828804879834 (T) = (0 -69.08687928965462) / Math.Sqrt((25.303246969295632 / (299)) + (12.159668663680323 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.12647283421083316 = (79.08956011372555 - 69.08687928965462) / 79.08956011372555 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 77.02813420738072 < 82.77442787329191.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/3/2022 11:00:05 AM, 10/3/2022 8:45:37 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 9.773451409594216 (T) = (0 -77.46067263631402) / Math.Sqrt((19.048834403253732 / (299)) + (15.853312839655757 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.09238759226377075 = (85.34554174894629 - 77.46067263631402) / 85.34554174894629 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 68.90685866324645 < 77.7641324460293.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 18.079207546362507 (T) = (0 -68.04642637851167) / Math.Sqrt((27.434179677483858 / (299)) + (3.6016286493041365 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.11195463396053212 = (76.62494392824459 - 68.04642637851167) / 76.62494392824459 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 57.90246012389708 < 91.54645270382078.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 54.14339059187692 (T) = (0 -58.27638607776367) / Math.Sqrt((19.04024765485459 / (299)) + (8.9371408430962 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.3722628398019897 = (92.83564805910368 - 58.27638607776367) / 92.83564805910368 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.548452709058132 < 25.468362986583223.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 53.05278625799625 (T) = (0 -17.015478050828616) / Math.Sqrt((3.2174374089704023 / (299)) + (0.6083599238988547 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.3597887065384466 = (26.577909862895048 - 17.015478050828616) / 26.577909862895048 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.250646451817474 < 26.231527902268663.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 56.50351202336398 (T) = (0 -16.902923190639918) / Math.Sqrt((3.8077776324374155 / (299)) + (0.43162963186964515 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.3666863520999511 = (26.689655665383004 - 16.902923190639918) / 26.689655665383004 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.660045724665007 < 28.795153265889105.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 37.398887470795316 (T) = (0 -17.402358117633113) / Math.Sqrt((2.171854732181823 / (299)) + (2.299493312274005 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.39934307100474126 = (28.97220905574616 - 17.402358117633113) / 28.97220905574616 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryFormat - Duration of single invocation 273.17 ns 197.58 ns 0.72 0.16 False
TryFormat - Duration of single invocation 365.78 ns 297.73 ns 0.81 0.13 False
TryFormat - Duration of single invocation 200.67 ns 151.82 ns 0.76 0.20 False
ToString - Duration of single invocation 458.17 ns 378.54 ns 0.83 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int32*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Int32.TryFormat(value: 2147483647)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 197.58439577378462 < 263.8218861132783.
IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 34.59022134549735 (T) = (0 -195.09276068382385) / Math.Sqrt((153.1633570247428 / (299)) + (58.539376943198796 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.22000688217546874 = (250.12113084786512 - 195.09276068382385) / 250.12113084786512 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.TryFormat(value: -2147483648)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 297.72947568636033 < 346.4505929383249.
IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 14.641447088314926 (T) = (0 -290.6669761579294) / Math.Sqrt((186.40485205113535 / (299)) + (130.19884591573296 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.10518302895736355 = (324.8340002081606 - 290.6669761579294) / 324.8340002081606 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.TryFormat(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 151.82019862481346 < 189.61428838668678.
IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 11.894689183365397 (T) = (0 -150.60941849285126) / Math.Sqrt((140.24261586274773 / (299)) + (25.127238674565938 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.08545185629585948 = (164.6817825061093 - 150.60941849285126) / 164.6817825061093 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToString(value: -2147483648)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 378.5399792952125 < 429.8586124036972.
IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 16.666918177983288 (T) = (0 -382.5444538583445) / Math.Sqrt((281.01799398410503 / (299)) + (57.64211750780493 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.07015166951826485 = (411.40521665523255 - 382.5444538583445) / 411.40521665523255 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare 2b07c8c46523b816b1eac5c8af092e52b20a69d2
Diff Diff

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AppendPrepend - Duration of single invocation 30.54 μs 27.96 μs 0.92 0.01 False
Prepend - Duration of single invocation 26.62 μs 24.13 μs 0.91 0.05 False
Append - Duration of single invocation 31.88 μs 29.24 μs 0.92 0.04 False
CastToSameType - Duration of single invocation 4.93 μs 4.24 μs 0.86 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Linq.Tests.Perf_Enumerable.AppendPrepend(input: IEnumerable)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.95620936702806 < 29.718391702184743.
IsChangePoint: Marked as a change because one of 7/27/2022 4:00:59 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 21.789514390131618 (T) = (0 -28951.150092951466) / Math.Sqrt((254276.8155017876 / (299)) + (368105.2171830407 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.08305548286735374 = (31573.50259695523 - 28951.150092951466) / 31573.50259695523 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Prepend(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.130852631500122 < 25.465089973763938.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 33.51170583505545 (T) = (0 -24542.664977343782) / Math.Sqrt((237341.18078931782 / (299)) + (144356.60116381405 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09528455961122838 = (27127.496538356172 - 24542.664977343782) / 27127.496538356172 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Append(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.24422486710444 < 30.31599666479168.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 28.042606417791774 (T) = (0 -29711.699228762533) / Math.Sqrt((290713.6498269061 / (299)) + (216009.58509964336 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.08206586712354258 = (32368.00786093152 - 29711.699228762533) / 32368.00786093152 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.CastToSameType(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.240897131872618 < 4.471323115887669.
IsChangePoint: Marked as a change because one of 7/30/2022 2:52:48 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 17.67268302961784 (T) = (0 -4360.150551862527) / Math.Sqrt((25373.442053781037 / (299)) + (12645.418123681877 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.08967526086872876 = (4789.665011217257 - 4360.150551862527) / 4789.665011217257 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Span - Duration of single invocation 6.11 ms 4.20 ms 0.69 0.11 False
ICollection - Duration of single invocation 10.51 ms 8.71 ms 0.83 0.17 False
Queue - Duration of single invocation 10.41 ms 8.52 ms 0.82 0.14 False
Array - Duration of single invocation 10.67 ms 8.44 ms 0.79 0.10 False
ImmutableArray - Duration of single invocation 10.85 ms 8.36 ms 0.77 0.09 False
List - Duration of single invocation 10.52 ms 8.40 ms 0.80 0.15 False
LinkedList - Duration of single invocation 11.21 ms 9.34 ms 0.83 0.11 False
Stack - Duration of single invocation 10.44 ms 8.42 ms 0.81 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsFalse<String>.Span(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.20227015625 < 5.808093610662203.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 103.3407592027267 (T) = (0 -4228226.919022376) / Math.Sqrt((84804111950.86203 / (299)) + (2121787212.2369914 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.31663428794527926 = (6187355.97123989 - 4228226.919022376) / 6187355.97123989 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;String&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.70715520982143 < 9.98685387734375.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 64.03508356706799 (T) = (0 -8461445.855755664) / Math.Sqrt((125714715925.81644 / (299)) + (22725289996.246685 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.21189056611565826 = (10736384.430842146 - 8461445.855755664) / 10736384.430842146 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;String&gt;.Queue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.519591570833333 < 10.244088526197915.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 59.19145992158486 (T) = (0 -8506757.816069668) / Math.Sqrt((96983199165.38431 / (299)) + (28138636364.95734 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.20695663381538218 = (10726724.639279466 - 8506757.816069668) / 10726724.639279466 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;String&gt;.Array(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.444199607291667 < 10.046047770729166.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 79.8548911751379 (T) = (0 -8478937.640400514) / Math.Sqrt((68208752258.84911 / (299)) + (14936356112.17517 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.20839353552767867 = (10711051.54005091 - 8478937.640400514) / 10711051.54005091 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;String&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.359276179166667 < 10.129275598333333.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 67.64750998041322 (T) = (0 -8490226.50941415) / Math.Sqrt((105096818310.265 / (299)) + (19220915062.860798 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.21054046790203074 = (10754479.696826994 - 8490226.50941415) / 10754479.696826994 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;String&gt;.List(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.399289618303571 < 9.998573811157852.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 48.43546576464836 (T) = (0 -8530632.434088828) / Math.Sqrt((102962711098.96107 / (299)) + (47221427162.97882 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.20139282572685763 = (10681888.053225968 - 8530632.434088828) / 10681888.053225968 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;String&gt;.LinkedList(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.341261028124999 < 10.417962146007897.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 49.30280446629442 (T) = (0 -8898014.500350766) / Math.Sqrt((126466410001.28928 / (299)) + (41067587918.805115 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1941095259051034 = (11041220.595570648 - 8898014.500350766) / 11041220.595570648 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;String&gt;.Stack(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.416146260416665 < 10.084478493764022.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 42.86830192975348 (T) = (0 -8551119.19140809) / Math.Sqrt((82594529149.04207 / (299)) + (63975652730.422806 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.20236267605042377 = (10720560.503696617 - 8551119.19140809) / 10720560.503696617 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ExclusiveOrOperatorBenchmark - Duration of single invocation 29.05 ns 17.11 ns 0.59 0.29 False
SubtractionOperatorBenchmark - Duration of single invocation 56.46 ns 46.36 ns 0.82 0.27 False
BitwiseAndBenchmark - Duration of single invocation 30.65 ns 17.07 ns 0.56 0.26 False
AddOperatorBenchmark - Duration of single invocation 57.22 ns 45.75 ns 0.80 0.24 False
BitwiseAndOperatorBenchmark - Duration of single invocation 26.91 ns 16.87 ns 0.63 0.24 False
MultiplyOperatorBenchmark - Duration of single invocation 55.15 ns 45.87 ns 0.83 0.19 False
LessThanAnyBenchmark - Duration of single invocation 79.18 ns 69.88 ns 0.88 0.27 False
OnesComplementBenchmark - Duration of single invocation 64.49 ns 34.57 ns 0.54 0.18 False
ConditionalSelectBenchmark - Duration of single invocation 144.87 ns 115.83 ns 0.80 0.24 False
LessThanOrEqualAllBenchmark - Duration of single invocation 134.67 ns 98.04 ns 0.73 0.30 False
BitwiseOrBenchmark - Duration of single invocation 28.29 ns 16.77 ns 0.59 0.22 False
UnaryNegateOperatorBenchmark - Duration of single invocation 56.56 ns 51.45 ns 0.91 0.25 False
XorBenchmark - Duration of single invocation 33.57 ns 16.86 ns 0.50 0.27 False
NegateBenchmark - Duration of single invocation 57.71 ns 46.13 ns 0.80 0.26 False
DivisionOperatorBenchmark - Duration of single invocation 55.93 ns 47.10 ns 0.84 0.21 False
DivideBenchmark - Duration of single invocation 59.50 ns 46.84 ns 0.79 0.21 False
BitwiseOrOperatorBenchmark - Duration of single invocation 26.49 ns 17.11 ns 0.65 0.25 False
AddBenchmark - Duration of single invocation 57.83 ns 45.72 ns 0.79 0.29 False
LessThanAllBenchmark - Duration of single invocation 122.47 ns 97.51 ns 0.80 0.26 False
OnesComplementOperatorBenchmark - Duration of single invocation 64.68 ns 31.43 ns 0.49 0.23 False
SubtractBenchmark - Duration of single invocation 58.43 ns 45.98 ns 0.79 0.24 False
AndNotBenchmark - Duration of single invocation 88.12 ns 49.01 ns 0.56 0.28 False
LessThanOrEqualBenchmark - Duration of single invocation 70.86 ns 51.48 ns 0.73 0.28 False
MultiplyBenchmark - Duration of single invocation 58.34 ns 45.90 ns 0.79 0.23 False
AbsBenchmark - Duration of single invocation 56.49 ns 44.03 ns 0.78 0.23 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Double>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.10710615607582 < 27.165866206114142.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 16.013240411942938 (T) = (0 -18.219316479465853) / Math.Sqrt((1.5385880854190723 / (299)) + (7.46974980527686 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.31427991803516087 = (26.56961194320114 - 18.219316479465853) / 26.56961194320114 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 46.363094558313314 < 53.36305163544436.
IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 15.3896814910778 (T) = (0 -46.36634600447913) / Math.Sqrt((7.291439395219362 / (299)) + (6.225827258426138 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.14153575413593703 = (54.01080619008237 - 46.36634600447913) / 54.01080619008237 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.067547304221094 < 27.556970695337508.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 86.87268309273468 (T) = (0 -16.85778962136815) / Math.Sqrt((2.308063583283262 / (299)) + (0.3094527376106431 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.4192290841264047 = (29.026573405472053 - 16.85778962136815) / 29.026573405472053 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 45.749055723484894 < 53.33025194425234.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 16.189299546167437 (T) = (0 -46.62293574386466) / Math.Sqrt((8.775907071146683 / (299)) + (4.72317487116575 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.13565396136664673 = (53.94012774973986 - 46.62293574386466) / 53.94012774973986 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.874275777776248 < 25.598842307459794.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 34.650689438552746 (T) = (0 -17.299232380967695) / Math.Sqrt((1.847035383331567 / (299)) + (1.701188150412599 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.34505784563327907 = (26.41337447227646 - 17.299232380967695) / 26.41337447227646 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 45.8731763879555 < 52.481260328298.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 33.52760176146045 (T) = (0 -45.653506642206096) / Math.Sqrt((5.999502556408722 / (299)) + (1.0937479909121701 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.15151861485301768 = (53.806138168013604 - 45.653506642206096) / 53.806138168013604 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 69.87892935318482 < 76.04619882591659.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 13.360812036957023 (T) = (0 -69.66482555781816) / Math.Sqrt((17.58410387583034 / (299)) + (8.407536889404845 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.10599628014692886 = (77.92453656598602 - 69.66482555781816) / 77.92453656598602 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.56824564633277 < 61.65297010778745.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 121.99720038293164 (T) = (0 -33.35994141342257) / Math.Sqrt((7.714882421573653 / (299)) + (0.8576427122541844 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.4699707112133953 = (62.939807514776085 - 33.35994141342257) / 62.939807514776085 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 115.82888767182422 < 137.96005162159784.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 28.390205108678746 (T) = (0 -75.08962620454895) / Math.Sqrt((39.22526733786518 / (299)) + (146.74038222668102 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.48582430894966633 = (146.0388491940555 - 75.08962620454895) / 146.0388491940555 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 98.03916925909637 < 120.78491217775418.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 26.844322113035844 (T) = (0 -98.63061650460716) / Math.Sqrt((48.658920983725295 / (299)) + (18.3449735272633 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.19531611798048312 = (122.57063762365128 - 98.63061650460716) / 122.57063762365128 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.77365515750315 < 27.69504958900764.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 71.40665036185895 (T) = (0 -17.15968882809529) / Math.Sqrt((2.956284513580909 / (299)) + (0.46961175074392936 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.40734117774567424 = (28.953738953592442 - 17.15968882809529) / 28.953738953592442 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.UnaryNegateOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 51.452178011121035 < 54.55631269673051.
IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 21.24070334799574 (T) = (0 -45.11025791292969) / Math.Sqrt((11.482440410900713 / (299)) + (5.614257392368405 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1870915616738325 = (55.49242176131756 - 45.11025791292969) / 55.49242176131756 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.863638823097027 < 27.633662829904726.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 38.873949514628734 (T) = (0 -17.412844630647996) / Math.Sqrt((4.040272239580382 / (299)) + (2.045562642649477 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.4001343899510964 = (29.02790948330648 - 17.412844630647996) / 29.02790948330648 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.NegateBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 46.133581955557474 < 54.477509505037105.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 29.231197808503428 (T) = (0 -45.29113889539424) / Math.Sqrt((8.823052619259718 / (299)) + (2.543540226567621 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1829401226919104 = (55.4318479627366 - 45.29113889539424) / 55.4318479627366 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 47.102860281396495 < 54.42309481450653.
IsChangePoint: Marked as a change because one of 9/27/2022 7:01:57 AM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 20.108344250848862 (T) = (0 -47.20863837294483) / Math.Sqrt((8.387999735016056 / (299)) + (3.3492829445664976 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.14245503739671206 = (55.05091911406189 - 47.20863837294483) / 55.05091911406189 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 46.84361757190513 < 56.2494486863374.
IsChangePoint: Marked as a change because one of 9/27/2022 4:04:11 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 20.656571627186157 (T) = (0 -46.97319205503739) / Math.Sqrt((11.957876532748545 / (299)) + (5.720942115713242 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.18316879709993183 = (57.506608327723406 - 46.97319205503739) / 57.506608327723406 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.111053688601906 < 25.619504398956572.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 65.83489946568133 (T) = (0 -16.98304539878068) / Math.Sqrt((3.0969088153130313 / (299)) + (0.3283131058133585 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.36336745199199877 = (26.676369990695534 - 16.98304539878068) / 26.676369990695534 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 45.72049858297049 < 55.21479137537449.
IsChangePoint: Marked as a change because one of 9/29/2022 2:35:34 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 20.389216143185546 (T) = (0 -46.14508238755108) / Math.Sqrt((12.121248981119308 / (299)) + (6.255919292163752 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.18501185404077436 = (56.62055652752984 - 46.14508238755108) / 56.62055652752984 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 97.51388655303835 < 116.65780561883324.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 33.06761498567219 (T) = (0 -98.02939998906615) / Math.Sqrt((28.384352438492176 / (299)) + (12.853884343245106 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.20568838041969537 = (123.41428423376527 - 98.02939998906615) / 123.41428423376527 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.433815218941284 < 61.62253817860524.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 135.18183205645482 (T) = (0 -31.513017657504154) / Math.Sqrt((13.208195448975522 / (299)) + (0.2982752840031805 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.50296340107719 = (63.401805271080526 - 31.513017657504154) / 63.401805271080526 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 45.97712146742371 < 55.08558107482213.
IsChangePoint: Marked as a change because one of 9/29/2022 5:24:34 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 36.98832453100957 (T) = (0 -45.58099104707928) / Math.Sqrt((8.652274728090969 / (299)) + (1.5379248192058153 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.19028301199340372 = (56.29249691215316 - 45.58099104707928) / 56.29249691215316 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.012235588097006 < 83.55815885099148.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 76.87000881043373 (T) = (0 -48.69579668781433) / Math.Sqrt((15.211698787921293 / (299)) + (5.512241068100284 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.44357650165592266 = (87.51570850751914 - 48.69579668781433) / 87.51570850751914 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 51.47821196977084 < 58.26198309577507.
IsChangePoint: Marked as a change because one of 7/11/2022 9:49:18 PM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 12.673680905652613 (T) = (0 -51.67986983498858) / Math.Sqrt((10.138761062494037 / (299)) + (6.773085663454205 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.11743700538619357 = (58.55657913416453 - 51.67986983498858) / 58.55657913416453 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 45.8985939381759 < 55.113126060837146.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 42.363679977502336 (T) = (0 -45.70930103950323) / Math.Sqrt((8.325512795105983 / (299)) + (0.8696747719811314 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.18827564162554514 = (56.311358120433745 - 45.70930103950323) / 56.311358120433745 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.AbsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 44.03492594822194 < 53.49497062351817.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 26.86465964008094 (T) = (0 -44.620656297177796) / Math.Sqrt((9.18696730820338 / (299)) + (3.1021053889856107 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.18910738277303804 = (55.02659088174785 - 44.620656297177796) / 55.02659088174785 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare 2b07c8c46523b816b1eac5c8af092e52b20a69d2
Diff Diff

Improvements in System.Numerics.Tests.Perf_Matrix4x4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DecomposeBenchmark - Duration of single invocation 743.84 ns 629.95 ns 0.85 0.03 False
CreateWorldBenchmark - Duration of single invocation 324.69 ns 216.16 ns 0.67 0.18 False
CreateLookAtBenchmark - Duration of single invocation 352.33 ns 255.96 ns 0.73 0.13 False
CreateConstrainedBillboardBenchmark - Duration of single invocation 430.04 ns 320.16 ns 0.74 0.15 False
CreateOrthographicOffCenterBenchmark - Duration of single invocation 47.96 ns 43.59 ns 0.91 0.21 False
TranslationBenchmark - Duration of single invocation 27.52 ns 24.13 ns 0.88 0.31 False
CreateScaleFromVectorBenchmark - Duration of single invocation 43.08 ns 34.73 ns 0.81 0.35 False
CreateBillboardBenchmark - Duration of single invocation 338.25 ns 295.05 ns 0.87 0.12 False
CreateTranslationFromVectorBenchmark - Duration of single invocation 43.01 ns 35.55 ns 0.83 0.27 False
CreateTranslationFromScalarXYZ - Duration of single invocation 23.94 ns 20.79 ns 0.87 0.37 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_Matrix4x4.DecomposeBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 629.950043394085 < 680.2502925815237.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 35.21380177859456 (T) = (0 -622.5319826072692) / Math.Sqrt((183.95278456761787 / (299)) + (178.15426713763156 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1299971009057527 = (715.5516185697566 - 622.5319826072692) / 715.5516185697566 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateWorldBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 216.16454459841762 < 308.6899377939924.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 51.06406580116921 (T) = (0 -217.02032735007194) / Math.Sqrt((227.52372627006042 / (299)) + (75.41979367672933 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.3042651194425063 = (311.92963500144356 - 217.02032735007194) / 311.92963500144356 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateLookAtBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 255.955636766433 < 343.5988392708618.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 46.34594833924935 (T) = (0 -253.00271622503868) / Math.Sqrt((313.69831103870206 / (299)) + (75.59002432923438 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.26705997058275815 = (345.1888368359418 - 253.00271622503868) / 345.1888368359418 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateConstrainedBillboardBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 320.1642915250492 < 411.7662172285151.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 63.61090714297005 (T) = (0 -323.92821074447716) / Math.Sqrt((120.96573348764196 / (299)) + (74.31973593952517 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.25866822082919916 = (436.9544377374452 - 323.92821074447716) / 436.9544377374452 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateOrthographicOffCenterBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.58796236762251 < 45.45334296475574.
IsChangePoint: Marked as a change because one of 7/5/2022 7:31:17 AM, 7/5/2022 7:01:28 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 16.781203986296912 (T) = (0 -42.94014702924255) / Math.Sqrt((4.0047206295341935 / (299)) + (1.2675474073971986 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.09431780087806993 = (47.411936627299895 - 42.94014702924255) / 47.411936627299895 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.TranslationBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.13026955197612 < 26.142234199205053.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 29.49014385671715 (T) = (0 -22.90733896618962) / Math.Sqrt((2.8162153486819426 / (299)) + (0.9354168068040722 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.21274481547805035 = (29.097730210693754 - 22.90733896618962) / 29.097730210693754 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromVectorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.73474010755102 < 40.98957295830598.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 15.846047512696229 (T) = (0 -35.81789946240762) / Math.Sqrt((4.665279820071374 / (299)) + (6.9856059081607125 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.18812479793848533 = (44.1174941314364 - 35.81789946240762) / 44.1174941314364 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateBillboardBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 295.04575465984374 < 336.62246055230713.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 35.88159467615203 (T) = (0 -270.7853241412821) / Math.Sqrt((78.9498617810625 / (299)) + (124.71271409603271 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.2234971313897173 = (348.72417744690387 - 270.7853241412821) / 348.72417744690387 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateTranslationFromVectorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 35.54858977193889 < 41.06618466731671.
IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 14.022116978275971 (T) = (0 -35.652996447703885) / Math.Sqrt((3.1946979444910992 / (299)) + (8.539060202684363 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.18634863165839444 = (43.818517162175084 - 35.652996447703885) / 43.818517162175084 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateTranslationFromScalarXYZ

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.789370877168775 < 23.163404596164185.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 17.970559800841375 (T) = (0 -20.898936972203863) / Math.Sqrt((1.5159394692416217 / (299)) + (0.41171649491886647 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.11205543382210313 = (23.536308197889042 - 20.898936972203863) / 23.536308197889042 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryFormat - Duration of single invocation 271.55 ns 193.94 ns 0.71 0.15 False
TryFormat - Duration of single invocation 193.10 ns 154.64 ns 0.80 0.20 False
ToString - Duration of single invocation 6.14 ns 2.84 ns 0.46 0.59 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt32*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_UInt32.TryFormat(value: 4294967295)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 193.9436911827612 < 262.3526110581937.
IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 41.670600332655106 (T) = (0 -193.1327675388603) / Math.Sqrt((157.5450877964773 / (299)) + (33.24772696429351 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.22027368743680104 = (247.69302308649017 - 193.1327675388603) / 247.69302308649017 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryFormat(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 154.64025808550977 < 189.03446149446063.
IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 8.41315100669035 (T) = (0 -150.0918550252714) / Math.Sqrt((166.70148616314833 / (299)) + (40.2360795028567 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.07425598966288507 = (162.13105712735273 - 150.0918550252714) / 162.13105712735273 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.ToString(value: 0)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.8449698509800787 < 5.834032362845779.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 22.008460411092063 (T) = (0 -3.5288963743997006) / Math.Sqrt((0.7854785026453375 / (299)) + (0.3115438503860529 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.4298162811155978 = (6.1890514539843275 - 3.5288963743997006) / 6.1890514539843275 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in System.Threading.Channels.Tests.BoundedChannelPerfTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadAsyncThenWriteAsync - Duration of single invocation 1.38 μs 1.28 μs 0.93 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Channels.Tests.BoundedChannelPerfTests*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Threading.Channels.Tests.BoundedChannelPerfTests.ReadAsyncThenWriteAsync

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2833698885821063 < 1.3070756768921454.
IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 12.487165606870432 (T) = (0 -1275.026471616315) / Math.Sqrt((642.9342483837248 / (299)) + (735.436963279434 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.05041597467178434 = (1342.7210627049174 - 1275.026471616315) / 1342.7210627049174 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in System.Threading.Tests.Perf_Thread

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CurrentThread - Duration of single invocation 11.66 ns 4.68 ns 0.40 0.47 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tests.Perf_Thread*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Threading.Tests.Perf_Thread.CurrentThread

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.681165005208468 < 10.618943481896876.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 21.701312500972563 (T) = (0 -5.50178899924083) / Math.Sqrt((4.044960706369663 / (299)) + (1.6087317325579398 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.5160971575826757 = (11.369614966005951 - 5.50178899924083) / 11.369614966005951 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 33.35 ns 23.45 ns 0.70 0.23 False
ToString - Duration of single invocation 554.09 ns 430.81 ns 0.78 0.08 False
TryParse - Duration of single invocation 429.79 ns 400.86 ns 0.93 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt64*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_UInt64.ToString(value: 0)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.449707552022957 < 30.665434618969584.
IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 55.58400443273513 (T) = (0 -21.322354657758083) / Math.Sqrt((6.857812433575172 / (299)) + (1.8626893957672046 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.444967060999897 = (38.4163770463254 - 21.322354657758083) / 38.4163770463254 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.ToString(value: 18446744073709551615)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 430.80938287029545 < 530.9394076002479.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 74.27344124929355 (T) = (0 -439.5543493893089) / Math.Sqrt((148.341146454457 / (299)) + (49.841536272537695 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.20791316714587701 = (554.9320240628981 - 439.5543493893089) / 554.9320240628981 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.TryParse(value: "18446744073709551615")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 400.8638364085123 < 406.2092431360426.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 26.033903742230425 (T) = (0 -389.112323582623) / Math.Sqrt((171.4356456638281 / (299)) + (35.37687294520628 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.08313841729152416 = (424.395929463155 - 389.112323582623) / 424.395929463155 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in SIMD.RayTracerBench

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Bench - Duration of single invocation 2.21 secs 2.02 secs 0.91 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'SIMD.RayTracerBench*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

SIMD.RayTracerBench.Bench

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.0160404895384616 < 2.0858482527466666.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 48.086616640559626 (T) = (0 -1933062831.478849) / Math.Sqrt((972797768982618.6 / (299)) + (804263614384201.4 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.12163072050313427 = (2200740481.9372973 - 1933062831.478849) / 2200740481.9372973 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in System.Tests.Perf_Type

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
op_Equality - Duration of single invocation 19.83 ns 11.16 ns 0.56 0.24 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Type*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Type.op_Equality

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.158557984619632 < 15.731254754355321.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 42.325638154399094 (T) = (0 -11.138912009004171) / Math.Sqrt((0.6572027729282717 / (299)) + (0.4706238763318798 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.35122809686624284 = (17.169226896540962 - 11.138912009004171) / 17.169226896540962 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in System.Numerics.Tests.Perf_Vector4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateFromScalarXYZWBenchmark - Duration of single invocation 16.69 ns 11.37 ns 0.68 0.33 False
AddFunctionBenchmark - Duration of single invocation 24.95 ns 21.23 ns 0.85 0.25 False
DivideOperatorBenchmark - Duration of single invocation 27.78 ns 21.09 ns 0.76 0.25 False
OneBenchmark - Duration of single invocation 32.17 ns 8.69 ns 0.27 0.27 False
MultiplyByScalarBenchmark - Duration of single invocation 49.66 ns 26.59 ns 0.54 0.18 False
UnitYBenchmark - Duration of single invocation 24.59 ns 17.65 ns 0.72 0.32 False
SubtractFunctionBenchmark - Duration of single invocation 28.61 ns 20.99 ns 0.73 0.25 False
SquareRootBenchmark - Duration of single invocation 30.22 ns 26.39 ns 0.87 0.20 False
MultiplyByScalarOperatorBenchmark - Duration of single invocation 49.28 ns 26.24 ns 0.53 0.17 False
TransformByQuaternionBenchmark - Duration of single invocation 110.70 ns 88.71 ns 0.80 0.14 False
NegateOperatorBenchmark - Duration of single invocation 28.30 ns 25.74 ns 0.91 0.25 False
MultiplyOperatorBenchmark - Duration of single invocation 27.39 ns 20.63 ns 0.75 0.26 False
CreateFromScalar - Duration of single invocation 23.80 ns 9.97 ns 0.42 0.30 False
AddOperatorBenchmark - Duration of single invocation 25.96 ns 22.01 ns 0.85 0.25 False
UnitZBenchmark - Duration of single invocation 24.47 ns 17.48 ns 0.71 0.25 False
DivideByScalarOperatorBenchmark - Duration of single invocation 52.93 ns 26.84 ns 0.51 0.17 False
MinBenchmark - Duration of single invocation 36.89 ns 28.87 ns 0.78 0.24 False
LerpBenchmark - Duration of single invocation 136.27 ns 79.93 ns 0.59 0.15 False
AbsBenchmark - Duration of single invocation 42.40 ns 11.62 ns 0.27 0.32 False
NormalizeBenchmark - Duration of single invocation 66.86 ns 42.98 ns 0.64 0.17 False
UnitXBenchmark - Duration of single invocation 25.18 ns 17.58 ns 0.70 0.25 False
TransformVector2ByQuaternionBenchmark - Duration of single invocation 95.65 ns 79.20 ns 0.83 0.16 False
TransformVector3ByQuaternionBenchmark - Duration of single invocation 110.06 ns 90.02 ns 0.82 0.19 False
ClampBenchmark - Duration of single invocation 73.82 ns 63.50 ns 0.86 0.24 False
UnitWBenchmark - Duration of single invocation 24.85 ns 17.23 ns 0.69 0.23 False
SubtractOperatorBenchmark - Duration of single invocation 28.62 ns 21.01 ns 0.73 0.21 False
NegateBenchmark - Duration of single invocation 27.16 ns 20.24 ns 0.75 0.25 False
TransformByMatrix4x4Benchmark - Duration of single invocation 69.82 ns 62.87 ns 0.90 0.17 False
MultiplyFunctionBenchmark - Duration of single invocation 26.88 ns 21.15 ns 0.79 0.23 False
CreateFromVector3WithScalarBenchmark - Duration of single invocation 27.47 ns 12.47 ns 0.45 0.28 False
MaxBenchmark - Duration of single invocation 42.51 ns 30.26 ns 0.71 0.23 False
CreateFromVector2WithScalarBenchmark - Duration of single invocation 27.89 ns 12.20 ns 0.44 0.22 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Vector4*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_Vector4.CreateFromScalarXYZWBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.366633230686363 < 16.114123284435845.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 25.11039655516978 (T) = (0 -10.874667703563455) / Math.Sqrt((1.4470681244691719 / (299)) + (2.1919374800358056 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.3957089544720245 = (17.99574523574504 - 10.874667703563455) / 17.99574523574504 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.AddFunctionBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.226412456340313 < 25.32584134307624.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 36.57615987101815 (T) = (0 -21.380871082303766) / Math.Sqrt((2.4586724452946833 / (299)) + (0.615491317243146 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.24240488283905645 = (28.222028624508166 - 21.380871082303766) / 28.222028624508166 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.DivideOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.088304581820815 < 26.499767335787034.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 50.37961825195748 (T) = (0 -21.286093640288346) / Math.Sqrt((2.7675227464009424 / (299)) + (0.3026424882967725 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.25294077989508984 = (28.49318108583028 - 21.286093640288346) / 28.49318108583028 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.69017012054346 < 30.780410040518944.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 207.22293756125583 (T) = (0 -8.925422392660623) / Math.Sqrt((2.1417274555998174 / (299)) + (0.12797029165718507 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.7169520618904781 = (31.53325352685325 - 8.925422392660623) / 31.53325352685325 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.MultiplyByScalarBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.591466538839445 < 47.46974118266794.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 96.8876079671267 (T) = (0 -26.28906959457337) / Math.Sqrt((6.240234874445797 / (299)) + (1.275287073851608 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.48712587760380777 = (51.258327231930835 - 26.28906959457337) / 51.258327231930835 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.UnitYBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.6492144870391 < 23.352195345069855.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 28.28449597622116 (T) = (0 -17.308321830302518) / Math.Sqrt((1.7658212941070457 / (299)) + (1.9617477086357855 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.31054042166370727 = (25.104186487724974 - 17.308321830302518) / 25.104186487724974 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.SubtractFunctionBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.985377388634824 < 26.83176587054639.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 55.56210735143912 (T) = (0 -21.123934399409002) / Math.Sqrt((3.708369541938452 / (299)) + (0.2276706881286739 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.2737359019163144 = (29.085747808746763 - 21.123934399409002) / 29.085747808746763 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.SquareRootBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.386945778574493 < 29.21417233624541.
IsChangePoint: Marked as a change because one of 8/1/2022 4:16:36 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 42.419189562730416 (T) = (0 -25.532915380454476) / Math.Sqrt((2.420299461808063 / (299)) + (0.2953283896961456 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.18812595786286296 = (31.44935550992971 - 25.532915380454476) / 31.44935550992971 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.MultiplyByScalarOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.236311501841083 < 46.6718199025978.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 62.746288918783314 (T) = (0 -27.120966058419725) / Math.Sqrt((5.821437228504062 / (299)) + (3.472016665564839 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.470966024833062 = (51.26507432695912 - 27.120966058419725) / 51.26507432695912 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.TransformByQuaternionBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 88.70818793704315 < 104.02672786959403.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 20.694752699188168 (T) = (0 -90.13024189646943) / Math.Sqrt((30.99228103818867 / (299)) + (3.9045701142808884 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.10169673077840867 = (100.3338682876777 - 90.13024189646943) / 100.3338682876777 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.NegateOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.74393590991387 < 26.729093007223433.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 17.17672804661388 (T) = (0 -20.980978285181074) / Math.Sqrt((2.0806817598154432 / (299)) + (4.247002554167399 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.2525340253871993 = (28.069476066853152 - 20.980978285181074) / 28.069476066853152 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.630097822372065 < 26.029426176072082.
IsChangePoint: Marked as a change because one of 8/1/2022 8:26:50 AM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 42.11411157712414 (T) = (0 -21.173672365925192) / Math.Sqrt((2.9674313504838334 / (299)) + (0.47154330242868864 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.2499148443819017 = (28.228358083526253 - 21.173672365925192) / 28.228358083526253 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.CreateFromScalar

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.967136277115454 < 22.785045649898915.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 52.103775695737774 (T) = (0 -8.913019923720144) / Math.Sqrt((1.4629483583328446 / (299)) + (2.3606106131212132 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.6397991176512812 = (24.74458103934138 - 8.913019923720144) / 24.74458103934138 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.014471437211544 < 25.70240081070752.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 35.62657506577687 (T) = (0 -21.23491233795314) / Math.Sqrt((2.3837160811437887 / (299)) + (0.8451163728521404 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.24952166171097687 = (28.295170232848456 - 21.23491233795314) / 28.295170232848456 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.UnitZBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.477678870842897 < 23.340791625872416.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 40.477238707421414 (T) = (0 -17.7887221811924) / Math.Sqrt((1.1543237490142817 / (299)) + (0.711175557411673 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.29027461645999353 = (25.064232721203876 - 17.7887221811924) / 25.064232721203876 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.DivideByScalarOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.83571027947524 < 49.96876070189856.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 64.15235924486241 (T) = (0 -26.757096922828556) / Math.Sqrt((5.9165112641599995 / (299)) + (3.5830723697206284 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.48354251327233605 = (51.808905109236974 - 26.757096922828556) / 51.808905109236974 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.MinBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.866395670404152 < 35.65613785530395.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 16.91815914765895 (T) = (0 -29.793317882793513) / Math.Sqrt((4.337707254021574 / (299)) + (6.67523216323088 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.22511524649436007 = (38.4487083376027 - 29.793317882793513) / 38.4487083376027 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.LerpBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 79.92556623635066 < 129.38276935472618.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 68.69896591812265 (T) = (0 -80.10055772753185) / Math.Sqrt((17.137958063210796 / (299)) + (18.770963791717463 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.4266089271219151 = (139.6962065095892 - 80.10055772753185) / 139.6962065095892 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.AbsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.623477072969195 < 39.67078528156521.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 47.373226714610134 (T) = (0 -19.680583896605683) / Math.Sqrt((15.706064725316834 / (299)) + (5.223050914767164 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.5517316812821119 = (43.90357978653273 - 19.680583896605683) / 43.90357978653273 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.NormalizeBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 42.982871863476674 < 62.57620190741453.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 75.65097261798364 (T) = (0 -42.873333917712486) / Math.Sqrt((5.492040620476782 / (299)) + (1.9325547935561513 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.342804933610252 = (65.23684688205884 - 42.873333917712486) / 65.23684688205884 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.UnitXBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.582525204874567 < 23.288708720159093.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 33.76208380988141 (T) = (0 -18.011222346282313) / Math.Sqrt((0.9763223703421409 / (299)) + (1.05050728544111 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.27795172860239564 = (24.944623593405463 - 18.011222346282313) / 24.944623593405463 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.TransformVector2ByQuaternionBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 79.20238068133173 < 93.07961944351838.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 9.101741540024703 (T) = (0 -78.27102383145943) / Math.Sqrt((37.98062601319659 / (299)) + (21.111257336470903 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.10127083864124097 = (87.09078017800607 - 78.27102383145943) / 87.09078017800607 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.TransformVector3ByQuaternionBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 90.01987901305013 < 105.0478405918412.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 10.646266973041007 (T) = (0 -91.06099111832067) / Math.Sqrt((32.376926008110864 / (299)) + (18.116971541799707 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.0922365016763047 = (100.31356326452514 - 91.06099111832067) / 100.31356326452514 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.ClampBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 63.503181813408936 < 70.1479943563412.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 18.428649406527956 (T) = (0 -60.092752042204765) / Math.Sqrt((12.366596644266172 / (299)) + (18.020313452561517 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.20811914400153325 = (75.88610279817286 - 60.092752042204765) / 75.88610279817286 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.UnitWBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.22630092496844 < 23.354396986035695.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 42.26515742761391 (T) = (0 -17.851221667029264) / Math.Sqrt((1.3761789126287265 / (299)) + (0.6768831257533265 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.28655021378922113 = (25.02099238383592 - 17.851221667029264) / 25.02099238383592 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.SubtractOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.00910716400062 < 27.01082631537377.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 30.83718687372434 (T) = (0 -21.43231993710202) / Math.Sqrt((1.3453001755342098 / (299)) + (1.2643867766318824 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.25598624826186683 = (28.806349193187287 - 21.43231993710202) / 28.806349193187287 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.NegateBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.244338092438554 < 26.27855748253456.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 23.708212335593544 (T) = (0 -20.780683004391047) / Math.Sqrt((3.1910099454263574 / (299)) + (2.122540325966693 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.26073918435400445 = (28.110083159530184 - 20.780683004391047) / 28.110083159530184 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.TransformByMatrix4x4Benchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 62.87408698800879 < 66.37521333955142.
IsChangePoint: Marked as a change because one of 7/14/2022 1:35:19 AM, 7/14/2022 2:24:36 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 15.015166579983596 (T) = (0 -64.62461376108367) / Math.Sqrt((14.780971052130626 / (299)) + (3.384059335550191 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.08963728918581298 = (70.98776453979134 - 64.62461376108367) / 70.98776453979134 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.MultiplyFunctionBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.151769673268536 < 25.461827993694754.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 46.15861161876238 (T) = (0 -21.065406325292933) / Math.Sqrt((2.294200244654401 / (299)) + (0.4348536720913427 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.25255470729445717 = (28.183208230587763 - 21.065406325292933) / 28.183208230587763 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.CreateFromVector3WithScalarBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.473610352623211 < 26.26963271404422.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 38.29218563547275 (T) = (0 -12.814753314761779) / Math.Sqrt((1.909934842339512 / (299)) + (4.384450849717393 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.5510845709059852 = (28.546030018669796 - 12.814753314761779) / 28.546030018669796 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.25926988010565 < 35.644719355974345.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 26.605826925898366 (T) = (0 -29.59894058850697) / Math.Sqrt((3.8634434254825445 / (299)) + (2.9704741780952477 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.23388646442123812 = (38.6351881462927 - 29.59894058850697) / 38.6351881462927 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.CreateFromVector2WithScalarBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.202058847389548 < 26.472846920939883.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 177.82390617733827 (T) = (0 -12.108789363100916) / Math.Sqrt((1.293524281821966 / (299)) + (0.12358845107440994 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.5808363908095475 = (28.887978578310044 - 12.108789363100916) / 28.887978578310044 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare 2b07c8c46523b816b1eac5c8af092e52b20a69d2
Diff Diff

Improvements in System.Numerics.Tests.Perf_Vector3

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DivideByScalarBenchmark - Duration of single invocation 51.09 ns 21.78 ns 0.43 0.26 False
MaxBenchmark - Duration of single invocation 30.15 ns 24.79 ns 0.82 0.17 False
MultiplyFunctionBenchmark - Duration of single invocation 22.81 ns 18.79 ns 0.82 0.22 False
MultiplyByScalarBenchmark - Duration of single invocation 42.27 ns 22.57 ns 0.53 0.17 False
SubtractOperatorBenchmark - Duration of single invocation 24.66 ns 17.68 ns 0.72 0.27 False
AbsBenchmark - Duration of single invocation 33.67 ns 15.82 ns 0.47 0.20 False
AddFunctionBenchmark - Duration of single invocation 24.11 ns 17.88 ns 0.74 0.28 False
MultiplyByScalarOperatorBenchmark - Duration of single invocation 41.80 ns 22.04 ns 0.53 0.19 False
CrossBenchmark - Duration of single invocation 33.71 ns 30.37 ns 0.90 0.19 False
CreateFromScalarXYZBenchmark - Duration of single invocation 14.38 ns 9.29 ns 0.65 0.34 False
CreateFromScalar - Duration of single invocation 22.28 ns 12.05 ns 0.54 0.34 False
DivideByVector3Benchmark - Duration of single invocation 22.97 ns 17.99 ns 0.78 0.29 False
ClampBenchmark - Duration of single invocation 60.72 ns 49.63 ns 0.82 0.20 False
CreateFromVector2WithScalarBenchmark - Duration of single invocation 26.21 ns 11.39 ns 0.43 0.25 False
UnitXBenchmark - Duration of single invocation 20.30 ns 14.45 ns 0.71 0.33 False
NegateOperatorBenchmark - Duration of single invocation 23.23 ns 16.89 ns 0.73 0.28 False
MinBenchmark - Duration of single invocation 30.91 ns 24.81 ns 0.80 0.23 False
OneBenchmark - Duration of single invocation 29.21 ns 8.23 ns 0.28 0.30 False
LerpBenchmark - Duration of single invocation 116.20 ns 52.98 ns 0.46 0.15 False
MultiplyOperatorBenchmark - Duration of single invocation 23.13 ns 18.09 ns 0.78 0.27 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Vector3*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_Vector3.DivideByScalarBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.777959941737844 < 42.99836697250873.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 59.519034101464534 (T) = (0 -21.48386788403749) / Math.Sqrt((5.5236148568986065 / (299)) + (3.7398341927599743 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.5192867203664497 = (44.691646339403675 - 21.48386788403749) / 44.691646339403675 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.78905960141875 < 28.613322234267404.
IsChangePoint: Marked as a change because one of 8/1/2022 8:26:50 AM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 30.3250028195556 (T) = (0 -24.24586868805566) / Math.Sqrt((2.269831968140861 / (299)) + (1.2424295186102632 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.2273970436576442 = (31.382055283402035 - 24.24586868805566) / 31.382055283402035 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.MultiplyFunctionBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.785186516382097 < 21.544650964890277.
IsChangePoint: Marked as a change because one of 8/1/2022 4:16:36 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 21.42283963170632 (T) = (0 -17.633562124413835) / Math.Sqrt((1.7690513709820865 / (299)) + (1.7817691925752037 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.25970716817147577 = (23.819712100762768 - 17.633562124413835) / 23.819712100762768 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.MultiplyByScalarBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.565718807162895 < 40.032512750717515.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 113.54388768217137 (T) = (0 -21.327057724982332) / Math.Sqrt((5.64510753497225 / (299)) + (0.550201719341614 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.5158182718839808 = (44.047630231663675 - 21.327057724982332) / 44.047630231663675 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.SubtractOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.676954684960027 < 22.69839809404365.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 63.491386628083546 (T) = (0 -17.726688193532055) / Math.Sqrt((0.7167450426973607 / (299)) + (0.2500799930289307 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.27888684470947817 = (24.582394681720004 - 17.726688193532055) / 24.582394681720004 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.AbsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.818480338981619 < 32.011925357520454.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 84.2594017901186 (T) = (0 -16.180976563878595) / Math.Sqrt((1.87056970747071 / (299)) + (1.0597077710704925 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.5303152178560805 = (34.45071498807995 - 16.180976563878595) / 34.45071498807995 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.AddFunctionBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.881477059046805 < 21.62940324756611.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 27.249810748466814 (T) = (0 -17.793121938562) / Math.Sqrt((1.3556592998993882 / (299)) + (1.2972800043964647 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.25990508079576863 = (24.04167557006554 - 17.793121938562) / 24.04167557006554 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.MultiplyByScalarOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.035077832219972 < 40.20338679067759.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 46.867020568897026 (T) = (0 -21.56096198025498) / Math.Sqrt((3.4754238573562235 / (299)) + (5.902750256142222 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.5105288849712543 = (44.04950837392876 - 21.56096198025498) / 44.04950837392876 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.CrossBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.37222616227428 < 32.70598770417777.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 21.6520762523906 (T) = (0 -27.94286640130053) / Math.Sqrt((2.073045452307931 / (299)) + (2.100299557981256 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.18941598491598596 = (34.472511030709576 - 27.94286640130053) / 34.472511030709576 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.CreateFromScalarXYZBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.291654980268616 < 13.640239217155244.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 18.409364444365316 (T) = (0 -8.824875713671386) / Math.Sqrt((1.2561344539438202 / (299)) + (3.1612842592041313 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.42524220213648123 = (15.354077398297306 - 8.824875713671386) / 15.354077398297306 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.CreateFromScalar

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.048609285606485 < 19.973789042040565.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 71.53561465939713 (T) = (0 -8.044439968735865) / Math.Sqrt((0.8917848351750289 / (299)) + (0.795840531535248 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.6283331112924312 = (21.644220169059263 - 8.044439968735865) / 21.644220169059263 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.DivideByVector3Benchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.99312976472614 < 21.928471889533032.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 42.41481276697705 (T) = (0 -17.936736707134113) / Math.Sqrt((1.3552964746636922 / (299)) + (0.45609996665941355 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.2544107626256704 = (24.05712932539129 - 17.936736707134113) / 24.05712932539129 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.ClampBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.625732737661075 < 58.13633460865825.
IsChangePoint: Marked as a change because one of 7/14/2022 2:24:36 PM, 10/11/2022 10:51:47 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 31.84684889655163 (T) = (0 -48.564164038821296) / Math.Sqrt((12.50615567330546 / (299)) + (4.477115263528777 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.23276165111173447 = (63.29736268943172 - 48.564164038821296) / 63.29736268943172 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.CreateFromVector2WithScalarBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.389128991161675 < 23.522874142097827.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 43.94387149389403 (T) = (0 -10.735731039154844) / Math.Sqrt((1.7578153167749122 / (299)) + (2.629283667792132 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.5724605047250065 = (25.110501270180013 - 10.735731039154844) / 25.110501270180013 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.UnitXBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.448314156984669 < 19.683874693533443.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 28.735649077502035 (T) = (0 -15.392320517343618) / Math.Sqrt((2.1630810488703665 / (299)) + (0.9072229346234986 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.27702705569708025 = (21.29031333556289 - 15.392320517343618) / 21.29031333556289 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.NegateOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.89097028483464 < 22.021659271886108.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 84.02448856109598 (T) = (0 -16.809386464385096) / Math.Sqrt((0.9651838342997667 / (299)) + (0.10226057330308487 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.2911883286100797 = (23.71488385825151 - 16.809386464385096) / 23.71488385825151 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.MinBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.80613876781276 < 29.59343014425472.
IsChangePoint: Marked as a change because one of 7/14/2022 2:24:36 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 34.70095440256622 (T) = (0 -24.752458535253254) / Math.Sqrt((2.505461327381036 / (299)) + (0.9632556242631288 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.23005980375229795 = (32.14854693375431 - 24.752458535253254) / 32.14854693375431 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.225255737834573 < 27.531081206894598.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 151.04918664219142 (T) = (0 -7.949252680978541) / Math.Sqrt((1.7343513558478967 / (299)) + (0.2828945236447941 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.7188194728843058 = (28.270992883186928 - 7.949252680978541) / 28.270992883186928 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.LerpBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 52.98062837844066 < 111.0333772360849.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 96.05957124149694 (T) = (0 -54.182772982143796) / Math.Sqrt((17.509277508577245 / (299)) + (9.96290739637516 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.5495851075507924 = (120.2952519787162 - 54.182772982143796) / 120.2952519787162 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.0932082348451 < 21.690828560700485.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 37.54620232953932 (T) = (0 -17.92418855457135) / Math.Sqrt((2.106428891806724 / (299)) + (0.4974114574482672 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.25053635119275386 = (23.916021254796384 - 17.92418855457135) / 23.916021254796384 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare 2b07c8c46523b816b1eac5c8af092e52b20a69d2
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitwiseOrBenchmark - Duration of single invocation 29.52 ns 17.50 ns 0.59 0.30 False
GreaterThanAllBenchmark - Duration of single invocation 94.33 ns 80.46 ns 0.85 0.29 False
XorBenchmark - Duration of single invocation 34.24 ns 17.11 ns 0.50 0.31 False
OnesComplementOperatorBenchmark - Duration of single invocation 65.47 ns 30.44 ns 0.47 0.26 False
SubtractionOperatorBenchmark - Duration of single invocation 49.87 ns 41.08 ns 0.82 0.26 False
BitwiseAndBenchmark - Duration of single invocation 28.84 ns 16.57 ns 0.57 0.27 False
UnaryNegateOperatorBenchmark - Duration of single invocation 55.67 ns 40.66 ns 0.73 0.26 False
LessThanAllBenchmark - Duration of single invocation 107.82 ns 91.56 ns 0.85 0.23 False
SubtractBenchmark - Duration of single invocation 53.78 ns 41.02 ns 0.76 0.20 False
AndNotBenchmark - Duration of single invocation 79.84 ns 48.00 ns 0.60 0.25 False
AddBenchmark - Duration of single invocation 61.38 ns 49.62 ns 0.81 0.26 False
OneBenchmark - Duration of single invocation 33.40 ns 19.06 ns 0.57 0.24 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 95.62 ns 80.19 ns 0.84 0.25 False
ConditionalSelectBenchmark - Duration of single invocation 139.51 ns 71.96 ns 0.52 0.23 False
OnesComplementBenchmark - Duration of single invocation 58.10 ns 32.29 ns 0.56 0.21 False
DivideBenchmark - Duration of single invocation 65.77 ns 54.78 ns 0.83 0.18 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Int64>.BitwiseOrBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.497840870382053 < 27.869771639203318.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 27.26283382768351 (T) = (0 -17.405645840574135) / Math.Sqrt((2.474105576425474 / (299)) + (4.340331992926176 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.4004830839297564 = (29.032785187557188 - 17.405645840574135) / 29.032785187557188 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 80.45775024973682 < 90.17633498530449.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 15.60214342141086 (T) = (0 -79.72568179417664) / Math.Sqrt((16.97377527201852 / (299)) + (24.258306213296297 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.16296647443096515 = (95.24789552482652 - 79.72568179417664) / 95.24789552482652 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.113717382620457 < 27.802660115574565.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 68.70358317607972 (T) = (0 -17.08018615987285) / Math.Sqrt((3.048356029344753 / (299)) + (0.5913171518454071 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.41296206551710124 = (29.09554077611388 - 17.08018615987285) / 29.09554077611388 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.441581795275567 < 53.87181534281615.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 92.89763691164279 (T) = (0 -30.900656821428356) / Math.Sqrt((8.3747564912294 / (299)) + (1.30616507966775 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.4538172369708789 = (56.57567194184927 - 30.900656821428356) / 56.57567194184927 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 41.07702533450234 < 47.36451866451203.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 22.45477567792499 (T) = (0 -40.80692064312544) / Math.Sqrt((7.918621245504102 / (299)) + (2.4109190041007644 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.15965771031044962 = (48.5598798772829 - 40.80692064312544) / 48.5598798772829 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.57499091243173 < 27.451042924249116.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 21.674648686696603 (T) = (0 -17.727076042295288) / Math.Sqrt((2.386625567729158 / (299)) + (7.2284781596832435 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.3867930723294176 = (28.90879936669332 - 17.727076042295288) / 28.90879936669332 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.UnaryNegateOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 40.65990356789587 < 49.78704435867792.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 24.502248109126317 (T) = (0 -39.94178935330688) / Math.Sqrt((17.93175795061809 / (299)) + (2.9084398828758773 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.20077299422058345 = (49.975525181803796 - 39.94178935330688) / 49.975525181803796 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 91.55862519498183 < 102.51180799775305.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 24.065035925278163 (T) = (0 -90.66195535764771) / Math.Sqrt((18.48465826445128 / (299)) + (9.625754077446365 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.14855331079861106 = (106.47989651904541 - 90.66195535764771) / 106.47989651904541 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 41.01905114397426 < 50.31784867203252.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 37.38076312599889 (T) = (0 -40.6474228191193) / Math.Sqrt((9.734895759660379 / (299)) + (1.130108436312796 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.20055415438800764 = (50.84449815109472 - 40.6474228191193) / 50.84449815109472 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 48.002321077377125 < 76.42680654294038.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 91.95131284322791 (T) = (0 -46.56956342094606) / Math.Sqrt((14.755183555295469 / (299)) + (2.3747815966054846 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.4225121087351495 = (80.64162751351627 - 46.56956342094606) / 80.64162751351627 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.61758416563849 < 51.337514157320776.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 18.609260146583953 (T) = (0 -41.364408834468904) / Math.Sqrt((8.319312239190904 / (299)) + (6.319715764009905 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.18714303327891874 = (50.88768446105037 - 41.364408834468904) / 50.88768446105037 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.055653270877297 < 31.371437966274925.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 60.082876388182 (T) = (0 -19.518180357123995) / Math.Sqrt((3.06642923707418 / (299)) + (0.9706263630813216 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.40174125411783945 = (32.62498123340182 - 19.518180357123995) / 32.62498123340182 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 80.18900236165771 < 93.08384265063917.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 20.11203338759207 (T) = (0 -81.1452907040437) / Math.Sqrt((13.623627463656744 / (299)) + (12.75696676371242 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.15361413072263352 = (95.87269075430633 - 81.1452907040437) / 95.87269075430633 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 71.95732904586833 < 132.81193082907868.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 153.75307043051762 (T) = (0 -69.61558450246169) / Math.Sqrt((33.15576116017207 / (299)) + (2.8741841682258245 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.5050972755025351 = (140.66518743284527 - 69.61558450246169) / 140.66518743284527 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.29438536826988 < 54.951763965036086.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 55.28354812999521 (T) = (0 -32.420887223764694) / Math.Sqrt((7.329589953214707 / (299)) + (4.465754518512574 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.42630941990856014 = (56.512845685207466 - 32.420887223764694) / 56.512845685207466 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 54.776308659131296 < 60.976635474154655.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 41.506799863011516 (T) = (0 -53.85326789089181) / Math.Sqrt((5.735982520654001 / (299)) + (0.8453412718860591 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.14911191864060588 = (63.29065957164996 - 53.85326789089181) / 63.29065957164996 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LessThanOrEqualAllBenchmark - Duration of single invocation 109.48 ns 93.54 ns 0.85 0.32 False
DivisionOperatorBenchmark - Duration of single invocation 58.95 ns 52.29 ns 0.89 0.15 False
BitwiseOrBenchmark - Duration of single invocation 30.03 ns 16.67 ns 0.55 0.31 False
BitwiseAndOperatorBenchmark - Duration of single invocation 26.65 ns 16.52 ns 0.62 0.26 False
ConditionalSelectBenchmark - Duration of single invocation 138.78 ns 79.26 ns 0.57 0.29 False
AddOperatorBenchmark - Duration of single invocation 50.46 ns 41.56 ns 0.82 0.19 False
OneBenchmark - Duration of single invocation 36.24 ns 19.07 ns 0.53 0.22 False
BitwiseAndBenchmark - Duration of single invocation 28.83 ns 18.55 ns 0.64 0.31 False
LessThanAllBenchmark - Duration of single invocation 108.17 ns 91.33 ns 0.84 0.26 False
AndNotBenchmark - Duration of single invocation 84.42 ns 48.52 ns 0.57 0.21 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 94.91 ns 81.45 ns 0.86 0.29 False
BitwiseOrOperatorBenchmark - Duration of single invocation 27.93 ns 19.64 ns 0.70 0.26 False
DivideBenchmark - Duration of single invocation 62.36 ns 52.27 ns 0.84 0.15 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<UInt64>.LessThanOrEqualAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 93.54276213561874 < 106.59230892115272.
IsChangePoint: Marked as a change because one of 9/29/2022 8:01:59 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 24.674771813699202 (T) = (0 -92.91557683994529) / Math.Sqrt((24.28256400965299 / (299)) + (9.096844429115958 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1484730424966824 = (109.11642434947021 - 92.91557683994529) / 109.11642434947021 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 52.28615564717259 < 55.76737766412967.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 31.17699476685005 (T) = (0 -51.100849029663905) / Math.Sqrt((4.4634360489198945 / (299)) + (0.8523347863328474 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.12211934221643148 = (58.20933469326105 - 51.100849029663905) / 58.20933469326105 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.668885418633568 < 27.655744561911657.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 56.591034242173556 (T) = (0 -17.24242518717904) / Math.Sqrt((3.391781444963576 / (299)) + (0.8932420674528281 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.4089086808572056 = (29.17049299959971 - 17.24242518717904) / 29.17049299959971 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.52254226081606 < 25.521951426740586.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 37.582367841866066 (T) = (0 -17.237722750091322) / Math.Sqrt((3.08117803579907 / (299)) + (1.2743385045425244 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.34683361715642247 = (26.39101338168451 - 17.237722750091322) / 26.39101338168451 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 79.26488917736887 < 131.7495713779048.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 89.11262136347582 (T) = (0 -71.03443463773013) / Math.Sqrt((38.14565511660811 / (299)) + (14.084214370460453 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.49555931124054436 = (140.8182095945142 - 71.03443463773013) / 140.8182095945142 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 41.55502441742099 < 47.11967186455208.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 33.6476527386858 (T) = (0 -40.81376932556342) / Math.Sqrt((6.491062712996102 / (299)) + (0.7261804613629204 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1582040864201815 = (48.484161858185935 - 40.81376932556342) / 48.484161858185935 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.06678479772614 < 31.92031094224848.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 74.99422821882831 (T) = (0 -19.26988164480082) / Math.Sqrt((2.170294931517125 / (299)) + (0.6896291310016378 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.41001750539322473 = (32.66178542745449 - 19.26988164480082) / 32.66178542745449 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.548879406916495 < 27.785679621406523.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 40.610696694447064 (T) = (0 -17.662947883503186) / Math.Sqrt((3.138308218299908 / (299)) + (1.8667164652634223 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.39350640863277864 = (29.123057745236054 - 17.662947883503186) / 29.123057745236054 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 91.3288732033147 < 104.03478559700915.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 16.358247862735414 (T) = (0 -91.18353846243086) / Math.Sqrt((22.12933819969691 / (299)) + (22.37277514135625 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.14562168016465396 = (106.72501437068716 - 91.18353846243086) / 106.72501437068716 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 48.519412600183614 < 78.77981915541248.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 67.50384192933731 (T) = (0 -47.076993488832464) / Math.Sqrt((13.325975636257676 / (299)) + (5.4335768225511085 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.41551949284974665 = (80.5450188892789 - 47.076993488832464) / 80.5450188892789 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 81.44696673747494 < 91.68521063987438.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 18.250173532475607 (T) = (0 -81.42956501595525) / Math.Sqrt((21.46305648167461 / (299)) + (17.414948650355498 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.15730890229541095 = (96.63038477297516 - 81.42956501595525) / 96.63038477297516 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.644460182934665 < 26.83637676089574.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 36.95695826251282 (T) = (0 -16.912726395649734) / Math.Sqrt((2.3988876350740256 / (299)) + (1.5428318648436947 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.3618960513351416 = (26.504657166026327 - 16.912726395649734) / 26.504657166026327 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 52.26822851326839 < 59.157654760175326.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 41.700498717120205 (T) = (0 -51.1933475155145) / Math.Sqrt((4.4322836183354335 / (299)) + (1.0019262796113877 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1565657543745245 = (60.696311278599374 - 51.1933475155145) / 60.696311278599374 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6dc320ed049427c93a6d04c51494d1072c376d62
Compare 2b07c8c46523b816b1eac5c8af092e52b20a69d2
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
XorBenchmark - Duration of single invocation 28.82 ns 16.86 ns 0.58 0.24 False
OnesComplementOperatorBenchmark - Duration of single invocation 78.53 ns 41.49 ns 0.53 0.23 False
BitwiseOrBenchmark - Duration of single invocation 34.54 ns 16.88 ns 0.49 0.31 False
AddOperatorBenchmark - Duration of single invocation 82.27 ns 68.75 ns 0.84 0.19 False
ConditionalSelectBenchmark - Duration of single invocation 156.41 ns 80.53 ns 0.51 0.29 False
SubtractBenchmark - Duration of single invocation 86.64 ns 68.28 ns 0.79 0.20 False
AddBenchmark - Duration of single invocation 84.63 ns 69.21 ns 0.82 0.23 False
LessThanOrEqualAllBenchmark - Duration of single invocation 147.96 ns 127.11 ns 0.86 0.23 False
MultiplyOperatorBenchmark - Duration of single invocation 89.90 ns 68.82 ns 0.77 0.18 False
OnesComplementBenchmark - Duration of single invocation 76.95 ns 43.05 ns 0.56 0.22 False
NegateBenchmark - Duration of single invocation 83.38 ns 73.10 ns 0.88 0.19 False
MultiplyBenchmark - Duration of single invocation 84.30 ns 68.26 ns 0.81 0.19 False
BitwiseAndBenchmark - Duration of single invocation 29.22 ns 17.46 ns 0.60 0.26 False
SubtractionOperatorBenchmark - Duration of single invocation 81.23 ns 68.95 ns 0.85 0.26 False
DivisionOperatorBenchmark - Duration of single invocation 90.83 ns 78.63 ns 0.87 0.14 False
UnaryNegateOperatorBenchmark - Duration of single invocation 83.20 ns 67.69 ns 0.81 0.20 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 26.66 ns 16.91 ns 0.63 0.23 False
BitwiseAndOperatorBenchmark - Duration of single invocation 26.85 ns 17.07 ns 0.64 0.21 False
BitwiseOrOperatorBenchmark - Duration of single invocation 27.03 ns 17.23 ns 0.64 0.24 False
DivideBenchmark - Duration of single invocation 95.34 ns 76.74 ns 0.80 0.17 False
AndNotBenchmark - Duration of single invocation 101.03 ns 58.00 ns 0.57 0.27 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<UInt32>.XorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.859998968712052 < 27.53299121658925.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 73.44326555651516 (T) = (0 -17.148776386887477) / Math.Sqrt((5.467389138236797 / (299)) + (0.22093630566213338 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.40929837190702933 = (29.031198783471826 - 17.148776386887477) / 29.031198783471826 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 41.48785848448852 < 68.5548347495649.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 71.62286460352847 (T) = (0 -41.72168083950863) / Math.Sqrt((12.691132946579724 / (299)) + (2.711275816467661 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.3904898868448123 = (68.45117076652319 - 41.72168083950863) / 68.45117076652319 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.883606890616615 < 27.67000941373064.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 107.09705730900976 (T) = (0 -16.894639223095716) / Math.Sqrt((2.1684346881484102 / (299)) + (0.14481714332015225 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.41786346881426156 = (29.021781520365117 - 16.894639223095716) / 29.021781520365117 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 68.74568847429148 < 77.7839377232312.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 20.307105741816212 (T) = (0 -68.30009832199897) / Math.Sqrt((28.56697206564022 / (299)) + (2.4775072374741347 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.11400602326885559 = (77.08867116003509 - 68.30009832199897) / 77.08867116003509 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 80.52858488548179 < 148.5009829683321.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 59.82283430146386 (T) = (0 -81.88255741513883) / Math.Sqrt((44.949105130028585 / (299)) + (33.913014304524445 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.46421306085143876 = (152.82671418840746 - 81.88255741513883) / 152.82671418840746 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 68.28194637857963 < 81.64523704356816.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 11.86309084403312 (T) = (0 -68.50042170265945) / Math.Sqrt((38.85821077856567 / (299)) + (18.463687027435256 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.13334942306432904 = (79.04041550963399 - 68.50042170265945) / 79.04041550963399 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 69.21325990060058 < 80.74543283960382.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 18.019259322372076 (T) = (0 -68.72415345606036) / Math.Sqrt((28.578646853698505 / (299)) + (6.598649921692125 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.13260716384417492 = (79.23071368751106 - 68.72415345606036) / 79.23071368751106 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 127.10969291537427 < 143.192887882746.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 18.18267381331621 (T) = (0 -126.35319883145023) / Math.Sqrt((47.23886112599248 / (299)) + (22.997499432586483 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.12810743278301392 = (144.9183117075535 - 126.35319883145023) / 144.9183117075535 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 68.81689540340076 < 80.48681544481029.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 18.452739196793164 (T) = (0 -68.65759117168464) / Math.Sqrt((26.35918835242302 / (299)) + (3.076529683431819 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.10994052541224454 = (77.1382060771663 - 68.65759117168464) / 77.1382060771663 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.052163874376994 < 68.03831618518382.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 52.853071950600864 (T) = (0 -43.98651346316467) / Math.Sqrt((10.928557978512885 / (299)) + (4.856874843769841 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.35510662831661743 = (68.20742062884828 - 43.98651346316467) / 68.20742062884828 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.NegateBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 73.10159260204473 < 79.85476321661281.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 16.410176213862453 (T) = (0 -67.21090536468532) / Math.Sqrt((37.164815620248085 / (299)) + (7.927830795798443 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.13957502810163322 = (78.11361543400702 - 67.21090536468532) / 78.11361543400702 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 68.25913694521859 < 80.46317951499285.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 29.24770627153774 (T) = (0 -68.10043282968502) / Math.Sqrt((31.676248074749285 / (299)) + (1.450173609749068 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.14571531932044243 = (79.71632216969311 - 68.10043282968502) / 79.71632216969311 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.45570483078013 < 28.23778621217659.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 62.94974366358631 (T) = (0 -17.1793594111855) / Math.Sqrt((2.257511720032352 / (299)) + (0.7612487499745176 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.4092540587650659 = (29.080791270901734 - 17.1793594111855) / 29.080791270901734 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 68.95118705782457 < 77.62348697990538.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 7.760255931454964 (T) = (0 -69.13730633043126) / Math.Sqrt((35.80676114532696 / (299)) + (21.915788706041244 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.09920391250622962 = (76.75133949880681 - 69.13730633043126) / 76.75133949880681 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 78.63365634381783 < 84.75306443624034.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 15.307014516005255 (T) = (0 -76.05646674761702) / Math.Sqrt((27.370427486035563 / (299)) + (5.85459248133899 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09830194408684031 = (84.34804339307773 - 76.05646674761702) / 84.34804339307773 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.UnaryNegateOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 67.69240743316905 < 79.78457721812802.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 30.769320477582678 (T) = (0 -66.31777823450105) / Math.Sqrt((29.13574159963202 / (299)) + (1.1309079459591393 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.14694141591727042 = (77.7411768334888 - 66.31777823450105) / 77.7411768334888 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.914216405204808 < 25.442533738541727.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 44.467450062611995 (T) = (0 -17.366261962739415) / Math.Sqrt((2.070193877279334 / (299)) + (0.926126370588363 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.3420592951711379 = (26.39487393815614 - 17.366261962739415) / 26.39487393815614 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.068434564487735 < 25.519542519196413.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 20.852052804021923 (T) = (0 -17.284983435380873) / Math.Sqrt((2.0028583609009174 / (299)) + (4.967702834999707 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.3450328775438356 = (26.390612357092355 - 17.284983435380873) / 26.390612357092355 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.23239038725939 < 25.724739071508058.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 57.55557866725895 (T) = (0 -17.027767400628992) / Math.Sqrt((3.9606667594248646 / (299)) + (0.41141532466236796 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.36324648260237874 = (26.74153645859798 - 17.027767400628992) / 26.74153645859798 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 76.73710466142293 < 86.22857999861972.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 23.916016392340506 (T) = (0 -75.81036918911832) / Math.Sqrt((21.55254913395948 / (299)) + (3.292283954959672 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.12611284473506665 = (86.75075349532418 - 75.81036918911832) / 86.75075349532418 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 58.00457101720939 < 90.31891243846894.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 104.47127127970346 (T) = (0 -57.46585546021434) / Math.Sqrt((21.20681290728423 / (299)) + (1.278092418766766 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.38298391064678455 = (93.13510044843837 - 57.46585546021434) / 93.13510044843837 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-improvement runtime-mono untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kotlarmilos