Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 144 Improvements on 10/11/2022 1:45:10 PM #9043

Closed
performanceautofiler bot opened this issue Oct 18, 2022 · 6 comments
Closed

[Perf] Linux/x64: 144 Improvements on 10/11/2022 1:45:10 PM #9043

performanceautofiler bot opened this issue Oct 18, 2022 · 6 comments
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-improvement runtime-mono untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Strings

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteStringsUtf16 - Duration of single invocation 244.31 ms 214.10 ms 0.88 0.02 False
WriteStringsUtf16 - Duration of single invocation 195.03 ms 171.69 ms 0.88 0.02 False
WriteStringsUtf16 - Duration of single invocation 236.70 ms 213.96 ms 0.90 0.05 False
WriteStringsUtf16 - Duration of single invocation 184.09 ms 160.44 ms 0.87 0.04 False
WriteStringsUtf16 - Duration of single invocation 192.31 ms 166.71 ms 0.87 0.03 False
WriteStringsUtf16 - Duration of single invocation 226.42 ms 204.03 ms 0.90 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Strings*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: False, Escaped: OneEscaped)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 214.09683692857143 < 230.70525998612635.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 45.97012836714184 (T) = (0 -218144242.1652174) / Math.Sqrt((41095040066585.21 / (299)) + (10610930808543.86 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1402048189931117 = (253716521.07860526 - 218144242.1652174) / 253716521.07860526 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: False, Escaped: NoneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 171.6851378 < 183.87810940166665.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 70.06763207190447 (T) = (0 -171672322.74143592) / Math.Sqrt((16653119507228.168 / (299)) + (2391066977384.3647 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.1370224420615964 = (198930228.44249827 - 171672322.74143592) / 198930228.44249827 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: True, Escaped: OneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 213.9570432 < 224.08778068.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 48.255676619812114 (T) = (0 -216550051.83771485) / Math.Sqrt((45441183095891.99 / (299)) + (9350922925721.586 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.13747916871954466 = (251066460.06014192 - 216550051.83771485) / 251066460.06014192 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: True, Escaped: NoneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 160.4428713 < 175.490151535.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 55.801971716137345 (T) = (0 -162391259.0773407) / Math.Sqrt((26046818232106.535 / (299)) + (3906261021592.486 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.14494656090125735 = (189919426.8471769 - 162391259.0773407) / 189919426.8471769 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: True, Escaped: NoneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 166.7052049 < 182.23059854307692.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 62.79602739594414 (T) = (0 -170330214.95951647) / Math.Sqrt((15073584081015.686 / (299)) + (3360505822006.916 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.1368152874474146 = (197327654.76791263 - 170330214.95951647) / 197327654.76791263 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: True, Escaped: OneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 204.0298972 < 214.96646857166664.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 44.75052984412466 (T) = (0 -206094343.44154176) / Math.Sqrt((44783990713260.836 / (299)) + (7306869601779.8125 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.13099040936280215 = (237160033.2861964 - 206094343.44154176) / 237160033.2861964 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LessThanOrEqualAllBenchmark - Duration of single invocation 233.86 ns 219.94 ns 0.94 0.17 False
OneBenchmark - Duration of single invocation 78.30 ns 51.68 ns 0.66 0.10 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 31.10 ns 16.78 ns 0.54 0.30 False
AddOperatorBenchmark - Duration of single invocation 167.87 ns 138.61 ns 0.83 0.08 False
DivideBenchmark - Duration of single invocation 177.66 ns 152.05 ns 0.86 0.14 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 220.77 ns 197.91 ns 0.90 0.12 False
MultiplyBenchmark - Duration of single invocation 169.70 ns 138.59 ns 0.82 0.14 False
LessThanAllBenchmark - Duration of single invocation 226.51 ns 195.77 ns 0.86 0.10 False
BitwiseAndOperatorBenchmark - Duration of single invocation 27.85 ns 16.96 ns 0.61 0.25 False
LessThanAnyBenchmark - Duration of single invocation 200.80 ns 165.29 ns 0.82 0.15 False
GreaterThanAnyBenchmark - Duration of single invocation 198.61 ns 181.86 ns 0.92 0.13 False
GreaterThanOrEqualBenchmark - Duration of single invocation 171.61 ns 158.60 ns 0.92 0.11 False
OnesComplementBenchmark - Duration of single invocation 106.12 ns 64.27 ns 0.61 0.20 False
DivisionOperatorBenchmark - Duration of single invocation 174.27 ns 152.79 ns 0.88 0.13 False
NegateBenchmark - Duration of single invocation 170.95 ns 139.79 ns 0.82 0.12 False
GreaterThanBenchmark - Duration of single invocation 164.33 ns 149.62 ns 0.91 0.13 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 207.16 ns 188.90 ns 0.91 0.11 False
MultiplyOperatorBenchmark - Duration of single invocation 167.32 ns 138.85 ns 0.83 0.09 False
GreaterThanAllBenchmark - Duration of single invocation 210.42 ns 183.31 ns 0.87 0.15 False
BitwiseOrOperatorBenchmark - Duration of single invocation 27.81 ns 17.09 ns 0.61 0.29 False
OnesComplementOperatorBenchmark - Duration of single invocation 107.78 ns 62.48 ns 0.58 0.17 False
EqualsStaticBenchmark - Duration of single invocation 163.41 ns 151.41 ns 0.93 0.13 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<UInt16>.LessThanOrEqualAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 219.94126438349937 < 221.86616222828349.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 32.56209496563133 (T) = (0 -202.71581377626254) / Math.Sqrt((55.35831248621779 / (299)) + (25.65784520729697 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.15023697777529701 = (238.55570138313027 - 202.71581377626254) / 238.55570138313027 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 51.682046575630395 < 74.41225833620963.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 116.22206801123741 (T) = (0 -52.0419446491872) / Math.Sqrt((7.70619236759787 / (299)) + (0.4644253607997828 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.31987072979277215 = (76.51772527497691 - 52.0419446491872) / 76.51772527497691 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.779846609650345 < 26.34343085301199.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 43.22800998521806 (T) = (0 -17.063909577424692) / Math.Sqrt((2.4849374130162567 / (299)) + (0.9917727930111705 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.3568770935182442 = (26.532890378254258 - 17.063909577424692) / 26.532890378254258 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 138.60857601059357 < 159.6955534150716.
IsChangePoint: Marked as a change because one of 9/28/2022 11:19:50 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 31.254221038490293 (T) = (0 -141.55468064534486) / Math.Sqrt((26.9143362462975 / (299)) + (12.95316896023667 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1428521370863692 = (165.14616295509379 - 141.55468064534486) / 165.14616295509379 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 152.05182862754663 < 168.81037347096694.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 30.96136808493978 (T) = (0 -154.27580536922974) / Math.Sqrt((108.5558417223153 / (299)) + (20.941610666984314 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.17638012512239837 = (187.31433040291398 - 154.27580536922974) / 187.31433040291398 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 197.91414082985332 < 208.2420975665627.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 37.49316071446321 (T) = (0 -193.20212428405495) / Math.Sqrt((64.29827995110979 / (299)) + (15.893978134478239 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.14995601864668112 = (227.2848564570342 - 193.20212428405495) / 227.2848564570342 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 138.58845870030947 < 161.42877224687743.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 25.511601813280453 (T) = (0 -142.9455040023057) / Math.Sqrt((32.69090561962889 / (299)) + (22.01159490708489 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.14856970654052437 = (167.8886752096862 - 142.9455040023057) / 167.8886752096862 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 195.7728408396509 < 215.86898322339928.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 33.361122954216384 (T) = (0 -193.11910502389827) / Math.Sqrt((36.92027171261182 / (299)) + (34.97248343313042 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.17310638207960374 = (233.54770291925212 - 193.11910502389827) / 233.54770291925212 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.96313874082783 < 25.480895738266682.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 35.16075726271195 (T) = (0 -17.297169675435654) / Math.Sqrt((2.723164625525337 / (299)) + (1.5644151577815748 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.3448445068782384 = (26.401625044790627 - 17.297169675435654) / 26.401625044790627 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 165.2862208243197 < 174.92609523944316.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 33.14915555131372 (T) = (0 -165.05736697482928) / Math.Sqrt((71.47686632692265 / (299)) + (11.204179879183462 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.13972421074701555 = (191.8656424332898 - 165.05736697482928) / 191.8656424332898 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.GreaterThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 181.8571613172415 < 188.72629995205864.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 18.208172367852917 (T) = (0 -183.77689694107463) / Math.Sqrt((46.28473863265438 / (299)) + (24.100010631070973 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09638192935915878 = (203.37895280330218 - 183.77689694107463) / 203.37895280330218 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 158.5952783783688 < 162.42270536922882.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 27.268942687309213 (T) = (0 -157.81229554458503) / Math.Sqrt((49.32138548806321 / (299)) + (8.842984033059109 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.10814637996587959 = (176.9486516616342 - 157.81229554458503) / 176.9486516616342 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 64.26903115226375 < 97.42952289724953.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 82.08904948012959 (T) = (0 -64.86916335307859) / Math.Sqrt((23.621256006742826 / (299)) + (2.9833687324820777 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.3577434172709527 = (101.00194392315842 - 64.86916335307859) / 101.00194392315842 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 152.7851248270516 < 166.11101895557914.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 34.32368396765061 (T) = (0 -153.79416210755647) / Math.Sqrt((118.55228283682106 / (299)) + (12.164630171456574 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.17040149399382898 = (185.3838465162478 - 153.79416210755647) / 185.3838465162478 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.NegateBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 139.790325294358 < 160.6918691310229.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 18.280259106705923 (T) = (0 -141.93192827456397) / Math.Sqrt((35.18775588797734 / (299)) + (37.205153298734395 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.1476457722969253 = (166.51753890755293 - 141.93192827456397) / 166.51753890755293 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 149.6164138034376 < 155.9440963339355.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 9.819427781581647 (T) = (0 -151.6172723881577) / Math.Sqrt((27.640096203142935 / (299)) + (60.46484288685747 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.09149437752702663 = (166.8864436693875 - 151.6172723881577) / 166.8864436693875 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 188.90290351017575 < 196.90505810007704.
IsChangePoint: Marked as a change because one of 7/31/2022 3:07:22 PM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 38.45336264475457 (T) = (0 -190.67109529265977) / Math.Sqrt((50.180671015549684 / (299)) + (4.026604286191731 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1019639626871547 = (212.32009337085927 - 190.67109529265977) / 212.32009337085927 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 138.85148408846769 < 159.21998571242347.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 29.669951092057822 (T) = (0 -141.8541087753871) / Math.Sqrt((27.398968057007988 / (299)) + (11.777966699401343 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.1420860760516173 = (165.34771707927413 - 141.8541087753871) / 165.34771707927413 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 183.3060131117501 < 200.08695482456045.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 43.10464051297788 (T) = (0 -184.46942971547963) / Math.Sqrt((56.9324863253596 / (299)) + (11.963452912800621 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.15515234035698452 = (218.34638187129016 - 184.46942971547963) / 218.34638187129016 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.09260825768483 < 25.861430583783857.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 47.01663865223364 (T) = (0 -17.212604316378847) / Math.Sqrt((3.5367059202770847 / (299)) + (0.7481905783150603 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.3550137908211663 = (26.686778835617478 - 17.212604316378847) / 26.686778835617478 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 62.48104783040085 < 96.75643044449515.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 127.79776996058987 (T) = (0 -63.03519164430873) / Math.Sqrt((12.379379061874001 / (299)) + (1.1531489483098714 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.37253018396562276 = (100.45932096414215 - 63.03519164430873) / 100.45932096414215 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 151.41483510041152 < 155.29149559392596.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/11/2022 10:51:47 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 18.316778555005644 (T) = (0 -150.22613556416292) / Math.Sqrt((49.279185703874205 / (299)) + (17.637222750688988 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.1021297422257541 = (167.3138566106003 - 150.22613556416292) / 167.3138566106003 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Memory.MemoryMarshal<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetReference - Duration of single invocation 16.40 ns 14.44 ns 0.88 0.23 False
AsBytes - Duration of single invocation 24.67 ns 21.06 ns 0.85 0.18 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.MemoryMarshal&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.MemoryMarshal<Int32>.GetReference

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.444472579211958 < 15.445068665787414.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 17.723716241128052 (T) = (0 -14.449566098716407) / Math.Sqrt((1.1602208542802968 / (299)) + (0.27593777715643375 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.13583790082403413 = (16.720897748807772 - 14.449566098716407) / 16.720897748807772 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.MemoryMarshal&lt;Int32&gt;.AsBytes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.06112619566395 < 26.589767740967453.
IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 37.61804252999748 (T) = (0 -20.963746245263273) / Math.Sqrt((2.4439392881959687 / (299)) + (0.8126947439835175 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.26895610138697296 = (28.676453336163174 - 20.963746245263273) / 28.676453336163174 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in Benchstone.BenchF.BenchMrk

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 604.76 ms 555.59 ms 0.92 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchF.BenchMrk*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Benchstone.BenchF.BenchMrk.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 555.5891834166666 < 571.8477311096153.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 66.70848949146944 (T) = (0 -550257003.8097509) / Math.Sqrt((42402771350212.43 / (299)) + (10080229875580.67 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.08214786077254344 = (599505062.1910568 - 550257003.8097509) / 599505062.1910568 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in Benchstone.BenchF.BenchMk2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 544.71 ms 498.92 ms 0.92 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchF.BenchMk2*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Benchstone.BenchF.BenchMk2.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 498.9220579285714 < 517.2437034375.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 40.39898902581048 (T) = (0 -503393750.41548157) / Math.Sqrt((33743526225002.13 / (299)) + (28612752066358.484 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.08122058244948065 = (547894022.0031674 - 503393750.41548157) / 547894022.0031674 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToWriter - Duration of single invocation 10.14 μs 9.60 μs 0.95 0.01 False
SerializeToStream - Duration of single invocation 17.53 μs 16.31 μs 0.93 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;LargeStructWithProperties&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToWriter

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.59809584494324 < 9.673304137366081.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 22.315018255887104 (T) = (0 -9487.738399603368) / Math.Sqrt((21753.783129188298 / (176)) + (22273.581931716937 / (25))) is greater than 1.9719565442516196 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (176) + (25) - 2, .975) and 0.06969407273421836 = (10198.51440427595 - 9487.738399603368) / 10198.51440427595 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;LargeStructWithProperties&gt;.SerializeToStream

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.314132326977404 < 16.566017144187143.
IsChangePoint: Marked as a change because one of 7/30/2022 3:39:54 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 35.77653812694803 (T) = (0 -16350.599118180382) / Math.Sqrt((63678.19118672458 / (299)) + (27914.938442066792 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.07277392040563918 = (17633.886144933906 - 16350.599118180382) / 17633.886144933906 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in Benchstone.BenchF.MatInv4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 24.75 ms 23.15 ms 0.94 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchF.MatInv4*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Benchstone.BenchF.MatInv4.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.153803292857145 < 23.301377450888886.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 29.968326469641884 (T) = (0 -23271806.72494359) / Math.Sqrt((105921458025.79878 / (299)) + (42846585343.27882 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.055322691131751374 = (24634662.55246874 - 23271806.72494359) / 24634662.55246874 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToUtf8Bytes - Duration of single invocation 6.52 μs 6.06 μs 0.93 0.00 False
SerializeToStream - Duration of single invocation 10.68 μs 9.87 μs 0.92 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;SimpleStructWithProperties&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToUtf8Bytes

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.062034738409738 < 6.1651522904900276.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 29.701519212240242 (T) = (0 -6044.26441199306) / Math.Sqrt((6447.999688965866 / (299)) + (5540.558252319989 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.07117687159270866 = (6507.4439116923395 - 6044.26441199306) / 6507.4439116923395 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;SimpleStructWithProperties&gt;.SerializeToStream

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.865820032600848 < 10.019981338197187.
IsChangePoint: Marked as a change because one of 8/3/2022 4:57:40 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 35.094179064038734 (T) = (0 -9911.348483737496) / Math.Sqrt((20965.793033188533 / (299)) + (9922.670136614663 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.06896369478578128 = (10645.501607434127 - 9911.348483737496) / 10645.501607434127 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Collections.IterateFor<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableArray - Duration of single invocation 4.05 μs 3.01 μs 0.74 0.05 False
Span - Duration of single invocation 3.36 μs 3.03 μs 0.90 0.09 False
ReadOnlySpan - Duration of single invocation 3.31 μs 3.03 μs 0.92 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateFor&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IterateFor<Int32>.ImmutableArray(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.011260699070488 < 3.8479432375553437.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 38.2577163264181 (T) = (0 -3087.7591244926875) / Math.Sqrt((184921.16852941836 / (299)) + (3247.3826662922856 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.25314415963207604 = (4134.344216912816 - 3087.7591244926875) / 4134.344216912816 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateFor&lt;Int32&gt;.Span(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.0283541869404442 < 3.1597456743954666.
IsChangePoint: Marked as a change because one of 8/5/2022 4:49:35 PM, 9/26/2022 10:49:59 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 41.01998254640281 (T) = (0 -3031.41569637868) / Math.Sqrt((29498.01313531163 / (299)) + (557.2070606409056 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1291360684349579 = (3480.929208918871 - 3031.41569637868) / 3480.929208918871 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateFor&lt;Int32&gt;.ReadOnlySpan(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.02787426756862 < 3.160342265204755.
IsChangePoint: Marked as a change because one of 8/5/2022 4:49:35 PM, 9/26/2022 10:49:59 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 38.10512892133434 (T) = (0 -3023.032192425628) / Math.Sqrt((38094.97221474877 / (299)) + (588.6829459198865 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1337520062979577 = (3489.7999353582813 - 3023.032192425628) / 3489.7999353582813 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FormatterDouble - Duration of single invocation 4.84 μs 4.56 μs 0.94 0.02 False
FormatterDecimal - Duration of single invocation 559.25 ns 520.17 ns 0.93 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: 1.7976931348623157E+308)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.561601034659256 < 4.58975584120581.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 40.64521042752788 (T) = (0 -4477.025910268649) / Math.Sqrt((17676.846619683954 / (299)) + (2949.7137425814703 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.10779620268178106 = (5017.940882706022 - 4477.025910268649) / 5017.940882706022 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDecimal(value: 123456.789)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 520.1695573874819 < 536.5320282231667.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 29.825381258378762 (T) = (0 -516.3351813327238) / Math.Sqrt((474.1775427692453 / (299)) + (112.56399838487162 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.12474969725101279 = (589.9285949528011 - 516.3351813327238) / 589.9285949528011 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Numerics.Tests.Perf_Matrix3x2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateTranslationFromVectorBenchmark - Duration of single invocation 31.10 ns 26.75 ns 0.86 0.18 False
CreateScaleFromScalarBenchmark - Duration of single invocation 21.44 ns 16.89 ns 0.79 0.35 False
CreateRotationBenchmark - Duration of single invocation 99.63 ns 82.24 ns 0.83 0.22 False
MultiplyByMatrixOperatorBenchmark - Duration of single invocation 75.54 ns 70.29 ns 0.93 0.19 False
CreateRotationWithCenterBenchmark - Duration of single invocation 113.36 ns 94.43 ns 0.83 0.24 False
CreateTranslationFromScalarXY - Duration of single invocation 22.71 ns 16.96 ns 0.75 0.29 False
CreateScaleFromScalarXYBenchmark - Duration of single invocation 21.44 ns 18.28 ns 0.85 0.24 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_Matrix3x2.CreateTranslationFromVectorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.754772738778396 < 29.303263076510355.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 6.334686253103226 (T) = (0 -26.404869076463488) / Math.Sqrt((2.592084973935035 / (299)) + (8.106313310954553 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.12841118861648737 = (30.29509871122581 - 26.404869076463488) / 30.29509871122581 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.890160095233103 < 20.39154753741699.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 25.722712504213494 (T) = (0 -16.609034726535366) / Math.Sqrt((1.516430002394386 / (299)) + (0.7748564782704145 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.22727267215661748 = (21.4940434071224 - 16.609034726535366) / 21.4940434071224 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.CreateRotationBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 82.24344709203196 < 94.67595862393851.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 30.05724586314407 (T) = (0 -79.3670183957245) / Math.Sqrt((15.980972737178414 / (299)) + (14.75635454167649 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.22681057883978736 = (102.64886743617106 - 79.3670183957245) / 102.64886743617106 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.MultiplyByMatrixOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 70.29444868872929 < 71.70584501407616.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 14.774817753161145 (T) = (0 -69.87284825288202) / Math.Sqrt((9.42258658185643 / (299)) + (0.8909376493831472 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.05144293452371871 = (73.66225058668249 - 69.87284825288202) / 73.66225058668249 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.CreateRotationWithCenterBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 94.42897330755075 < 108.60489081310891.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 19.28805802692718 (T) = (0 -95.1250063755298) / Math.Sqrt((18.90950753580763 / (299)) + (30.84942341885606 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1821082929842008 = (116.30513619291688 - 95.1250063755298) / 116.30513619291688 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.CreateTranslationFromScalarXY

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.955020766465584 < 20.854630993495164.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 24.466111642563295 (T) = (0 -17.622129014817183) / Math.Sqrt((1.493644621474144 / (299)) + (0.565747083420554 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.18749388630170244 = (21.688610975007002 - 17.622129014817183) / 21.688610975007002 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarXYBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.27607565809798 < 21.5973576900921.
IsChangePoint: Marked as a change because one of 10/10/2022 3:25:05 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 22.017130325107956 (T) = (0 -17.478035481136114) / Math.Sqrt((1.3124678291566356 / (299)) + (0.9637283666367901 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.20412891291927418 = (21.96088759204203 - 17.478035481136114) / 21.96088759204203 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Threading.Tests.Perf_Volatile

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Read_double - Duration of single invocation 5.89 ns 2.71 ns 0.46 0.46 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tests.Perf_Volatile*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Threading.Tests.Perf_Volatile.Read_double

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.7068123538136026 < 5.435567370201825.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 26.241973395802795 (T) = (0 -2.1947109471708486) / Math.Sqrt((0.6360874072016054 / (299)) + (0.6123309011095397 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.6531651103211278 = (6.327826330283233 - 2.1947109471708486) / 6.327826330283233 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Equals_Valid - Duration of single invocation 189.02 ns 174.79 ns 0.92 0.20 False
LastIndexOf - Duration of single invocation 86.74 ns 78.38 ns 0.90 0.15 False
StartsWith - Duration of single invocation 120.39 ns 105.35 ns 0.88 0.19 False
Equals_String - Duration of single invocation 144.96 ns 126.30 ns 0.87 0.23 False
IndexOfAny - Duration of single invocation 114.42 ns 103.42 ns 0.90 0.24 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_Valid

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 174.79487431639404 < 180.94292149112923.
IsChangePoint: Marked as a change because one of 7/31/2022 12:39:41 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 22.764948369243218 (T) = (0 -175.24926290829004) / Math.Sqrt((33.10381295380658 / (299)) + (18.96440670340758 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.10802978330493797 = (196.47434368114392 - 175.24926290829004) / 196.47434368114392 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.LastIndexOf

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 78.37746849038561 < 82.47841726044582.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 18.40494994700404 (T) = (0 -78.79575316605853) / Math.Sqrt((52.351586200871736 / (299)) + (10.524663017586434 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.15100974266652378 = (92.81113945116596 - 78.79575316605853) / 92.81113945116596 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.StartsWith

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 105.34550522777 < 109.85435214624918.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 19.834297220227263 (T) = (0 -104.0053349003469) / Math.Sqrt((22.533182502493748 / (299)) + (16.153487233310653 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.13168257336454642 = (119.77801171553769 - 104.0053349003469) / 119.77801171553769 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_String

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 126.29583922503267 < 138.64332109205125.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 30.036832510585224 (T) = (0 -124.74801067370274) / Math.Sqrt((40.54152119342871 / (299)) + (16.658028190928466 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1727855961149295 = (150.80493048454542 - 124.74801067370274) / 150.80493048454542 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOfAny

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 103.41828599985166 < 108.64296119150173.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 16.795593252238586 (T) = (0 -101.94872741696268) / Math.Sqrt((107.91642592393865 / (299)) + (14.453008251161757 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.136251253503967 = (118.03053588272954 - 101.94872741696268) / 118.03053588272954 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SliceTenSegments - Duration of single invocation 265.02 ns 232.30 ns 0.88 0.06 False
IterateGetPositionTenSegments - Duration of single invocation 1.00 μs 922.06 ns 0.92 0.11 False
FirstTenSegments - Duration of single invocation 46.01 ns 42.94 ns 0.93 0.09 False
SliceSingleSegment - Duration of single invocation 95.74 ns 89.19 ns 0.93 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.SliceTenSegments

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 232.29893386073906 < 249.8991856921993.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 31.70772670601774 (T) = (0 -230.79404321393682) / Math.Sqrt((37.6834194498073 / (299)) + (18.310567240405273 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.11291792237986605 = (260.1721408160017 - 230.79404321393682) / 260.1721408160017 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;.IterateGetPositionTenSegments

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 922.0626037830816 < 936.0829373411776.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 22.276044518479203 (T) = (0 -903.3831498982178) / Math.Sqrt((615.7919755812 / (299)) + (247.7855781921896 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.07531030050244707 = (976.9581627102449 - 903.3831498982178) / 976.9581627102449 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;.FirstTenSegments

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 42.94056077781797 < 43.2964623250364.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 30.43338698723242 (T) = (0 -42.66845167336645) / Math.Sqrt((1.94753874260441 / (299)) + (0.43721518361905576 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.09704762293460047 = (47.25437659518563 - 42.66845167336645) / 47.25437659518563 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;.SliceSingleSegment

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 89.18894040031219 < 92.22872638655576.
IsChangePoint: Marked as a change because one of 7/30/2022 2:52:48 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 10.452456606224805 (T) = (0 -87.95802919270687) / Math.Sqrt((7.095813157915306 / (299)) + (19.371489436434448 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.09281240909763681 = (96.95682577097048 - 87.95802919270687) / 96.95682577097048 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.IO.Tests.BinaryWriterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteUInt64 - Duration of single invocation 64.06 ns 52.50 ns 0.82 0.15 False
WriteDouble - Duration of single invocation 61.64 ns 52.58 ns 0.85 0.15 False
WriteSingle - Duration of single invocation 62.03 ns 53.20 ns 0.86 0.14 False
WriteUInt32 - Duration of single invocation 61.21 ns 53.67 ns 0.88 0.15 False
WriteUInt16 - Duration of single invocation 62.66 ns 50.89 ns 0.81 0.19 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.BinaryWriterTests*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.BinaryWriterTests.WriteUInt64

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 52.498127886423454 < 59.58005645022986.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 34.02611291916693 (T) = (0 -52.85029024961955) / Math.Sqrt((6.226596718673039 / (299)) + (1.489160035896487 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.15636096741603267 = (62.64561999667715 - 52.85029024961955) / 62.64561999667715 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterTests.WriteDouble

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 52.57674704839336 < 58.500919033007996.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 32.46738826484407 (T) = (0 -52.66656851080878) / Math.Sqrt((6.9854622642001525 / (299)) + (1.848772959606088 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.16341073003138445 = (62.953913469132296 - 52.66656851080878) / 62.953913469132296 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterTests.WriteSingle

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 53.19979227123914 < 58.92751165780529.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 15.191579695325265 (T) = (0 -53.38005263405394) / Math.Sqrt((5.613358807104033 / (299)) + (7.514022025882729 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.14590622431120492 = (62.49905356236193 - 53.38005263405394) / 62.49905356236193 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterTests.WriteUInt32

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 53.67422930879348 < 58.86766520280173.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 33.30618849791652 (T) = (0 -52.046633723846355) / Math.Sqrt((6.186260117254663 / (299)) + (1.9742954279450844 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.16806771828394332 = (62.56114213585737 - 52.046633723846355) / 62.56114213585737 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterTests.WriteUInt16

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 50.8890181394747 < 59.48013839633887.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 17.36576109460082 (T) = (0 -53.57018582143137) / Math.Sqrt((5.3010107220838885 / (299)) + (7.827311634206947 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.162459942265304 = (63.961341701462324 - 53.57018582143137) / 63.961341701462324 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseBool - Duration of single invocation 45.10 ns 40.81 ns 0.90 0.21 False
TryParseInt64 - Duration of single invocation 320.40 ns 295.85 ns 0.92 0.06 False
TryParseBool - Duration of single invocation 50.97 ns 45.08 ns 0.88 0.18 False
TryParseUInt64Hex - Duration of single invocation 120.43 ns 111.68 ns 0.93 0.27 False
TryParseBool - Duration of single invocation 56.99 ns 49.49 ns 0.87 0.24 False
TryParseInt64 - Duration of single invocation 303.90 ns 285.74 ns 0.94 0.06 False
TryParseBool - Duration of single invocation 55.27 ns 50.29 ns 0.91 0.25 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Buffers.Text.Tests.Utf8ParserTests.TryParseBool(value: true)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 40.811727677019775 < 42.69848333757105.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 13.79781476339236 (T) = (0 -40.11813455367287) / Math.Sqrt((5.562100557568566 / (299)) + (5.488218126009731 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.14371325639033702 = (46.8512853352786 - 40.11813455367287) / 46.8512853352786 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: -9223372036854775808)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 295.852841842694 < 303.66676322926.
IsChangePoint: Marked as a change because one of 7/4/2022 6:29:12 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 32.45896309590165 (T) = (0 -294.55187428607246) / Math.Sqrt((54.73604980907186 / (299)) + (10.03904540764619 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.07585958011999298 = (318.73064736668255 - 294.55187428607246) / 318.73064736668255 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseBool(value:  True )

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 45.07922850756991 < 48.069350660485746.
IsChangePoint: Marked as a change because one of 7/16/2022 5:58:27 PM, 7/17/2022 2:49:16 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 20.813257958194534 (T) = (0 -44.25044316376217) / Math.Sqrt((6.2640395256420085 / (299)) + (2.809582719439988 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1445249676189524 = (51.72616556745053 - 44.25044316376217) / 51.72616556745053 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value: 3039)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 111.68473691872074 < 112.0338129808884.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 13.40337473812083 (T) = (0 -108.53711652567821) / Math.Sqrt((39.34752909720977 / (299)) + (32.80895803281442 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.1292177926053278 = (124.6432409895176 - 108.53711652567821) / 124.6432409895176 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseBool(value: false)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.49043418479042 < 53.082923481434754.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 9.707489864242072 (T) = (0 -49.67523210791737) / Math.Sqrt((7.097926416979809 / (299)) + (14.6127446936779 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1280717722824147 = (56.97169850544971 - 49.67523210791737) / 56.97169850544971 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: 9223372036854775807)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 285.7434541826999 < 287.6460667028728.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 30.99459770703697 (T) = (0 -284.1363768119115) / Math.Sqrt((37.419626660153064 / (299)) + (8.899820861763237 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.06852181104197552 = (305.03814279296626 - 284.1363768119115) / 305.03814279296626 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseBool(value: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 50.294966240776866 < 52.82158049796456.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 16.027036848258984 (T) = (0 -49.625692014639924) / Math.Sqrt((9.783758648185497 / (299)) + (5.135131002312054 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.13230602267629765 = (57.19262010749965 - 49.625692014639924) / 57.19262010749965 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3435b39194137676d9b2da12c8ebd21d69cd06b9
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in Microsoft.Extensions.Logging.LoggingOverhead

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TwoArguments_DefineMessage_FilteredByLevel - Duration of single invocation 86.17 ns 66.06 ns 0.77 0.22 False
NoArguments_FilteredByLevel - Duration of single invocation 135.06 ns 126.64 ns 0.94 0.21 False
NoArguments_DefineMessage - Duration of single invocation 218.11 ns 206.04 ns 0.94 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Logging.LoggingOverhead*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Microsoft.Extensions.Logging.LoggingOverhead.TwoArguments_DefineMessage_FilteredByLevel

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 66.06426656976952 < 69.59047226967773.
IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 25.352985109217407 (T) = (0 -66.40986686997145) / Math.Sqrt((11.121594293295253 / (299)) + (1.4787897425794705 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.10594513453160552 = (74.27940883155911 - 66.40986686997145) / 74.27940883155911 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.LoggingOverhead.NoArguments_FilteredByLevel

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 126.63729071267666 < 127.14286540457925.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 11.09514395319474 (T) = (0 -125.34997657397818) / Math.Sqrt((24.370407555056993 / (299)) + (13.286332248857681 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.06282482538991015 = (133.75298446859716 - 125.34997657397818) / 133.75298446859716 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.LoggingOverhead.NoArguments_DefineMessage

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 206.04245977237048 < 206.57890141155283.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 15.506609825869367 (T) = (0 -203.5300084299811) / Math.Sqrt((31.194195930281836 / (299)) + (17.49545871097591 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.06294054803992888 = (217.20074217729965 - 203.5300084299811) / 217.20074217729965 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Tests.Perf_Guid

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 1.21 μs 960.71 ns 0.79 0.04 False
TryWriteBytes - Duration of single invocation 38.04 ns 34.47 ns 0.91 0.24 False
ParseExactD - Duration of single invocation 1.13 μs 905.92 ns 0.80 0.04 False
ctor_str - Duration of single invocation 1.21 μs 980.03 ns 0.81 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Guid*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Guid.Parse

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 960.7078696504105 < 1.1419651394842711.
IsChangePoint: Marked as a change because one of 8/1/2022 8:26:50 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 61.85902060348981 (T) = (0 -979.2389018374384) / Math.Sqrt((1361.1156203804433 / (299)) + (348.5150071678226 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.21116165134770765 = (1241.3682771767371 - 979.2389018374384) / 1241.3682771767371 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Guid.TryWriteBytes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.46654300479882 < 36.45770171408875.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 5.338633908033321 (T) = (0 -33.67035373854895) / Math.Sqrt((4.561413301599151 / (299)) + (22.28195065929943 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.13590879055917304 = (38.96620330200766 - 33.67035373854895) / 38.96620330200766 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Guid.ParseExactD

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 905.9193570924282 < 1.072119145256871.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 48.772805443403584 (T) = (0 -926.2180663131919) / Math.Sqrt((1149.3550519091457 / (299)) + (569.7588379986026 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.21368812096022474 = (1177.9270935652996 - 926.2180663131919) / 1177.9270935652996 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Guid.ctor_str

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 980.025290441408 < 1.1457803626507248.
IsChangePoint: Marked as a change because one of 8/1/2022 8:26:50 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 65.73941217163227 (T) = (0 -990.0197627378443) / Math.Sqrt((1408.6152407492737 / (299)) + (283.4176285154013 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.2056882634873907 = (1246.3869249678753 - 990.0197627378443) / 1246.3869249678753 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf8 - Duration of single invocation 2.49 μs 2.08 μs 0.83 0.09 False
EncodeUtf16 - Duration of single invocation 13.49 μs 11.61 μs 0.86 0.08 False
EncodeUtf8 - Duration of single invocation 3.45 μs 3.24 μs 0.94 0.16 False
EncodeUtf16 - Duration of single invocation 1.22 μs 1.08 μs 0.89 0.06 False
EncodeUtf16 - Duration of single invocation 1.16 μs 978.96 ns 0.84 0.04 False
EncodeUtf16 - Duration of single invocation 12.73 μs 10.94 μs 0.86 0.05 False
EncodeUtf16 - Duration of single invocation 1.27 μs 1.15 μs 0.91 0.04 False
EncodeUtf16 - Duration of single invocation 12.83 μs 10.46 μs 0.82 0.05 False
EncodeUtf8 - Duration of single invocation 3.40 μs 2.92 μs 0.86 0.17 False
EncodeUtf8 - Duration of single invocation 2.45 μs 2.28 μs 0.93 0.12 False
EncodeUtf16 - Duration of single invocation 12.89 μs 10.74 μs 0.83 0.07 False
EncodeUtf16 - Duration of single invocation 13.27 μs 11.33 μs 0.85 0.05 False
EncodeUtf16 - Duration of single invocation 12.75 μs 10.63 μs 0.83 0.06 False
EncodeUtf8 - Duration of single invocation 2.98 μs 2.57 μs 0.86 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,&Hello+<World>!,16)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.0798941871091876 < 2.178992004701286.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 22.218851468883752 (T) = (0 -2073.5508507751033) / Math.Sqrt((2503.769734434324 / (299)) + (2474.3883407571307 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.10344944183014039 = (2312.809725987868 - 2073.5508507751033) / 2312.809725987868 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,&lorem ipsum=dolor sit amet,512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.612249738023953 < 12.858991089091234.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 62.823157476521054 (T) = (0 -11455.945064426523) / Math.Sqrt((356912.8563918271 / (299)) + (14677.65102089932 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1860482388052332 = (14074.476658924856 - 11455.945064426523) / 14074.476658924856 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,no &lt;escaping /&gt; required,16)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.2404871719582267 < 3.2731594217501874.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 18.738207159754545 (T) = (0 -3198.594318002866) / Math.Sqrt((22012.241527773796 / (299)) + (5904.850206909166 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.09347566083601838 = (3528.415266767891 - 3198.594318002866) / 3528.415266767891 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,hello "there",16)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0827611809253073 < 1.154421943527445.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 27.63367601394612 (T) = (0 -1094.2571717574488) / Math.Sqrt((1166.726123723505 / (299)) + (1172.2380486861746 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.15020343124702784 = (1287.669557625078 - 1094.2571717574488) / 1287.669557625078 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,no &lt;escaping /&gt; required,16)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 978.9598731168352 < 1.1138548012884746.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 52.57039832582193 (T) = (0 -996.6799297165718) / Math.Sqrt((3275.3400570558965 / (299)) + (143.44152609526355 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.17632213652276557 = (1210.0360758865033 - 996.6799297165718) / 1210.0360758865033 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,�2020,512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.941416698657617 < 12.188029959371763.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 25.182148928864166 (T) = (0 -10852.81529385427) / Math.Sqrt((132520.4826182284 / (299)) + (221506.44273860668 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.1828774142564745 = (13281.746806666655 - 10852.81529385427) / 13281.746806666655 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,�2020,16)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1474504110631965 < 1.210487252125964.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/29/2022 2:35:34 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 50.678500748924144 (T) = (0 -1150.9474640791466) / Math.Sqrt((1778.133198210456 / (299)) + (248.99496016385123 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.14502216125234496 = (1346.1722771259413 - 1150.9474640791466) / 1346.1722771259413 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: JavaScript,no escaping required,512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.459408253738605 < 12.120929789589038.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 66.43715231520227 (T) = (0 -10590.092145988037) / Math.Sqrt((300224.35835795844 / (299)) + (11352.811364732182 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.19232685725269186 = (13111.853775359838 - 10590.092145988037) / 13111.853775359838 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,no escaping required,16)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.9219618969521624 < 3.0200977435757763.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 13.119242020301007 (T) = (0 -2929.458822095682) / Math.Sqrt((13076.357729016998 / (299)) + (9876.577392772477 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.08439353831005325 = (3199.4737309834422 - 2929.458822095682) / 3199.4737309834422 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,hello "there",16)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2777387877997914 < 2.3331023901606316.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 14.317695714207904 (T) = (0 -2291.0306155243256) / Math.Sqrt((4632.546499596162 / (299)) + (4448.190313115254 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.08610887721070357 = (2506.8966733497177 - 2291.0306155243256) / 2506.8966733497177 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,hello "there",512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.739799925189761 < 12.092279581531061.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 18.563021695344666 (T) = (0 -10803.440900898002) / Math.Sqrt((172905.52731404552 / (299)) + (390019.33075665974 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.18227396687938155 = (13211.565320563108 - 10803.440900898002) / 13211.565320563108 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: JavaScript,&Hello+&lt;World&gt;!,512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.326395098824785 < 12.666980382135659.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 61.66383942510976 (T) = (0 -11218.87093642297) / Math.Sqrt((189645.92046483545 / (299)) + (28929.83426901058 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.18681488230259388 = (13796.207889527092 - 11218.87093642297) / 13796.207889527092 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,no &lt;escaping /&gt; required,512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.630270368876506 < 12.074454764812133.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 70.58784282497272 (T) = (0 -10698.438666930582) / Math.Sqrt((234380.9905372937 / (299)) + (11837.228033530027 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1874369696363844 = (13166.287742802075 - 10698.438666930582) / 13166.287742802075 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,&lorem ipsum=dolor sit amet,16)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.573282754439769 < 2.68421957441497.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 24.248104279103238 (T) = (0 -2545.1213012193602) / Math.Sqrt((4783.822539164535 / (299)) + (3849.584692852671 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.1163447316400861 = (2880.219687868968 - 2545.1213012193602) / 2880.219687868968 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Get - Duration of single invocation 470.06 ms 436.06 ms 0.93 0.01 False
Get - Duration of single invocation 212.49 ms 199.69 ms 0.94 0.01 False
Get - Duration of single invocation 50.53 ms 47.25 ms 0.94 0.03 False
Get - Duration of single invocation 32.35 ms 30.37 ms 0.94 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 32, KeysCountPerProvider: 40)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 436.0559171333333 < 444.988171045.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 23.443323076872957 (T) = (0 -444380507.8346917) / Math.Sqrt((568944158165276.9 / (141)) + (16997292413069.799 / (24))) is greater than 1.9746246209661458 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (141) + (24) - 2, .975) and 0.10305511681209213 = (495437920.6170186 - 444380507.8346917) / 495437920.6170186 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 32, KeysCountPerProvider: 20)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 199.68666866666666 < 202.03156168511904.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 25.3416955762903 (T) = (0 -199141762.54977614) / Math.Sqrt((97823515459885.94 / (144)) + (3424372829335.8643 / (27))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (144) + (27) - 2, .975) and 0.10254113457118226 = (221895142.18528953 - 199141762.54977614) / 221895142.18528953 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 16, KeysCountPerProvider: 20)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 47.24708866071428 < 48.0929612551488.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 26.773392740940114 (T) = (0 -47681322.41092523) / Math.Sqrt((4284166896791.013 / (144)) + (167834874703.82898 / (28))) is greater than 1.9740167076307873 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (144) + (28) - 2, .975) and 0.09597233838491454 = (52743211.77931706 - 47681322.41092523) / 52743211.77931706 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 8, KeysCountPerProvider: 40)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.37323575 < 30.712851367142854.
IsChangePoint: Marked as a change because one of 9/22/2022 3:16:12 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 25.939669272240636 (T) = (0 -30506759.657399837) / Math.Sqrt((1422746530317.6008 / (144)) + (74251593348.16982 / (26))) is greater than 1.9741851911431834 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (144) + (26) - 2, .975) and 0.08755700032081828 = (33434153.87933945 - 30506759.657399837) / 33434153.87933945 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Tests.Perf_Int64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseSpan - Duration of single invocation 457.01 ns 417.12 ns 0.91 0.09 False
Parse - Duration of single invocation 431.99 ns 397.59 ns 0.92 0.10 False
Parse - Duration of single invocation 443.23 ns 398.53 ns 0.90 0.12 False
ToString - Duration of single invocation 643.36 ns 521.98 ns 0.81 0.08 False
TryFormat - Duration of single invocation 449.29 ns 344.92 ns 0.77 0.09 False
ToString - Duration of single invocation 536.39 ns 448.56 ns 0.84 0.05 False
TryFormat - Duration of single invocation 538.96 ns 422.14 ns 0.78 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int64*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Int64.ParseSpan(value: "-9223372036854775808")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 417.1166474803514 < 431.4588921777883.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:26:25 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 10.317807834647112 (T) = (0 -421.36590743763975) / Math.Sqrt((142.039697236593 / (299)) + (148.51791503501562 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.06175727469040397 = (449.10117187276865 - 421.36590743763975) / 449.10117187276865 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.Parse(value: "9223372036854775807")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 397.59227237750287 < 413.5386356008287.
IsChangePoint: Marked as a change because one of 8/4/2022 6:25:19 PM, 8/5/2022 6:57:51 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 9.033573525396488 (T) = (0 -398.0836962294898) / Math.Sqrt((354.4024401328099 / (299)) + (227.87949826069675 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.06577250529977301 = (426.11001976261264 - 398.0836962294898) / 426.11001976261264 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.Parse(value: "-9223372036854775808")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 398.53474898021574 < 411.36532852155614.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 13.011815445959861 (T) = (0 -401.6027298355674) / Math.Sqrt((163.46674646869033 / (299)) + (125.90616487653027 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.06886059275872367 = (431.3024738426781 - 401.6027298355674) / 431.3024738426781 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.ToString(value: -9223372036854775808)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 521.9781772015599 < 604.4648741139035.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 48.585674842661355 (T) = (0 -521.1967158154506) / Math.Sqrt((427.8965167609537 / (299)) + (98.43241220992739 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.17172439552568097 = (629.2551814878552 - 521.1967158154506) / 629.2551814878552 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryFormat(value: 9223372036854775807)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 344.92318115925406 < 425.26577478600234.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 88.65095804221698 (T) = (0 -337.33748256040377) / Math.Sqrt((118.84428142564472 / (299)) + (25.9572210534816 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.23692125071519096 = (442.0742719890592 - 337.33748256040377) / 442.0742719890592 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.ToString(value: 9223372036854775807)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 448.56142378793527 < 508.54023120787343.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 70.38119644153655 (T) = (0 -445.065954220157) / Math.Sqrt((230.3837488262497 / (299)) + (22.181896242936862 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.16505585861258867 = (533.0487779465092 - 445.065954220157) / 533.0487779465092 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryFormat(value: -9223372036854775808)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 422.1362531177641 < 512.0492055913194.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 45.695123402192905 (T) = (0 -426.285885367847) / Math.Sqrt((188.61214837715633 / (299)) + (131.18953012090157 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.20928892807528218 = (539.1171320393917 - 426.285885367847) / 539.1171320393917 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToWriter - Duration of single invocation 338.27 μs 312.60 μs 0.92 0.05 False
SerializeToUtf8Bytes - Duration of single invocation 333.50 μs 289.16 μs 0.87 0.01 False
SerializeObjectProperty - Duration of single invocation 369.39 μs 315.35 μs 0.85 0.01 False
SerializeToStream - Duration of single invocation 335.33 μs 297.73 μs 0.89 0.01 False
SerializeToString - Duration of single invocation 354.67 μs 307.69 μs 0.87 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToWriter

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 312.59594205357143 < 316.88986284531836.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 31.796198558298066 (T) = (0 -300074.77318572387) / Math.Sqrt((25405465.14170624 / (176)) + (30784777.811850276 / (25))) is greater than 1.9719565442516196 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (176) + (25) - 2, .975) and 0.11054473457695711 = (337369.15711326106 - 300074.77318572387) / 337369.15711326106 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;.SerializeToUtf8Bytes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 289.15617189685315 < 318.5445281715425.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 33.22440890801946 (T) = (0 -299356.0293615499) / Math.Sqrt((18417896.029401377 / (299)) + (32124407.218006104 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.11221996655851856 = (337196.1725711441 - 299356.0293615499) / 337196.1725711441 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;.SerializeObjectProperty

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 315.3515200480769 < 345.6015907650162.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 34.036250141703924 (T) = (0 -327628.6557305649) / Math.Sqrt((19946352.609900553 / (299)) + (27837417.72172209 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.10506375889475257 = (366091.6171256436 - 327628.6557305649) / 366091.6171256436 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;.SerializeToStream

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 297.7262303852201 < 323.30149637179403.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 40.181437737431395 (T) = (0 -303936.1207302208) / Math.Sqrt((21077694.700422958 / (299)) + (23537671.923335873 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.11550869826373754 = (343628.16246309277 - 303936.1207302208) / 343628.16246309277 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;.SerializeToString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 307.68928780976677 < 337.1200040374677.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 33.81435583486581 (T) = (0 -319415.79357073415) / Math.Sqrt((30170163.952218268 / (299)) + (31738989.382344365 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.10682620311868243 = (357618.8583745222 - 319415.79357073415) / 357618.8583745222 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Globalization.Tests.StringEquality

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Compare_Same_Upper - Duration of single invocation 372.66 ns 343.15 ns 0.92 0.07 False
Compare_DifferentFirstChar - Duration of single invocation 157.47 ns 144.80 ns 0.92 0.20 False
Compare_DifferentFirstChar - Duration of single invocation 179.19 ns 154.21 ns 0.86 0.18 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringEquality*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count: 1024, Options: (en-US, Ordinal))

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 343.147841014426 < 351.7385857883508.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 19.482853343065415 (T) = (0 -326.9597473694889) / Math.Sqrt((73.20648691645337 / (299)) + (88.93068669993163 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.10409503094977603 = (364.94913932234226 - 326.9597473694889) / 364.94913932234226 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringEquality.Compare_DifferentFirstChar(Count: 1024, Options: (en-US, OrdinalIgnoreCase))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 144.79875222685362 < 151.03233585338614.
IsChangePoint: Marked as a change because one of 8/2/2022 9:12:32 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 17.956070895088875 (T) = (0 -145.8778457960394) / Math.Sqrt((44.63034817532062 / (299)) + (24.391916765168727 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11730261487871563 = (165.26371127291318 - 145.8778457960394) / 165.26371127291318 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringEquality.Compare_DifferentFirstChar(Count: 1024, Options: (en-US, Ordinal))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 154.21183136846972 < 172.63100079589805.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 38.997310270349175 (T) = (0 -139.48293724039777) / Math.Sqrt((34.757221394527434 / (299)) + (22.052728276366313 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.21263449054139674 = (177.15144436070992 - 139.48293724039777) / 177.15144436070992 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Memory.ReadOnlySequence

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Slice_StartPosition_And_EndPosition - Duration of single invocation 80.43 ns 72.11 ns 0.90 0.18 False
Slice_Start_And_Length - Duration of single invocation 77.93 ns 70.14 ns 0.90 0.18 False
Slice_Start_And_EndPosition - Duration of single invocation 112.99 ns 104.51 ns 0.92 0.19 False
Slice_StartPosition_And_Length - Duration of single invocation 114.27 ns 103.15 ns 0.90 0.22 False
Slice_StartPosition_And_EndPosition - Duration of single invocation 60.40 ns 51.72 ns 0.86 0.10 False
Slice_StartPosition_And_Length - Duration of single invocation 90.66 ns 80.75 ns 0.89 0.20 False
Slice_Repeat_StartPosition_And_EndPosition - Duration of single invocation 455.86 ns 419.63 ns 0.92 0.11 False
Slice_Repeat_StartPosition_And_EndPosition - Duration of single invocation 353.17 ns 306.20 ns 0.87 0.13 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySequence*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.ReadOnlySequence.Slice_StartPosition_And_EndPosition(Segment: Multiple)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 72.10510002101559 < 76.69407823888503.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 17.945951779027766 (T) = (0 -72.39227999814463) / Math.Sqrt((11.977379243253894 / (299)) + (6.961033591259489 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.11917382671796557 = (82.18679484557633 - 72.39227999814463) / 82.18679484557633 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySequence.Slice_Start_And_Length(Segment: Single)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 70.1426283114227 < 73.31192196845424.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 10.70351659315381 (T) = (0 -70.47682422177778) / Math.Sqrt((10.489723991333612 / (299)) + (9.978884962733032 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.0894971809610134 = (77.40428996822254 - 70.47682422177778) / 77.40428996822254 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySequence.Slice_Start_And_EndPosition(Segment: Multiple)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 104.50616191292389 < 109.94406398600741.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 19.924863500696116 (T) = (0 -104.32317413850483) / Math.Sqrt((17.890980974456713 / (299)) + (8.32657469247319 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10858559782514639 = (117.0310619662184 - 104.32317413850483) / 117.0310619662184 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySequence.Slice_StartPosition_And_Length(Segment: Multiple)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 103.14786011584901 < 108.78904385574275.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 15.884986201237844 (T) = (0 -104.13995904673959) / Math.Sqrt((18.274232507926833 / (299)) + (14.478914597879864 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.10877740701266751 = (116.85067217345532 - 104.13995904673959) / 116.85067217345532 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySequence.Slice_StartPosition_And_EndPosition(Segment: Single)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 51.723820970740384 < 57.14863971051663.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 36.54212401204226 (T) = (0 -50.92312628090365) / Math.Sqrt((5.437262739326994 / (299)) + (1.4941949615097476 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.16288543885822596 = (60.83172918584472 - 50.92312628090365) / 60.83172918584472 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySequence.Slice_StartPosition_And_Length(Segment: Single)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 80.74816770680948 < 87.33258154659508.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 18.95109027010748 (T) = (0 -81.72869861534512) / Math.Sqrt((12.727478768615022 / (299)) + (6.543315309906415 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.11013807524342105 = (91.84424722712116 - 81.72869861534512) / 91.84424722712116 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySequence.Slice_Repeat_StartPosition_And_EndPosition(Segment: Multiple)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 419.6267415713895 < 431.17399145609915.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 34.92481217099934 (T) = (0 -406.6520417274158) / Math.Sqrt((184.94004948622984 / (299)) + (45.31237572980091 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.11809404215575914 = (461.10590149708145 - 406.6520417274158) / 461.10590149708145 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySequence.Slice_Repeat_StartPosition_And_EndPosition(Segment: Single)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 306.19952113421584 < 333.65103990544867.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 27.26013412222278 (T) = (0 -303.5716332992881) / Math.Sqrt((120.69742581442868 / (299)) + (89.08299292584724 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1494695162731925 = (356.9203445467524 - 303.5716332992881) / 356.9203445467524 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToWriter - Duration of single invocation 1.89 μs 1.70 μs 0.90 0.24 False
SerializeToUtf8Bytes - Duration of single invocation 4.22 μs 3.96 μs 0.94 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToWriter

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7032960218520508 < 1.7673432644893894.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 21.91210615597774 (T) = (0 -1694.4613142769938) / Math.Sqrt((3430.9649425076013 / (174)) + (940.280412850423 / (22))) is greater than 1.9722675325819379 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (174) + (22) - 2, .975) and 0.09272293263047361 = (1867.6337970161146 - 1694.4613142769938) / 1867.6337970161146 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;.SerializeToUtf8Bytes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.956176047008547 < 4.050924824960299.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 22.328960343914655 (T) = (0 -4037.478597860785) / Math.Sqrt((8472.03894175685 / (299)) + (5424.377599006474 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.07845430431413729 = (4381.2028169215 - 4037.478597860785) / 4381.2028169215 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableArray - Duration of single invocation 11.67 μs 10.80 μs 0.93 0.10 False
LinkedList - Duration of single invocation 20.42 μs 16.60 μs 0.81 0.05 False
HashSet - Duration of single invocation 17.43 μs 16.09 μs 0.92 0.08 False
Span - Duration of single invocation 4.13 μs 3.73 μs 0.90 0.07 False
ReadOnlySpan - Duration of single invocation 4.13 μs 3.73 μs 0.90 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IterateForEach<Int32>.ImmutableArray(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.802644212206712 < 11.090115807249866.
IsChangePoint: Marked as a change because one of 8/1/2022 8:26:50 AM, 9/28/2022 3:03:11 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 23.69026421570603 (T) = (0 -10974.784322474023) / Math.Sqrt((195718.8297105012 / (299)) + (31263.889942679707 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.0851050307581582 = (11995.676762293975 - 10974.784322474023) / 11995.676762293975 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;Int32&gt;.LinkedList(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.60089026744598 < 17.197965263367124.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 21.47581118513716 (T) = (0 -16923.18027226599) / Math.Sqrt((115040.70036725512 / (299)) + (78108.92831086276 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.07238162426338035 = (18243.688045557883 - 16923.18027226599) / 18243.688045557883 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;Int32&gt;.HashSet(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.093895678387334 < 16.574746491459194.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 21.244514972285877 (T) = (0 -16213.997665022653) / Math.Sqrt((511909.01494026964 / (299)) + (72565.47758379845 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.08173291161216929 = (17657.169542566313 - 16213.997665022653) / 17657.169542566313 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;Int32&gt;.Span(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.729321944311473 < 3.927367394416925.
IsChangePoint: Marked as a change because one of 9/13/2022 12:23:36 PM, 9/26/2022 6:11:26 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 19.844377981664298 (T) = (0 -3676.5185833340747) / Math.Sqrt((92410.04475499118 / (299)) + (3284.047809974458 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.10073353741255409 = (4088.35282565268 - 3676.5185833340747) / 4088.35282565268 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;Int32&gt;.ReadOnlySpan(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.734166200480192 < 3.928548798295487.
IsChangePoint: Marked as a change because one of 9/13/2022 12:23:36 PM, 9/26/2022 10:49:59 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 25.11087017502023 (T) = (0 -3674.92293860749) / Math.Sqrt((18923.318070957255 / (299)) + (4710.593651801679 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.09531789955770019 = (4062.115230102173 - 3674.92293860749) / 4062.115230102173 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Formats.Cbor.Tests.Perf_CborWriter

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteCoseKey - Duration of single invocation 17.96 μs 16.32 μs 0.91 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Formats.Cbor.Tests.Perf_CborWriter*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Formats.Cbor.Tests.Perf_CborWriter.WriteCoseKey(publicKey: ECDSA_P521)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.320931545102184 < 16.725002000104165.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 35.377155056216516 (T) = (0 -16486.64514861552) / Math.Sqrt((108486.54986158774 / (299)) + (50126.97376253756 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.09180879999161101 = (18153.275597102496 - 16486.64514861552) / 18153.275597102496 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToWriter - Duration of single invocation 7.15 μs 6.51 μs 0.91 0.00 False
SerializeToUtf8Bytes - Duration of single invocation 9.73 μs 8.75 μs 0.90 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;LoginViewModel&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToWriter

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.514213962297392 < 6.802247798392955.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 42.2683043542986 (T) = (0 -6541.223082193493) / Math.Sqrt((15116.259482017786 / (176)) + (6857.193272127628 / (25))) is greater than 1.9719565442516196 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (176) + (25) - 2, .975) and 0.10923820886725996 = (7343.403306371421 - 6541.223082193493) / 7343.403306371421 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;LoginViewModel&gt;.SerializeToUtf8Bytes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.749782946594395 < 9.072937403667318.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 38.45909033474382 (T) = (0 -8825.146324873736) / Math.Sqrt((18063.989420046637 / (299)) + (12512.484128398379 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.09208469313578381 = (9720.23079482411 - 8825.146324873736) / 9720.23079482411 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_ParseThenWrite

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseThenWrite - Duration of single invocation 455.84 μs 404.94 μs 0.89 0.02 False
ParseThenWrite - Duration of single invocation 703.22 μs 644.44 μs 0.92 0.01 False
ParseThenWrite - Duration of single invocation 136.75 μs 125.12 μs 0.91 0.00 False
ParseThenWrite - Duration of single invocation 583.19 μs 536.18 μs 0.92 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_ParseThenWrite*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: DeepTree)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 404.94295758928575 < 426.9899908883929.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 60.23626014878787 (T) = (0 -400574.60463355674) / Math.Sqrt((119137233.15468997 / (299)) + (22313645.303130284 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.14592606072153363 = (469016.3066817936 - 400574.60463355674) / 469016.3066817936 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: BroadTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 644.4387079861111 < 665.0968193751561.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 48.3773197453425 (T) = (0 -639021.3905804417) / Math.Sqrt((364017786.7496114 / (299)) + (74451672.67245676 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.13264883213637924 = (736750.4815315118 - 639021.3905804417) / 736750.4815315118 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: LotsOfNumbers)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 125.11693080357145 < 127.44118710323818.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 37.5679377724151 (T) = (0 -124245.05381037956) / Math.Sqrt((15156167.084153919 / (299)) + (3546846.9189748433 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1203922538356398 = (141250.52257914472 - 124245.05381037956) / 141250.52257914472 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 536.1758932360742 < 540.4968428651148.
IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 32.125899741584 (T) = (0 -530314.7179817919) / Math.Sqrt((159633665.05855912 / (299)) + (76410681.86373203 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.10815050092978763 = (594623.55311593 - 530314.7179817919) / 594623.55311593 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_DocumentParse

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 54.00 μs 50.66 μs 0.94 0.00 False
Parse - Duration of single invocation 13.99 ms 12.59 ms 0.90 0.02 False
Parse - Duration of single invocation 4.72 μs 4.13 μs 0.87 0.04 False
Parse - Duration of single invocation 20.57 μs 18.97 μs 0.92 0.01 False
Parse - Duration of single invocation 28.72 μs 27.07 μs 0.94 0.01 False
Parse - Duration of single invocation 24.45 μs 22.62 μs 0.93 0.01 False
Parse - Duration of single invocation 78.34 μs 73.04 μs 0.93 0.00 False
Parse - Duration of single invocation 25.36 μs 23.87 μs 0.94 0.01 False
Parse - Duration of single invocation 124.25 ms 108.40 ms 0.87 0.06 False
Parse - Duration of single invocation 17.35 ms 15.85 ms 0.91 0.02 False
Parse - Duration of single invocation 11.06 μs 10.19 μs 0.92 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: BasicJson)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 50.66497711038962 < 50.94551591592089.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 43.99654381642947 (T) = (0 -49756.51912433016) / Math.Sqrt((1359399.702242669 / (299)) + (397943.05272699817 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.11228910191682522 = (56050.364180240336 - 49756.51912433016) / 56050.364180240336 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.585318632142856 < 13.358077975961539.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 47.796423998689306 (T) = (0 -12671837.161378453) / Math.Sqrt((438781887277.58325 / (299)) + (34273533684.754547 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.16732831056087014 = (15218287.498058131 - 12671837.161378453) / 15218287.498058131 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: HelloWorld)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.125160010950504 < 4.348515809559307.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 42.98208140322183 (T) = (0 -4159.242001413707) / Math.Sqrt((41342.9160381867 / (299)) + (4746.768279694841 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.15768129550736634 = (4937.8483218166275 - 4159.242001413707) / 4937.8483218166275 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: BasicJson)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.974597659551986 < 19.716421542843293.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 48.99123525257467 (T) = (0 -18927.086146251993) / Math.Sqrt((646530.3812759866 / (299)) + (58261.5265809792 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.14784606769280226 = (22210.87696562886 - 18927.086146251993) / 22210.87696562886 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.07366825539568 < 27.298033619124638.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 47.141510502143994 (T) = (0 -26618.277394469078) / Math.Sqrt((1448385.0746384156 / (299)) + (107087.2739281854 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.14359109530644185 = (31081.271164495516 - 26618.277394469078) / 31081.271164495516 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.61544228316326 < 23.26426976090879.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 46.40517300542846 (T) = (0 -22602.2700658555) / Math.Sqrt((987213.4186565798 / (299)) + (76973.82959067493 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.1447956571400202 = (26429.087100129596 - 22602.2700658555) / 26429.087100129596 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 73.03832432827102 < 75.16628862422588.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 48.86397987953829 (T) = (0 -72320.68502464204) / Math.Sqrt((2531875.747794581 / (299)) + (873655.1527389401 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.12171588082282149 = (82343.15461879893 - 72320.68502464204) / 82343.15461879893 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: BasicJson)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.8674601717033 < 24.053153297040456.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 49.89543612576699 (T) = (0 -23310.610904431498) / Math.Sqrt((973803.3710409033 / (299)) + (73814.3015757964 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.14212516641135672 = (27172.508146577817 - 23310.610904431498) / 27172.508146577817 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 108.40467696666667 < 118.50261825380952.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 53.14498359510777 (T) = (0 -105940069.49250677) / Math.Sqrt((8527684733474.968 / (299)) + (3222931189896.5073 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.170820941476671 = (127765008.53889588 - 105940069.49250677) / 127765008.53889588 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.849471099999999 < 16.623161346858975.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 48.811091933314216 (T) = (0 -15915158.516789015) / Math.Sqrt((739965414621.2706 / (299)) + (48016536613.681854 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1666778601316595 = (19098446.753501 - 15915158.516789015) / 19098446.753501 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: HelloWorld)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.194205929382232 < 10.417591155687052.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 52.4303895122101 (T) = (0 -10137.239699404248) / Math.Sqrt((71005.3532607177 / (299)) + (10990.099057300125 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.11730169370450458 = (11484.376515854181 - 10137.239699404248) / 11484.376515854181 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-improvement runtime-mono untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kotlarmilos