Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Perf] Windows/x64: 14 Regressions on 10/6/2022 2:13:23 PM #8292

Closed
performanceautofiler bot opened this issue Oct 11, 2022 · 1 comment
Closed

[Perf] Windows/x64: 14 Regressions on 10/6/2022 2:13:23 PM #8292

performanceautofiler bot opened this issue Oct 11, 2022 · 1 comment

Comments

@performanceautofiler
Copy link

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 01cc57335aa6dacd28c928940642d0dc05c2bf32
Compare d462be072d71d98534da318d3418465f788ecea1
Diff Diff

Regressions in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
List - Duration of single invocation 329.63 ns 370.14 ns 1.12 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IterateForEach<Int32>.List(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 370.14429173043567 > 346.11316150528904.
IsChangePoint: Marked as a change because one of 9/29/2022 5:24:34 PM, 10/6/2022 11:18:47 AM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -30.219498926774577 (T) = (0 -370.807098287436) / Math.Sqrt((72.89164638010277 / (45)) + (0.7437748625508006 / (10))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (10) - 2, .025) and -0.11866436446543774 = (331.4730584670309 - 370.807098287436) / 331.4730584670309 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline 01cc57335aa6dacd28c928940642d0dc05c2bf32
Compare d462be072d71d98534da318d3418465f788ecea1
Diff Diff

Regressions in PerfLabTests.GetMember

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetMethod1 - Duration of single invocation 81.33 μs 89.30 μs 1.10 0.04 False
GetMethod10 - Duration of single invocation 842.24 μs 909.06 μs 1.08 0.01 False
GetMethod2 - Duration of single invocation 163.64 μs 179.10 μs 1.09 0.01 False
GetMethod12 - Duration of single invocation 1.01 ms 1.11 ms 1.10 0.01 False
GetMethod20 - Duration of single invocation 1.70 ms 1.82 ms 1.07 0.02 False
GetMethod4 - Duration of single invocation 326.84 μs 367.84 μs 1.13 0.02 False
GetMethod15 - Duration of single invocation 1.26 ms 1.38 ms 1.09 0.01 False
GetMethod5 - Duration of single invocation 426.72 μs 451.70 μs 1.06 0.01 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.GetMember*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

PerfLabTests.GetMember.GetMethod1


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 89.2952035440613 > 85.26204582253887.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 9/29/2022 5:24:34 PM, 10/6/2022 11:18:47 AM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -19.576743986564807 (T) = (0 -88922.66115866687) / Math.Sqrt((4233002.109395116 / (45)) + (234167.73839553585 / (11))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (11) - 2, .025) and -0.0808160104555788 = (82273.63427118807 - 88922.66115866687) / 82273.63427118807 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.GetMember.GetMethod10

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 909.0563988095237 > 874.3461677631581.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 9/29/2022 5:24:34 PM, 10/6/2022 11:18:47 AM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -18.314290456470086 (T) = (0 -905183.0151214525) / Math.Sqrt((276215324.93534225 / (45)) + (85013722.42404062 / (11))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (11) - 2, .025) and -0.08148137387950859 = (836984.3780798225 - 905183.0151214525) / 836984.3780798225 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.GetMember.GetMethod2

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 179.09641283524905 > 170.70821496710528.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 9/29/2022 5:24:34 PM, 10/6/2022 11:18:47 AM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -22.268687606886274 (T) = (0 -177814.9051593277) / Math.Sqrt((10618817.761439014 / (45)) + (1938802.603953755 / (11))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (11) - 2, .025) and -0.08743794803900683 = (163517.28894506945 - 177814.9051593277) / 163517.28894506945 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.GetMember.GetMethod12

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.1070109444444447 > 1044.64409765625.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 9/29/2022 5:24:34 PM, 10/6/2022 11:18:47 AM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -17.88519980929033 (T) = (0 -1083179.996142746) / Math.Sqrt((350303462.1907095 / (45)) + (118464416.87306523 / (11))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (11) - 2, .025) and -0.07656918945883696 = (1006140.624075664 - 1083179.996142746) / 1006140.624075664 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.GetMember.GetMethod20

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.821947546296296 > 1.7702588020833334.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 9/29/2022 5:24:34 PM, 10/6/2022 11:18:47 AM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -21.577170152159745 (T) = (0 -1813066.1898749399) / Math.Sqrt((920135280.416391 / (45)) + (95124020.05002782 / (11))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (11) - 2, .025) and -0.06859691541168957 = (1696679.22836595 - 1813066.1898749399) / 1696679.22836595 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.GetMember.GetMethod4

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 367.8414772727272 > 342.508538877097.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 9/29/2022 8:01:59 PM, 10/6/2022 11:18:47 AM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -14.74536877460617 (T) = (0 -360167.8552891209) / Math.Sqrt((62785035.16774487 / (45)) + (23299609.24855808 / (11))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (11) - 2, .025) and -0.08311660667510541 = (332529.15989788517 - 360167.8552891209) / 332529.15989788517 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.GetMember.GetMethod15

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.3770091840277778 > 1.3167525000000002.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 9/29/2022 5:24:34 PM, 10/6/2022 11:18:47 AM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -18.678831715263097 (T) = (0 -1355008.9630783403) / Math.Sqrt((512427013.67605823 / (45)) + (136463013.85086185 / (10))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (10) - 2, .025) and -0.07408083863384007 = (1261552.1237692148 - 1355008.9630783403) / 1261552.1237692148 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.GetMember.GetMethod5

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 451.6992142857143 > 443.931390224359.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 9/29/2022 5:24:34 PM, 10/6/2022 11:18:47 AM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -20.242557464060237 (T) = (0 -448016.97224432696) / Math.Sqrt((85901716.92728645 / (45)) + (5646466.800367275 / (11))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (11) - 2, .025) and -0.07563908607738183 = (416512.35813505616 - 448016.97224432696) / 416512.35813505616 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 01cc57335aa6dacd28c928940642d0dc05c2bf32
Compare d462be072d71d98534da318d3418465f788ecea1
Diff Diff

Regressions in StoreBlock.AnyLocation

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
InitBlockAllZeros128 - Duration of single invocation 1.39 ns 2.83 ns 2.04 0.18 False
InitBlockAllZeros64 - Duration of single invocation 0.78 ns 2.46 ns 3.16 0.20 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'StoreBlock.AnyLocation*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

StoreBlock.AnyLocation.InitBlockAllZeros128


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 2.831049293313455 > 1.4571446467348488.
IsChangePoint: Marked as a change because one of 8/14/2022 4:06:45 PM, 8/16/2022 10:54:40 PM, 9/13/2022 12:23:36 PM, 9/22/2022 3:11:11 PM, 9/27/2022 11:22:50 AM, 10/6/2022 11:18:47 AM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -6.287134077146315 (T) = (0 -2.261389181863345) / Math.Sqrt((0.13976570387654655 / (45)) + (0.12034250092855729 / (11))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (11) - 2, .025) and -0.49142492427850143 = (1.5162608221512224 - 2.261389181863345) / 1.5162608221512224 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### StoreBlock.AnyLocation.InitBlockAllZeros64

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 2.461553901213087 > 0.8123818312623254.
IsChangePoint: Marked as a change because one of 9/13/2022 12:23:36 PM, 9/20/2022 1:28:44 PM, 9/20/2022 10:35:57 PM, 9/27/2022 11:22:50 AM, 10/6/2022 2:13:23 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -2.4894372434992453 (T) = (0 -1.2840412370582304) / Math.Sqrt((0.23101106802411214 / (46)) + (0.17118152173316745 / (10))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (10) - 2, .025) and -0.40543741503681713 = (0.913623917593366 - 1.2840412370582304) / 0.913623917593366 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 01cc57335aa6dacd28c928940642d0dc05c2bf32
Compare d462be072d71d98534da318d3418465f788ecea1
Diff Diff

Regressions in Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ForEach_Array - Duration of single invocation 3.84 ns 7.20 ns 1.87 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark.ForEach_Array


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 7.204405364301362 > 4.037773623723169.
IsChangePoint: Marked as a change because one of 9/29/2022 5:24:34 PM, 10/6/2022 11:18:47 AM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -34.71595722104643 (T) = (0 -7.217389947190043) / Math.Sqrt((0.3140980794851172 / (45)) + (0.0036787378768660694 / (11))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (11) - 2, .025) and -0.6988747065063575 = (4.248335630372771 - 7.217389947190043) / 4.248335630372771 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 01cc57335aa6dacd28c928940642d0dc05c2bf32
Compare d462be072d71d98534da318d3418465f788ecea1
Diff Diff

Regressions in System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Enumerate - Duration of single invocation 197.96 ns 212.14 ns 1.07 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Guid, Guid&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Enumerate(Size: 100)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 212.14293946614083 > 208.00963074875236.
IsChangePoint: Marked as a change because one of 8/29/2022 8:26:03 AM, 8/29/2022 6:09:35 PM, 9/29/2022 5:24:34 PM, 10/6/2022 11:18:47 AM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -14.735759449318182 (T) = (0 -215.09685793798974) / Math.Sqrt((20.569715882805127 / (45)) + (7.655530712813791 / (11))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (11) - 2, .025) and -0.07940527386766197 = (199.273491750941 - 215.09685793798974) / 199.273491750941 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 01cc57335aa6dacd28c928940642d0dc05c2bf32
Compare d462be072d71d98534da318d3418465f788ecea1
Diff Diff

Regressions in LinqBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count00ForX - Duration of single invocation 124.37 ms 179.21 ms 1.44 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'LinqBenchmarks*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

LinqBenchmarks.Count00ForX


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 179.212855 > 131.02571868750002.
IsChangePoint: Marked as a change because one of 9/29/2022 5:24:34 PM, 10/6/2022 11:18:47 AM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -21.438208182164807 (T) = (0 -169032994.60206458) / Math.Sqrt((77353801923461.88 / (45)) + (23497154459513.773 / (11))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (11) - 2, .025) and -0.3315927470133917 = (126940459.07142857 - 169032994.60206458) / 126940459.07142857 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@DrewScoggins
Copy link
Owner

From revert here dotnet/runtime#76695

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant