Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Windows/x64: 14 Improvements on 1/11/2023 8:19:46 AM #11699

Closed
performanceautofiler bot opened this issue Jan 17, 2023 · 1 comment
Closed

[Perf] Windows/x64: 14 Improvements on 1/11/2023 8:19:46 AM #11699

performanceautofiler bot opened this issue Jan 17, 2023 · 1 comment
Assignees
@kunalspathak
Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-improvement runtime-coreclr untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 71bb0d481086b8b8a89a99a17ec80a861f32dc5d
Compare 5385f045f9aee91c22922d21552c4317198710f1
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromUtf8Bytes - Duration of single invocation 93.89 ns 85.21 ns 0.91 0.05 False
DeserializeFromUtf8Bytes - Duration of single invocation 90.23 ns 84.13 ns 0.93 0.06 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 85.20970505457376 < 89.1088766561678.
IsChangePoint: Marked as a change because one of 12/5/2022 3:14:07 PM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 34.104810308942206 (T) = (0 -85.32910318933537) / Math.Sqrt((4.044500669603929 / (212)) + (0.3567130638480651 / (30))) is greater than 1.9698976350765727 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (212) + (30) - 2, .975) and 0.06571383867593349 = (91.33080069216408 - 85.32910318933537) / 91.33080069216408 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;Int32&gt;.DeserializeFromUtf8Bytes(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 84.13326843552649 < 85.64543916496137.
IsChangePoint: Marked as a change because one of 12/5/2022 3:14:07 PM, 12/21/2022 9:23:17 AM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 23.57350995032281 (T) = (0 -84.3535140581799) / Math.Sqrt((8.868451619954685 / (212)) + (1.375205813808394 / (30))) is greater than 1.9698976350765727 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (212) + (30) - 2, .975) and 0.07642329671517059 = (91.3335229853296 - 84.3535140581799) / 91.3335229853296 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 71bb0d481086b8b8a89a99a17ec80a861f32dc5d
Compare 5385f045f9aee91c22922d21552c4317198710f1
Diff Diff

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsSuffix_SecondHalf - Duration of single invocation 152.12 ns 141.53 ns 0.93 0.00 False
IsPrefix_FirstHalf - Duration of single invocation 156.29 ns 143.85 ns 0.92 0.00 False
IsPrefix_FirstHalf - Duration of single invocation 156.30 ns 144.06 ns 0.92 0.01 False
LastIndexOf_Word_NotFound - Duration of single invocation 523.03 ns 410.27 ns 0.78 0.01 False
IsPrefix_FirstHalf - Duration of single invocation 154.70 ns 143.69 ns 0.93 0.01 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, IgnoreNonSpace, False))

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 141.53478791013893 < 144.19295089514353.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 32.15266621123199 (T) = (0 -142.36948410996519) / Math.Sqrt((0.3176423292308934 / (299)) + (2.2284907189910506 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.058372400469021764 = (151.19510534831284 - 142.36948410996519) / 151.19510534831284 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, IgnoreNonSpace, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 143.8493887377669 < 148.5307354644846.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 66.33741469648247 (T) = (0 -143.76580606503248) / Math.Sqrt((0.8713461899315349 / (299)) + (0.8749073740171796 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07633432969091569 = (155.6470167576158 - 143.76580606503248) / 155.6470167576158 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (, None, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 144.05810298256017 < 148.5101016614829.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 81.4761338565778 (T) = (0 -143.85698615137252) / Math.Sqrt((0.9374310556460869 / (299)) + (0.5306371836784859 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07555343964997945 = (155.6141721127767 - 143.85698615137252) / 155.6141721127767 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 410.2653103279839 < 496.91457913480815.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/13/2023 4:18:46 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 200.99477330730764 (T) = (0 -410.3670199347482) / Math.Sqrt((97.56841615563015 / (299)) + (0.03679627889522617 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.21905846179680535 = (525.4772602811327 - 410.3670199347482) / 525.4772602811327 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, None, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 143.69166620151574 < 148.3287937290018.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 68.41536216969016 (T) = (0 -143.82687737497216) / Math.Sqrt((1.5267323860976605 / (299)) + (0.7539673830245344 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07639729723157855 = (155.723751071606 - 143.82687737497216) / 155.723751071606 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 71bb0d481086b8b8a89a99a17ec80a861f32dc5d
Compare 5385f045f9aee91c22922d21552c4317198710f1
Diff Diff

Improvements in System.Hashing

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetStringHashCode - Duration of single invocation 389.32 ns 368.43 ns 0.95 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Hashing*'

Payloads

Baseline
Compare

Histogram

System.Hashing.GetStringHashCode(BytesCount: 1000)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 368.42988927697394 < 369.9746705242599.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 279.4438753107469 (T) = (0 -368.5042468602642) / Math.Sqrt((0.22598904725683003 / (299)) + (0.13646159826786033 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.05233938744102421 = (388.8567721150604 - 368.5042468602642) / 388.8567721150604 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 71bb0d481086b8b8a89a99a17ec80a861f32dc5d
Compare 5385f045f9aee91c22922d21552c4317198710f1
Diff Diff

Improvements in PerfLabTests.GetMember

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetMethod12 - Duration of single invocation 841.44 μs 781.55 μs 0.93 0.00 False
GetMethod20 - Duration of single invocation 1.41 ms 1.29 ms 0.91 0.01 False
GetMethod10 - Duration of single invocation 706.41 μs 659.09 μs 0.93 0.02 False
GetMethod2 - Duration of single invocation 136.22 μs 125.07 μs 0.92 0.04 False
GetMethod15 - Duration of single invocation 1.05 ms 986.96 μs 0.94 0.01 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.GetMember*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.GetMember.GetMethod12

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 781.5505158730158 < 808.0918854166667.
IsChangePoint: Marked as a change because one of 11/3/2022 9:01:15 AM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 64.88340029601899 (T) = (0 -778311.9861276456) / Math.Sqrt((77284597.6994202 / (299)) + (27669830.263572462 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.0830631375636488 = (848817.4246366628 - 778311.9861276456) / 848817.4246366628 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.GetMember.GetMethod20

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2927220486111113 < 1.3418383374368685.
IsChangePoint: Marked as a change because one of 11/2/2022 9:17:08 PM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 41.850126497711486 (T) = (0 -1307810.7866427556) / Math.Sqrt((217103758.72657248 / (299)) + (192468780.8328541 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07878013704206276 = (1419651.0944124856 - 1307810.7866427556) / 1419651.0944124856 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.GetMember.GetMethod10

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 659.0939756944443 < 669.2397741429766.
IsChangePoint: Marked as a change because one of 11/2/2022 9:17:08 PM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 53.34291505934309 (T) = (0 -648783.4343451747) / Math.Sqrt((66945657.736959726 / (299)) + (29057586.97007345 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.08238774486603812 = (707034.4044723542 - 648783.4343451747) / 707034.4044723542 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.GetMember.GetMethod2

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 125.0664857142857 < 126.02416591000808.
IsChangePoint: Marked as a change because one of 11/2/2022 9:17:08 PM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 40.76294043737647 (T) = (0 -125387.62066810903) / Math.Sqrt((3919491.163388816 / (299)) + (1025331.66093151 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.06602581728469749 = (134251.69880346698 - 125387.62066810903) / 134251.69880346698 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.GetMember.GetMethod15

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 986.9561764705882 < 0.9976992031249999.
IsChangePoint: Marked as a change because one of 11/2/2022 9:17:08 PM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 54.193159931445855 (T) = (0 -974301.0763346005) / Math.Sqrt((126250352.67666104 / (299)) + (49708944.14355533 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07424958807381864 = (1052444.6587146542 - 974301.0763346005) / 1052444.6587146542 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 71bb0d481086b8b8a89a99a17ec80a861f32dc5d
Compare 5385f045f9aee91c22922d21552c4317198710f1
Diff Diff

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ForeachOverList100Elements - Duration of single invocation 9.02 ms 6.27 ms 0.69 0.09 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.LowLevelPerf.ForeachOverList100Elements

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.265312980769231 < 8.588168017113096.
IsChangePoint: Marked as a change because one of 11/4/2022 9:11:54 AM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 35.57966911402608 (T) = (0 -6357071.632187439) / Math.Sqrt((197855673427.0797 / (299)) + (160998098586.72095 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.3029189834630732 = (9119559.249754269 - 6357071.632187439) / 9119559.249754269 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kunalspathak kunalspathak

Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-improvement runtime-coreclr untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@EgorBo @kunalspathak