Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 72 Improvements on 10/23/2022 1:14:54 PM #9236

Open
performanceautofiler bot opened this issue Oct 25, 2022 · 1 comment
Open

[Perf] Linux/x64: 72 Improvements on 10/23/2022 1:14:54 PM #9236

performanceautofiler bot opened this issue Oct 25, 2022 · 1 comment
Labels
arch-wasm arch-x64 branch-refs/heads/main kind-micro os-linux perf-improvement untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Oct 25, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare e232f7918900c30aa15cf2fca8056296a6bef1e5
Diff Diff

Improvements in Span.IndexerBench

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SameIndex1 - Duration of single invocation 22.49 μs 18.67 μs 0.83 0.12 False
CoveredIndex1 - Duration of single invocation 25.57 μs 21.75 μs 0.85 0.12 False
SameIndex2 - Duration of single invocation 25.91 μs 21.74 μs 0.84 0.05 False
ReadOnlyIndexer1 - Duration of single invocation 13.27 μs 11.29 μs 0.85 0.05 False
Indexer4 - Duration of single invocation 131.37 μs 111.90 μs 0.85 0.10 False
Indexer1 - Duration of single invocation 13.27 μs 11.28 μs 0.85 0.05 False
CoveredIndex2 - Duration of single invocation 25.62 μs 21.55 μs 0.84 0.11 False
WriteViaIndexer1 - Duration of single invocation 24.31 μs 20.50 μs 0.84 0.10 False
CoveredIndex3 - Duration of single invocation 36.67 μs 30.56 μs 0.83 0.14 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Span.IndexerBench*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Span.IndexerBench.SameIndex1(length: 1024)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.66843579404467 < 21.363974391276457.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM.
IsImprovementStdDev: Marked as improvement because 61.36269824675997 (T) = (0 -18684.780310627506) / Math.Sqrt((3146913.328461795 / (299)) + (340.64797268907967 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.2525179444187288 = (24996.961694414847 - 18684.780310627506) / 24996.961694414847 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.CoveredIndex1(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.751784464534424 < 24.28950444401011.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM.
IsImprovementStdDev: Marked as improvement because 60.11656477594083 (T) = (0 -21753.554802545994) / Math.Sqrt((3368032.4019381464 / (299)) + (217.07614379267082 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.22712342390370147 = (28146.220852520124 - 21753.554802545994) / 28146.220852520124 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.SameIndex2(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.74483512650882 < 24.61377649478303.
IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM.
IsImprovementStdDev: Marked as improvement because 70.31377404040067 (T) = (0 -21765.72605821868) / Math.Sqrt((2507512.1791640874 / (299)) + (659.2529721687966 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.22967536563149688 = (28255.264192689086 - 21765.72605821868) / 28255.264192689086 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.ReadOnlyIndexer1(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.293232569997588 < 12.57327492038875.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM.
IsImprovementStdDev: Marked as improvement because 55.175454301647974 (T) = (0 -11310.169137732346) / Math.Sqrt((704571.4323538989 / (299)) + (4982.076222136535 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.21592773222401085 = (14424.906481915876 - 11310.169137732346) / 14424.906481915876 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.Indexer4(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 111.90188105867348 < 124.82942034340657.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM.
IsImprovementStdDev: Marked as improvement because 57.13258983054446 (T) = (0 -111938.38882568025) / Math.Sqrt((95847851.7486639 / (299)) + (1074.6172333295312 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.2242481290860705 = (144296.64048866977 - 111938.38882568025) / 144296.64048866977 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.Indexer1(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.283766702866506 < 12.579714865581408.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM.
IsImprovementStdDev: Marked as improvement because 66.90709097518882 (T) = (0 -11276.734744144873) / Math.Sqrt((684373.8153952889 / (299)) + (27.722400311401817 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.22127085299910226 = (14480.946022856228 - 11276.734744144873) / 14480.946022856228 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.CoveredIndex2(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.553279133679254 < 24.352113165406937.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM.
IsImprovementStdDev: Marked as improvement because 57.48573485029748 (T) = (0 -21569.52502858697) / Math.Sqrt((4028300.9679123373 / (299)) + (92.3788401593988 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.23638374738595536 = (28246.550482325678 - 21569.52502858697) / 28246.550482325678 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.WriteViaIndexer1(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.497727517488958 < 23.091603526538748.
IsChangePoint: Marked as a change because one of 10/11/2022 1:45:10 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM.
IsImprovementStdDev: Marked as improvement because 49.609399806446525 (T) = (0 -20297.30747918355) / Math.Sqrt((958816.8811812779 / (299)) + (45717.588327159596 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.20275004010124423 = (25459.151458296896 - 20297.30747918355) / 25459.151458296896 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.CoveredIndex3(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.558031767775606 < 34.90362324277091.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM.
IsImprovementStdDev: Marked as improvement because 55.97209010364292 (T) = (0 -30564.663579155356) / Math.Sqrt((9368335.175218407 / (299)) + (1509.5296529140496 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.24553921166111914 = (40511.93123826952 - 30564.663579155356) / 40511.93123826952 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare bbcff6bc422da3d9b1904281a86014236ce50f2e
Diff Diff

Improvements in System.Collections.IndexerSet<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Span - Duration of single invocation 7.04 μs 6.02 μs 0.86 0.09 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IndexerSet<Int32>.Span(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.022314739865739 < 6.692420791721815.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM.
IsImprovementStdDev: Marked as improvement because 14.80597131557725 (T) = (0 -6143.517914762402) / Math.Sqrt((216538.57881169367 / (299)) + (63275.019270168654 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.21799423779551172 = (7856.103128247693 - 6143.517914762402) / 7856.103128247693 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare e232f7918900c30aa15cf2fca8056296a6bef1e5
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseUInt64Hex - Duration of single invocation 157.46 ns 146.23 ns 0.93 0.17 False
TryParseSByte - Duration of single invocation 220.91 ns 192.04 ns 0.87 0.21 False
TryParseInt64 - Duration of single invocation 582.74 ns 545.18 ns 0.94 0.09 False
TryParseUInt32Hex - Duration of single invocation 349.15 ns 311.17 ns 0.89 0.11 False
TryParseByte - Duration of single invocation 192.40 ns 168.30 ns 0.87 0.19 False
TryParseInt64 - Duration of single invocation 565.06 ns 530.61 ns 0.94 0.07 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value: 0)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 146.23265385271716 < 151.5500832976479.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM.
IsImprovementStdDev: Marked as improvement because 43.25438147702853 (T) = (0 -146.16583209572283) / Math.Sqrt((88.75403118085093 / (299)) + (0.2596694709703565 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.147181553600911 = (171.39149922575945 - 146.16583209572283) / 171.39149922575945 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSByte(value: 127)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 192.04279303645248 < 209.17648783727014.
IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM.
IsImprovementStdDev: Marked as improvement because 8.32451233347546 (T) = (0 -197.9682477986811) / Math.Sqrt((84.99961819016242 / (299)) + (71.3042776594162 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.13820564661109172 = (229.71634360354471 - 197.9682477986811) / 229.71634360354471 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: -9223372036854775808)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 545.177192381728 < 553.1323862719198.
IsChangePoint: Marked as a change because one of 10/11/2022 5:13:36 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM.
IsImprovementStdDev: Marked as improvement because 52.7390048163061 (T) = (0 -545.4899957289873) / Math.Sqrt((563.8128613407517 / (299)) + (0.9072971251679636 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.12125275577337669 = (620.7586986051576 - 545.4899957289873) / 620.7586986051576 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32Hex(value: FFFFFFFFFFFFFFFF)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 311.16627081488457 < 335.05660857570876.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM.
IsImprovementStdDev: Marked as improvement because 35.79385642467714 (T) = (0 -311.14437110127176) / Math.Sqrt((428.62503496347296 / (299)) + (15.141369443533694 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.19549311846913334 = (386.75165899041974 - 311.14437110127176) / 386.75165899041974 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseByte(value: 255)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 168.29521539320538 < 183.0489854434753.
IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM.
IsImprovementStdDev: Marked as improvement because 65.8441480151179 (T) = (0 -168.77782697937923) / Math.Sqrt((48.941824432929266 / (299)) + (0.6614830518172201 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.16956265591385694 = (203.23968831761923 - 168.77782697937923) / 203.23968831761923 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: 9223372036854775807)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 530.6095557350566 < 535.5728620422815.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/24/2022 9:01:44 AM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM.
IsImprovementStdDev: Marked as improvement because 52.24127951460963 (T) = (0 -528.163148836587) / Math.Sqrt((350.4653743064082 / (299)) + (2.6631404147320614 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.11823851498359444 = (598.9864127789164 - 528.163148836587) / 598.9864127789164 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare e232f7918900c30aa15cf2fca8056296a6bef1e5
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Guids

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteGuids - Duration of single invocation 213.24 ms 191.15 ms 0.90 0.04 False
WriteGuids - Duration of single invocation 209.11 ms 189.07 ms 0.90 0.05 False
WriteGuids - Duration of single invocation 195.34 ms 174.47 ms 0.89 0.01 False
WriteGuids - Duration of single invocation 190.43 ms 172.80 ms 0.91 0.01 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Guids*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: True, SkipValidation: False)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 191.14538453846154 < 201.31490728083332.
IsChangePoint: Marked as a change because one of 9/19/2022 9:42:49 AM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM.
IsImprovementStdDev: Marked as improvement because 47.60450366288845 (T) = (0 -191446847.12823567) / Math.Sqrt((28078475437498.047 / (299)) + (1474347906138.0073 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.12657046574114522 = (219189802.51874256 - 191446847.12823567) / 219189802.51874256 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: True, SkipValidation: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 189.0700664 < 198.98332667119047.
IsChangePoint: Marked as a change because one of 9/19/2022 6:07:30 AM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM.
IsImprovementStdDev: Marked as improvement because 23.917799514520702 (T) = (0 -189345522.6858608) / Math.Sqrt((115364745585300.44 / (299)) + (4725275538562.707 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.12719104027265224 = (216938106.07192835 - 189345522.6858608) / 216938106.07192835 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: False, SkipValidation: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 174.4681997333333 < 186.55295679333332.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM.
IsImprovementStdDev: Marked as improvement because 38.6799108060424 (T) = (0 -176043462.32307693) / Math.Sqrt((23214494315453.51 / (299)) + (2160368631320.4597 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.12691457777902015 = (201633720.87378633 - 176043462.32307693) / 201633720.87378633 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: False, SkipValidation: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 172.797667 < 183.907365.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM.
IsImprovementStdDev: Marked as improvement because 42.4842095142074 (T) = (0 -174141661.14603174) / Math.Sqrt((23910876394585.254 / (299)) + (1551946446544.1658 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.12431754990028634 = (198863938.77849475 - 174141661.14603174) / 198863938.77849475 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare e232f7918900c30aa15cf2fca8056296a6bef1e5
Diff Diff

Improvements in System.Collections.CreateAddAndClear<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Span - Duration of single invocation 11.70 μs 10.73 μs 0.92 0.08 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.CreateAddAndClear<String>.Span(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.731538536641397 < 11.126323359930813.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM.
IsImprovementStdDev: Marked as improvement because 29.998574315531453 (T) = (0 -10734.029806148266) / Math.Sqrt((223341.14624077833 / (299)) + (1599.832538088753 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.0836501027687705 = (11713.898630404568 - 10734.029806148266) / 11713.898630404568 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare bbcff6bc422da3d9b1904281a86014236ce50f2e
Diff Diff

Improvements in System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FixedTimeEquals_256Bit_Equal - Duration of single invocation 700.05 ns 581.92 ns 0.83 0.10 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_Equal

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 581.9154799155707 < 668.0363925019811.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM.
IsImprovementStdDev: Marked as improvement because 16.227996707668204 (T) = (0 -590.2209076241182) / Math.Sqrt((260.35678492835103 / (299)) + (280.62212818569907 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.17189223508462545 = (712.7344201203496 - 590.2209076241182) / 712.7344201203496 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-wasm arch-x64 branch-refs/heads/main kind-micro os-linux perf-improvement untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@lewing