[Perf] Linux/x64: 60 Improvements on 11/15/2022 10:02:59 PM #10015

performanceautofiler · 2022-11-22T12:22:00Z

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
CachedDefaultOptions - Duration of single invocation	7.08 μs	6.36 μs	0.90	0.17	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ColdStartSerialization&lt;SimpleStructWithProperties&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.CachedDefaultOptions

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.364282601192869 < 6.405159461130761.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 21.24367556526057 (T) = (0 -6204.476914363309) / Math.Sqrt((41697.520242600905 / (299)) + (18551.66555816073 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.09527883799764254 = (6857.888568264906 - 6204.476914363309) / 6857.888568264906 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
DeserializeFromReader - Duration of single invocation	12.82 μs	10.45 μs	0.82	0.17	False
DeserializeFromUtf8Bytes - Duration of single invocation	9.94 μs	8.68 μs	0.87	0.14	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;LargeStructWithProperties&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromReader

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.447304247762863 < 11.272954284819775.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 6.299221176897203 (T) = (0 -10360.760943138483) / Math.Sqrt((92734.25484316758 / (299)) + (878882.3109426539 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.1087274231210351 = (11624.682742309345 - 10360.760943138483) / 11624.682742309345 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;LargeStructWithProperties&gt;.DeserializeFromUtf8Bytes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.682833430989584 < 9.448288448883154.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 41.47343100842432 (T) = (0 -8652.222672179503) / Math.Sqrt((112908.4560607862 / (299)) + (19636.168980149592 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1421878952815839 = (10086.384447815255 - 8652.222672179503) / 10086.384447815255 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Select - Duration of single invocation	34.11 μs	29.93 μs	0.88	0.15	False
SelectToList - Duration of single invocation	33.23 μs	30.53 μs	0.92	0.13	False
WhereSingle_LastElementMatches - Duration of single invocation	22.05 μs	19.14 μs	0.87	0.28	False
TakeLastHalf - Duration of single invocation	47.41 μs	42.38 μs	0.89	0.16	False
Distinct - Duration of single invocation	49.69 μs	37.90 μs	0.76	0.22	False
SelectToList - Duration of single invocation	24.77 μs	21.28 μs	0.86	0.19	False
Select - Duration of single invocation	33.55 μs	30.55 μs	0.91	0.12	False
SelectToArray - Duration of single invocation	25.95 μs	22.79 μs	0.88	0.18	False
EmptyTakeSelectToArray - Duration of single invocation	1.04 μs	927.75 ns	0.89	0.15	False
Reverse - Duration of single invocation	33.02 μs	31.23 μs	0.95	0.17	False
TakeLastHalf - Duration of single invocation	32.93 μs	30.13 μs	0.92	0.12	False
Take_All - Duration of single invocation	42.08 μs	38.35 μs	0.91	0.13	False
WhereSingleOrDefault_LastElementMatches - Duration of single invocation	20.85 μs	18.94 μs	0.91	0.28	False
WhereAny_LastElementMatches - Duration of single invocation	21.67 μs	19.36 μs	0.89	0.25	False
WhereSelect - Duration of single invocation	27.55 μs	25.76 μs	0.94	0.21	False
Zip - Duration of single invocation	66.94 μs	61.50 μs	0.92	0.14	False
Intersect - Duration of single invocation	61.37 μs	55.78 μs	0.91	0.15	False
WhereLast_LastElementMatches - Duration of single invocation	21.92 μs	19.35 μs	0.88	0.25	False
WhereSingle_LastElementMatches - Duration of single invocation	11.72 μs	10.67 μs	0.91	0.26	False
Where - Duration of single invocation	34.37 μs	30.68 μs	0.89	0.16	False
WhereLast_LastElementMatches - Duration of single invocation	11.73 μs	9.90 μs	0.84	0.29	False
WhereFirst_LastElementMatches - Duration of single invocation	11.69 μs	9.65 μs	0.83	0.24	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.Select(input: IEnumerable)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.93468175158577 < 32.14946732961341.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 27.75735080467029 (T) = (0 -30608.871203590683) / Math.Sqrt((1035782.9864974944 / (299)) + (251112.73751630355 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.09959681211979309 = (33994.62775742971 - 30608.871203590683) / 33994.62775742971 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.SelectToList(input: Range)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.529572011689293 < 31.62371052500257.
IsChangePoint: Marked as a change because one of 9/26/2022 2:03:14 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 14.758639397642419 (T) = (0 -30490.396988121083) / Math.Sqrt((769878.6074709136 / (299)) + (876444.6667323072 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08734562416341116 = (33408.4816721247 - 30490.396988121083) / 33408.4816721247 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.141086747964412 < 20.766928354073.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 17.972655088489198 (T) = (0 -19163.291750454664) / Math.Sqrt((692005.6676548977 / (299)) + (477249.46631397656 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.12260588688005235 = (21841.144662244697 - 19163.291750454664) / 21841.144662244697 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.TakeLastHalf(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 42.377988143631434 < 46.73278723773345.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 28.251954176187795 (T) = (0 -42247.44549874583) / Math.Sqrt((2125896.3321546917 / (299)) + (1095387.696204278 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1331412010178165 = (48736.24810447836 - 42247.44549874583) / 48736.24810447836 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Distinct(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 37.90193921925566 < 41.999008658285746.
IsChangePoint: Marked as a change because one of 11/15/2022 10:02:59 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 22.430605213902478 (T) = (0 -39325.35113257605) / Math.Sqrt((1990360.7796212218 / (299)) + (1148229.6175185996 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.11946617176280051 = (44660.80674186492 - 39325.35113257605) / 44660.80674186492 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.SelectToList(input: List)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.28190507933812 < 22.97135241825298.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 8.773594599405472 (T) = (0 -22234.654307596353) / Math.Sqrt((409198.37838659826 / (299)) + (651077.8431715008 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.06245097384722518 = (23715.724391327123 - 22234.654307596353) / 23715.724391327123 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Select(input: IList)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.549845217589983 < 32.03813902828495.
IsChangePoint: Marked as a change because one of 9/14/2022 7:20:13 PM, 11/15/2022 5:29:43 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 14.418265123599529 (T) = (0 -30931.07196643195) / Math.Sqrt((794522.1894647963 / (299)) + (966549.7337518391 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.08655218097063835 = (33861.89262491163 - 30931.07196643195) / 33861.89262491163 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: List)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.79375919243986 < 23.74627025937361.
IsChangePoint: Marked as a change because one of 9/23/2022 4:51:48 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 15.231836634326584 (T) = (0 -22776.643342055264) / Math.Sqrt((472979.7308821746 / (299)) + (349332.0489243372 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.07830113168313016 = (24711.58870320421 - 22776.643342055264) / 24711.58870320421 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.EmptyTakeSelectToArray

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 927.7530042778711 < 0.9935173743136791.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 24.320338007467104 (T) = (0 -933.2598173018672) / Math.Sqrt((788.3769672417045 / (299)) + (402.78084539298334 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10300818136125663 = (1040.43292024465 - 933.2598173018672) / 1040.43292024465 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Reverse(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.226502868852464 < 31.252773377369458.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 17.747096464533062 (T) = (0 -30695.970036681392) / Math.Sqrt((1010826.8744245196 / (299)) + (413090.3470085547 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.07788269072464862 = (33288.57372908867 - 30695.970036681392) / 33288.57372908867 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.TakeLastHalf(input: List)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.133971467207576 < 31.235556281865065.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 11.475704763171905 (T) = (0 -30888.534816238418) / Math.Sqrt((552508.5088682533 / (299)) + (916906.1248847486 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.06920681392131775 = (33185.17505093482 - 30888.534816238418) / 33185.17505093482 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Take_All(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.3468229296875 < 39.89210587290761.
IsChangePoint: Marked as a change because one of 9/7/2022 8:46:47 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 15.873772923120578 (T) = (0 -38463.11752624083) / Math.Sqrt((1246348.2725117344 / (299)) + (1149678.8521985488 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08606956226562323 = (42085.388491481324 - 38463.11752624083) / 42085.388491481324 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.94208014959858 < 20.244078219796318.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 23.37826032610994 (T) = (0 -19080.117791410048) / Math.Sqrt((560314.3660246409 / (299)) + (282911.70656383305 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1273054044436211 = (21863.453593688962 - 19080.117791410048) / 21863.453593688962 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.356912307145755 < 20.17769286421366.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 16.231879367069332 (T) = (0 -19190.969760656855) / Math.Sqrt((555846.54539294 / (299)) + (508899.6677333824 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11382807358070589 = (21656.03444266255 - 19190.969760656855) / 21656.03444266255 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: List)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.76024737616792 < 26.007651542466537.
IsChangePoint: Marked as a change because one of 9/19/2022 6:07:30 AM, 9/23/2022 2:49:56 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 10.872131631845429 (T) = (0 -25592.883708694557) / Math.Sqrt((1292407.0762813103 / (299)) + (696213.4859154865 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.07197496656395344 = (27577.79454928707 - 25592.883708694557) / 27577.79454928707 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Zip(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 61.49767184660812 < 63.60895922074265.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 4.904976709722865 (T) = (0 -61393.72350937481) / Math.Sqrt((3078993.8309865906 / (299)) + (30713614.25800245 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08319375151616809 = (66964.77430308166 - 61393.72350937481) / 66964.77430308166 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Intersect(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 55.78285370203837 < 57.78746783400732.
IsChangePoint: Marked as a change because one of 9/5/2022 11:32:52 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 20.771215540814637 (T) = (0 -54489.456710327206) / Math.Sqrt((2814636.9147373773 / (299)) + (1422935.9575483557 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.09400718428831474 = (60143.36511876649 - 54489.456710327206) / 60143.36511876649 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.351976966583642 < 20.8318821221408.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 16.298933427865137 (T) = (0 -19532.818563279707) / Math.Sqrt((553326.1384880532 / (299)) + (474928.7391344392 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.11124132334201517 = (21977.640360968755 - 19532.818563279707) / 21977.640360968755 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: List)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.66567493236871 < 11.356428361714775.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 26.33261381061666 (T) = (0 -10449.511431831243) / Math.Sqrt((226500.1558861266 / (299)) + (87393.44051707059 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1432020919913754 = (12196.004838665009 - 10449.511431831243) / 12196.004838665009 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Where(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.68444409899376 < 33.216983588982934.
IsChangePoint: Marked as a change because one of 11/15/2022 10:02:59 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 16.235314063233155 (T) = (0 -31399.874119870863) / Math.Sqrt((1863001.8260344295 / (299)) + (865250.0567975903 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.09939417592174492 = (34865.27988202581 - 31399.874119870863) / 34865.27988202581 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input: List)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.903824312632699 < 11.123904215912146.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 18.059251945350745 (T) = (0 -10472.249088468201) / Math.Sqrt((173839.25531653862 / (299)) + (198732.00158477275 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.14207562344583877 = (12206.494388852561 - 10472.249088468201) / 12206.494388852561 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input: List)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.649436090225564 < 11.192787063000948.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 37.41485325807009 (T) = (0 -10125.103366764006) / Math.Sqrt((197174.22045103568 / (299)) + (40940.20205983746 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.15434247046261343 = (11973.054118376858 - 10125.103366764006) / 11973.054118376858 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in System.Collections.IterateForEach<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedDictionary - Duration of single invocation	341.70 μs	316.09 μs	0.93	0.15	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<String>.SortedDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 316.08925 < 325.09361757577335.
IsChangePoint: Marked as a change because one of 9/28/2022 6:26:25 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 14.97436891488003 (T) = (0 -311955.15160441643) / Math.Sqrt((75983765.9550731 / (299)) + (72486329.53798014 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.0813762381243099 = (339589.6824696233 - 311955.15160441643) / 339589.6824696233 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

performanceautofiler · 2022-11-22T12:22:01Z

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in System.Collections.Sort<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
LinqQuery - Duration of single invocation	1.61 ms	1.31 ms	0.81	0.10	False
LinqOrderByExtension - Duration of single invocation	1.52 ms	1.32 ms	0.87	0.15	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<Int32>.LinqQuery(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3114702266133338 < 1.501605466635.
IsChangePoint: Marked as a change because one of 10/2/2022 11:57:04 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 6.474860739604996 (T) = (0 -1306439.8241985666) / Math.Sqrt((42926960521.25258 / (299)) + (393020030.65328157 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.05910586859182242 = (1388508.845562996 - 1306439.8241985666) / 1388508.845562996 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;Int32&gt;.LinqOrderByExtension(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3244374400400003 < 1.4748929676476668.
IsChangePoint: Marked as a change because one of 10/2/2022 11:57:04 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 5.385430472949561 (T) = (0 -1311749.6673839898) / Math.Sqrt((41756174075.73501 / (299)) + (1158278997.8954613 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.05328079041302431 = (1385574.1534559815 - 1311749.6673839898) / 1385574.1534559815 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in System.Net.Primitives.Tests.CredentialCacheTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ForEach - Duration of single invocation	724.11 ns	656.22 ns	0.91	0.20	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Primitives.Tests.CredentialCacheTests*'

Payloads

Baseline
Compare

Histogram

System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 0, hostPortCount: 0)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 656.2236917265525 < 713.3743231291177.
IsChangePoint: Marked as a change because one of 9/24/2022 7:43:48 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 13.681671180392204 (T) = (0 -687.6884149917504) / Math.Sqrt((531.6293265392435 / (299)) + (356.30085753892575 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.0777850515335461 = (745.6921145501964 - 687.6884149917504) / 745.6921145501964 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
SerializeToString - Duration of single invocation	2.53 μs	2.34 μs	0.92	0.21	False
SerializeToUtf8Bytes - Duration of single invocation	2.32 μs	2.07 μs	0.89	0.19	False
SerializeToWriter - Duration of single invocation	2.15 μs	1.92 μs	0.90	0.20	False
SerializeObjectProperty - Duration of single invocation	3.89 μs	3.46 μs	0.89	0.20	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;SimpleStructWithProperties&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.337367361420992 < 2.4509651993594423.
IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 15.347081897063953 (T) = (0 -2337.248909782314) / Math.Sqrt((7428.355771573706 / (299)) + (5552.723983002771 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09510729899213612 = (2582.90171550627 - 2337.248909782314) / 2582.90171550627 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;SimpleStructWithProperties&gt;.SerializeToUtf8Bytes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.0741275165453343 < 2.215520957486843.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 16.04556780202822 (T) = (0 -2113.7878136314503) / Math.Sqrt((5026.785317852901 / (299)) + (5140.237812956894 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.10528268721264539 = (2362.5202993404346 - 2113.7878136314503) / 2362.5202993404346 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;SimpleStructWithProperties&gt;.SerializeToWriter

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.9246628022178571 < 2.047254985541929.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 21.391256689759057 (T) = (0 -1914.5036680565315) / Math.Sqrt((3441.452431196379 / (299)) + (2418.566558784316 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.1096323635989005 = (2150.2395075758027 - 1914.5036680565315) / 2150.2395075758027 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;SimpleStructWithProperties&gt;.SerializeObjectProperty

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.4590853194444446 < 3.7406309276818264.
IsChangePoint: Marked as a change because one of 10/26/2022 7:11:30 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 21.181804889189106 (T) = (0 -3584.702254685024) / Math.Sqrt((25644.538764037596 / (299)) + (10691.579033502938 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.12177084062498411 = (4081.739050017476 - 3584.702254685024) / 4081.739050017476 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in System.Collections.CreateAddAndClear<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ImmutableDictionary - Duration of single invocation	4.52 ms	4.09 ms	0.91	0.09	False
ImmutableHashSet - Duration of single invocation	4.45 ms	3.90 ms	0.88	0.08	False
SortedDictionary - Duration of single invocation	2.08 ms	1.89 ms	0.91	0.15	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<String>.ImmutableDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.086866668852459 < 4.163887793304416.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 18.97060991069282 (T) = (0 -4055633.2911202167) / Math.Sqrt((9274440430.960209 / (299)) + (9792272835.062908 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08926131520773113 = (4453125.093775136 - 4055633.2911202167) / 4453125.093775136 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;String&gt;.ImmutableHashSet(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.89754875170068 < 4.103425555.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 25.765770523834846 (T) = (0 -3953934.5366216786) / Math.Sqrt((8998329928.142607 / (299)) + (3906311375.0151505 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.08436810702352526 = (4318257.770345344 - 3953934.5366216786) / 4318257.770345344 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;String&gt;.SortedDictionary(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.8894851916666666 < 1.9667221622005209.
IsChangePoint: Marked as a change because one of 9/20/2022 3:55:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 19.337927430220194 (T) = (0 -1862756.5768336698) / Math.Sqrt((2919400671.3780723 / (299)) + (2502015772.987797 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.10343228154988703 = (2077652.9630732157 - 1862756.5768336698) / 2077652.9630732157 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in Microsoft.Extensions.DependencyInjection.GetService

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Scoped - Duration of single invocation	1.48 μs	1.38 μs	0.93	0.17	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetService*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.GetService.Scoped

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3798433794366278 < 1.393287155772749.
IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 13.370319071166632 (T) = (0 -1337.0523802625214) / Math.Sqrt((1757.0954014557117 / (299)) + (1003.3616451991597 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.06573128323562187 = (1431.1218563467378 - 1337.0523802625214) / 1431.1218563467378 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in System.Collections.IndexerSet<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedDictionary - Duration of single invocation	722.01 μs	655.86 μs	0.91	0.07	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSet<Int32>.SortedDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 655.8599350818452 < 686.0518505714338.
IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 16.506751076371536 (T) = (0 -662933.0329097221) / Math.Sqrt((289515572.0667214 / (299)) + (320899427.9621947 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08616311765564665 = (725439.1300218004 - 662933.0329097221) / 725439.1300218004 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in System.Collections.CreateAddAndRemove<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedDictionary - Duration of single invocation	3.39 ms	2.91 ms	0.86	0.12	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndRemove<Int32>.SortedDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.9081831333333334 < 3.229427903973837.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 39.083038453209895 (T) = (0 -2901220.5340080974) / Math.Sqrt((7591829941.74184 / (299)) + (3086086999.0448546 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.14322060281842536 = (3386193.159583237 - 2901220.5340080974) / 3386193.159583237 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedDictionary - Duration of single invocation	753.80 μs	667.07 μs	0.88	0.10	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyTrue<Int32, Int32>.SortedDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 667.0667805059525 < 704.4754651368507.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 20.032269139714064 (T) = (0 -682680.212052093) / Math.Sqrt((402771301.6733246 / (299)) + (277047227.23438483 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09531244804476495 = (754603.2998648718 - 682680.212052093) / 754603.2998648718 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedDictionary - Duration of single invocation	728.13 μs	666.76 μs	0.92	0.09	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueTrue<Int32, Int32>.SortedDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 666.7597041925466 < 697.1109842177353.
IsChangePoint: Marked as a change because one of 9/16/2022 12:08:08 AM, 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 22.89833053561754 (T) = (0 -673208.7959329006) / Math.Sqrt((443894800.057844 / (299)) + (200650594.10481983 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09643248869141603 = (745056.4429413046 - 673208.7959329006) / 745056.4429413046 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedDictionary - Duration of single invocation	1.61 ms	1.24 ms	0.77	0.14	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<Int32>.SortedDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.238971157967033 < 1.3696092422443178.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 19.701882851759503 (T) = (0 -1266425.0242110074) / Math.Sqrt((1355484807.7188342 / (299)) + (2387576874.068244 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.13936227202119825 = (1471496.0581442236 - 1266425.0242110074) / 1471496.0581442236 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
DataContractSerializer_ - Duration of single invocation	299.54 μs	276.38 μs	0.92	0.07	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>.DataContractSerializer_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 276.38133873749564 < 278.4288549774712.
IsChangePoint: Marked as a change because one of 9/23/2022 10:31:05 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 23.184954721231282 (T) = (0 -268600.2333241491) / Math.Sqrt((44330846.638141066 / (299)) + (19259543.337412514 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.07763122006342647 = (291206.9870172962 - 268600.2333241491) / 291206.9870172962 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2022-11-22T12:22:01Z

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
SerializeToString - Duration of single invocation	7.22 μs	6.46 μs	0.90	0.14	False
SerializeToUtf8Bytes - Duration of single invocation	7.03 μs	6.11 μs	0.87	0.11	False
SerializeToWriter - Duration of single invocation	6.58 μs	6.09 μs	0.93	0.14	False
SerializeObjectProperty - Duration of single invocation	8.41 μs	7.63 μs	0.91	0.12	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;LargeStructWithProperties&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.464836886588597 < 6.991932955780134.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 27.177912323254933 (T) = (0 -6544.2593880179475) / Math.Sqrt((35235.72621064329 / (299)) + (17264.482250077872 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10727258012620898 = (7330.635580727586 - 6544.2593880179475) / 7330.635580727586 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;LargeStructWithProperties&gt;.SerializeToUtf8Bytes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.1088443347091195 < 6.666087658473176.
IsChangePoint: Marked as a change because one of 9/21/2022 2:58:31 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 21.932010517295247 (T) = (0 -6263.332559216201) / Math.Sqrt((47000.03537543292 / (299)) + (24348.183603511854 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.10881153285956058 = (7028.0673394634305 - 6263.332559216201) / 7028.0673394634305 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;LargeStructWithProperties&gt;.SerializeToWriter

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.090551839534114 < 6.61108473922123.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 25.2374193465507 (T) = (0 -6005.596679876637) / Math.Sqrt((30008.695879358747 / (299)) + (19513.46597778724 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.11460591567566587 = (6782.9645422350595 - 6005.596679876637) / 6782.9645422350595 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;LargeStructWithProperties&gt;.SerializeObjectProperty

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.632646114317826 < 8.315096478380402.
IsChangePoint: Marked as a change because one of 10/26/2022 2:14:52 PM, 11/15/2022 10:02:59 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 29.35595023751346 (T) = (0 -7871.598627190161) / Math.Sqrt((60565.56834167616 / (299)) + (25767.7685014033 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1194413492521468 = (8939.32348572677 - 7871.598627190161) / 8939.32348572677 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in System.Collections.ContainsTrueComparer<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ImmutableSortedSet - Duration of single invocation	1.48 ms	1.10 ms	0.74	0.17	False
SortedSet - Duration of single invocation	1.18 ms	951.43 μs	0.81	0.24	False
HashSet - Duration of single invocation	240.79 μs	217.61 μs	0.90	0.18	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrueComparer&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrueComparer<Int32>.ImmutableSortedSet(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0997577727777776 < 1.2381430531965432.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 34.10672214829722 (T) = (0 -1105631.0122270624) / Math.Sqrt((1522194595.3658912 / (299)) + (717584272.2296628 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.15671155154192093 = (1311094.6963031057 - 1105631.0122270624) / 1311094.6963031057 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrueComparer&lt;Int32&gt;.SortedSet(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 951.4335364819003 < 1.1300524619861165.
IsChangePoint: Marked as a change because one of 9/22/2022 4:52:57 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 24.40709865072123 (T) = (0 -986820.5880132205) / Math.Sqrt((1690937974.8283827 / (299)) + (1356048064.0112052 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1658403438153552 = (1183011.646147969 - 986820.5880132205) / 1183011.646147969 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrueComparer&lt;Int32&gt;.HashSet(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 217.61409834474884 < 232.98959899478413.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 27.955738841870005 (T) = (0 -210278.80339763258) / Math.Sqrt((45920419.89554594 / (299)) + (15596017.243665768 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10647593074414353 = (235336.47344582077 - 210278.80339763258) / 235336.47344582077 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in Microsoft.Extensions.DependencyInjection.TimeToFirstService

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Scoped - Duration of single invocation	39.94 μs	35.94 μs	0.90	0.04	False
Scoped - Duration of single invocation	38.31 μs	36.23 μs	0.95	0.03	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.TimeToFirstService*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "ILEmit")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 35.93740501792115 < 36.80219616389607.
IsChangePoint: Marked as a change because one of 9/20/2022 8:04:40 PM, 10/16/2022 1:15:28 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 13.54857838254888 (T) = (0 -36434.062701303505) / Math.Sqrt((938795.5440653797 / (299)) + (746863.7614171382 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.06439701680640193 = (38941.79834371525 - 36434.062701303505) / 38941.79834371525 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Expressions")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.226857245958435 < 36.42801778416054.
IsChangePoint: Marked as a change because one of 9/18/2022 2:31:46 PM, 10/18/2022 1:35:00 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 21.062466221563245 (T) = (0 -36217.15870059921) / Math.Sqrt((1274015.4667347705 / (299)) + (279592.7231255136 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.07042338258023405 = (38960.91835993842 - 36217.15870059921) / 38960.91835993842 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in Microsoft.Extensions.Logging.FormattingOverhead

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
NoArguments - Duration of single invocation	564.40 ns	506.20 ns	0.90	0.11	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.FormattingOverhead*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.FormattingOverhead.NoArguments

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 506.20392272098286 < 527.9676314974757.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 18.570314443287312 (T) = (0 -497.8193784628699) / Math.Sqrt((157.53467848208132 / (299)) + (168.9294111120635 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.09474389317523138 = (549.9210385986751 - 497.8193784628699) / 549.9210385986751 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in System.Collections.CtorFromCollection<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ImmutableSortedDictionary - Duration of single invocation	1.98 ms	1.81 ms	0.91	0.12	False
SortedDictionary - Duration of single invocation	1.50 ms	1.29 ms	0.86	0.13	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<Int32>.ImmutableSortedDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.8094782603864736 < 1.8791750865988757.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 25.873098152221683 (T) = (0 -1759011.2974968916) / Math.Sqrt((2744808599.5472255 / (299)) + (1833954929.0957932 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11978249958728171 = (1998382.5550754473 - 1759011.2974968916) / 1998382.5550754473 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorFromCollection&lt;Int32&gt;.SortedDictionary(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2883333333333336 < 1.4197394223457793.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 19.06935217139162 (T) = (0 -1296861.4947355608) / Math.Sqrt((1589849171.869009 / (299)) + (2774401901.8875217 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1416438038434816 = (1510866.3519207386 - 1296861.4947355608) / 1510866.3519207386 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in System.Collections.IterateForEach<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedDictionary - Duration of single invocation	343.70 μs	317.50 μs	0.92	0.11	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<Int32>.SortedDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 317.5003568750001 < 324.8839559485816.
IsChangePoint: Marked as a change because one of 9/13/2022 6:36:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 15.787360086352045 (T) = (0 -310665.66646759777) / Math.Sqrt((75241336.77536298 / (299)) + (70559847.6334192 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.08635060754580874 = (340027.22382719035 - 310665.66646759777) / 340027.22382719035 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in System.Collections.CtorFromCollection<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedDictionary - Duration of single invocation	2.18 ms	1.91 ms	0.88	0.14	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<String>.SortedDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.91186409375 < 2.0408857437779018.
IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 20.401720882239402 (T) = (0 -1908299.6224346808) / Math.Sqrt((3003673050.60261 / (299)) + (2364426107.094974 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.10198758879363524 = (2125025.8889753255 - 1908299.6224346808) / 2125025.8889753255 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in Microsoft.Extensions.Logging.EventSourceLogger

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
NestedScopes_TwoMessages - Duration of single invocation	1.44 μs	1.26 μs	0.87	0.15	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.EventSourceLogger*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: False, Json: False)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2568545552483943 < 1.312665517542277.
IsChangePoint: Marked as a change because one of 9/23/2022 4:51:48 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 7.069362575051201 (T) = (0 -1283.3467024641643) / Math.Sqrt((1458.251575589203 / (299)) + (4964.2539896967455 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.07718123011247807 = (1390.681187185416 - 1283.3467024641643) / 1390.681187185416 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
XmlSerializer_ - Duration of single invocation	2.05 ms	1.93 ms	0.94	0.15	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;CollectionsOfPrimitives&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>.XmlSerializer_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.9349235552455357 < 1.9499640262673847.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 12.899795666251128 (T) = (0 -1907321.6052289752) / Math.Sqrt((6018252240.729262 / (299)) + (3618820346.016324 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.0812361316435989 = (2075964.9687150072 - 1907321.6052289752) / 2075964.9687150072 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	dd871a350bff581b6d313e88b8f29043a9772d18
Compare	715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff	Diff

Improvements in LinqBenchmarks

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Order00LinqMethodX - Duration of single invocation	11.15 secs	9.76 secs	0.88	0.07	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'

Payloads

Baseline
Compare

Histogram

LinqBenchmarks.Order00LinqMethodX

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.760769266666665 < 10.529240355126667.
IsChangePoint: Marked as a change because one of 10/2/2022 11:57:04 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 7.051904675478815 (T) = (0 -9864653030.586126) / Math.Sqrt((1.0636235986950143E+18 / (299)) + (47055400836517830 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.050792785185160845 = (10392517963.014444 - 9864653030.586126) / 10392517963.014444 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

lewing · 2022-11-28T18:42:40Z

this is dotnet/runtime#76477 cc @kg

performanceautofiler bot added AOT=true untriaged labels Nov 22, 2022

performanceautofiler bot added perf-improvement arch-x64 os-linux branch-refs/heads/main arch-wasm kind-micro wasm-aot and removed refs/heads/main labels Mar 23, 2023

[Perf] Linux/x64: 60 Improvements on 11/15/2022 10:02:59 PM #10015

[Perf] Linux/x64: 60 Improvements on 11/15/2022 10:02:59 PM #10015

Comments

performanceautofiler bot commented Nov 22, 2022

Run Information

Improvements in System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>

Repro

Payloads

Histogram

System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.CachedDefaultOptions

Description of detection logic

Docs

Improvements in System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>

Repro

Payloads

Histogram

System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromReader

Description of detection logic

Description of detection logic

Docs

Run Information

Improvements in System.Linq.Tests.Perf_Enumerable

Repro

Payloads

Histogram

System.Linq.Tests.Perf_Enumerable.Select(input: IEnumerable)

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

Run Information

Improvements in System.Collections.IterateForEach<String>

Repro

Payloads

Histogram

System.Collections.IterateForEach<String>.SortedDictionary(Size: 512)

Description of detection logic

Docs

performanceautofiler bot commented Nov 22, 2022 • edited Loading

Run Information

Improvements in System.Collections.Sort<Int32>

Repro

Payloads

Histogram

System.Collections.Sort<Int32>.LinqQuery(Size: 512)

Description of detection logic

Description of detection logic

Docs

Improvements in System.Net.Primitives.Tests.CredentialCacheTests

Repro

Payloads

Histogram

System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 0, hostPortCount: 0)

Description of detection logic

Docs

Run Information

Improvements in System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>

Repro

Payloads

Histogram

System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToString

performanceautofiler bot commented Nov 22, 2022 •

edited

Loading