[Perf] Linux/x64: 111 Regressions on 1/15/2023 6:48:05 PM #11780

performanceautofiler · 2023-01-17T14:01:40Z

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare	51a8dd5323721b363e61069575511f783e7ea6d3
Diff	Diff

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ToStringD - Duration of single invocation	81.71 ns	147.65 ns	1.81	0.06	False
Parse - Duration of single invocation	3.14 μs	3.39 μs	1.08	0.00	False
Parse - Duration of single invocation	343.13 ns	411.33 ns	1.20	0.03	False
Ctor_ByteArray - Duration of single invocation	216.82 ns	261.68 ns	1.21	0.02	False
Parse - Duration of single invocation	230.16 ns	315.94 ns	1.37	0.10	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.ToStringD(numberString: 123)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 147.65489454511007 > 85.92460287014578.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -28.387650195258207 (T) = (0 -150.14839315906485) / Math.Sqrt((2.3394406858622507 / (39)) + (51.05189738865337 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.8270373872011048 = (82.18134681364229 - 150.14839315906485) / 82.18134681364229 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Parse(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.3898662771149675 > 3.2802785083723074.
IsChangePoint: Marked as a change because one of 11/4/2022 9:06:14 PM, 12/8/2022 9:11:55 AM, 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -15.66541553239526 (T) = (0 -3341.693364697608) / Math.Sqrt((993.8820266789622 / (23)) + (3520.8059836395296 / (25))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (25) - 2, .025) and -0.06791629058128518 = (3129.1716346780954 - 3341.693364697608) / 3129.1716346780954 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Parse(numberString: -2147483648)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 411.3266638933929 > 344.87736347615953.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -74.84359467766949 (T) = (0 -408.6110230556528) / Math.Sqrt((21.977817395648028 / (30)) + (6.241204418589107 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.23255428332726266 = (331.51564079807764 - 408.6110230556528) / 331.51564079807764 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 261.6781889715202 > 226.95799630754746.
IsChangePoint: Marked as a change because one of 11/8/2022 11:25:18 AM, 12/10/2022 11:25:25 PM, 1/3/2023 6:29:42 AM, 1/9/2023 8:29:54 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -84.17648276375144 (T) = (0 -261.5481519406784) / Math.Sqrt((5.158597678690484 / (19)) + (0.38091367631906903 / (30))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (30) - 2, .025) and -0.20710923140151283 = (216.67314368643193 - 261.5481519406784) / 216.67314368643193 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Parse(numberString: 123)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 315.939324410614 > 239.25411439328974.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -45.278248726378806 (T) = (0 -303.8507534347267) / Math.Sqrt((16.977732711508445 / (29)) + (40.21669909921717 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.3244187333853978 = (229.42196887991918 - 303.8507534347267) / 229.42196887991918 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare	51a8dd5323721b363e61069575511f783e7ea6d3
Diff	Diff

Regressions in System.Tests.Perf_UInt64

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TryFormat - Duration of single invocation	33.59 ns	83.72 ns	2.49	0.02	False
TryFormat - Duration of single invocation	134.69 ns	188.29 ns	1.40	0.00	False
ToString - Duration of single invocation	51.07 ns	106.60 ns	2.09	0.02	False
TryFormat - Duration of single invocation	53.64 ns	104.55 ns	1.95	0.00	False
ToString - Duration of single invocation	135.25 ns	188.80 ns	1.40	0.01	False
Parse - Duration of single invocation	25.57 ns	27.26 ns	1.07	0.08	False
ParseSpan - Duration of single invocation	29.40 ns	32.32 ns	1.10	0.07	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.TryFormat(value: 0)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 83.72395951484465 > 34.53797012292523.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -140.65957473714312 (T) = (0 -84.13803652750784) / Math.Sqrt((0.24645836599884077 / (39)) + (1.131611795406109 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -1.5478303751536984 = (33.02340585465082 - 84.13803652750784) / 33.02340585465082 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.TryFormat(value: 18446744073709551615)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 188.28869632112563 > 141.45635110756615.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -110.29529836119723 (T) = (0 -185.930467860038) / Math.Sqrt((0.22814102787672424 / (39)) + (1.886815541910122 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.3800259316247819 = (134.72969137698132 - 185.930467860038) / 134.72969137698132 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.ToString(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 106.60347806499043 > 53.350562361620455.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -132.75645772061586 (T) = (0 -104.57743647723856) / Math.Sqrt((0.046718660799538646 / (39)) + (1.4755961681336653 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -1.0656657319721172 = (50.62650498510094 - 104.57743647723856) / 50.62650498510094 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.TryFormat(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 104.55346137414837 > 56.363027674032594.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -161.20111321842046 (T) = (0 -104.66586684506365) / Math.Sqrt((0.032235944736524094 / (39)) + (0.892029950941993 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.948916692026124 = (53.70463872226954 - 104.66586684506365) / 53.70463872226954 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.ToString(value: 18446744073709551615)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 188.79892475272013 > 141.78783688743667.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -33.66865977557549 (T) = (0 -191.36345095734112) / Math.Sqrt((0.1342143444001093 / (40)) + (25.329350511032743 / (9))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (9) - 2, .025) and -0.41911599549945305 = (134.84694102823596 - 191.36345095734112) / 134.84694102823596 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.Parse(value: "0")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.256119702766153 > 26.896952195124403.
IsChangePoint: Marked as a change because one of 10/25/2022 7:09:53 PM, 11/20/2022 12:28:28 PM, 11/24/2022 1:04:57 PM, 12/28/2022 5:13:50 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -17.569825211731445 (T) = (0 -27.294972115939732) / Math.Sqrt((0.21420604442597577 / (39)) + (0.01194830677098002 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.056143784889761864 = (25.84399255712017 - 27.294972115939732) / 25.84399255712017 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.ParseSpan(value: "0")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.32227631245608 > 31.19178695905178.
IsChangePoint: Marked as a change because one of 1/14/2023 12:48:25 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -9.968429398901648 (T) = (0 -31.88374168996897) / Math.Sqrt((0.38456631226187404 / (31)) + (0.2833987739308265 / (18))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (18) - 2, .025) and -0.05535989332886147 = (30.211250106728905 - 31.88374168996897) / 30.211250106728905 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare	51a8dd5323721b363e61069575511f783e7ea6d3
Diff	Diff

Regressions in System.Perf_Convert

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ChangeType - Duration of single invocation	57.22 ns	111.45 ns	1.95	0.02	False
ToDateTime_String - Duration of single invocation	340.02 ns	484.37 ns	1.42	0.04	False
ToDateTime_String - Duration of single invocation	939.66 ns	1.12 μs	1.19	0.09	False
ToDateTime_String - Duration of single invocation	538.31 ns	745.39 ns	1.38	0.18	False
ToDateTime_String - Duration of single invocation	2.43 μs	2.66 μs	1.09	0.02	False
FromBase64String - Duration of single invocation	149.72 ns	160.50 ns	1.07	0.03	False
ToDateTime_String - Duration of single invocation	787.29 ns	1.01 μs	1.28	0.11	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Perf_Convert*'

Payloads

Baseline
Compare

Histogram

System.Perf_Convert.ChangeType

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 111.44865681406002 > 60.15170390805974.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -150.01320322512694 (T) = (0 -112.41528558717387) / Math.Sqrt((0.6028047496923902 / (39)) + (1.0596218564162336 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.9493803976554376 = (57.66718785229306 - 112.41528558717387) / 57.66718785229306 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Perf_Convert.ToDateTime_String(value: "12/12/1999")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 484.37132721137095 > 356.6602321752282.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -29.516520096291554 (T) = (0 -491.8514497950046) / Math.Sqrt((7.364940092860565 / (29)) + (492.35727937589814 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.4429631402306377 = (340.8621024902888 - 491.8514497950046) / 340.8621024902888 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Perf_Convert.ToDateTime_String(value: "Thursday, February 26, 2009")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1195863302841946 > 965.3531614390172.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -20.064534158793137 (T) = (0 -1090.613675372394) / Math.Sqrt((609.8312461401326 / (30)) + (869.6955548773294 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.17588439269234793 = (927.4837578848079 - 1090.613675372394) / 927.4837578848079 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Perf_Convert.ToDateTime_String(value: "February 26, 2009")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 745.3896375370458 > 571.8753221262901.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -20.786367484664403 (T) = (0 -720.5592340091184) / Math.Sqrt((198.20139497110011 / (29)) + (1095.7630829406671 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.3015603104352585 = (553.6118674117791 - 720.5592340091184) / 553.6118674117791 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Perf_Convert.ToDateTime_String(value: "Fri, 27 Feb 2009 03:11:21 GMT")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.6578070010713377 > 2.5264336129838574.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -18.862204364595264 (T) = (0 -2607.128522983303) / Math.Sqrt((695.8883953509338 / (29)) + (1841.5486250576896 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.08643343478494168 = (2399.7130790616557 - 2607.128522983303) / 2399.7130790616557 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Perf_Convert.FromBase64String

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 160.50035003524985 > 156.05295703251528.
IsChangePoint: Marked as a change because one of 10/25/2022 7:09:53 PM, 12/6/2022 2:12:43 AM, 1/3/2023 6:29:42 AM, 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -24.950193272013408 (T) = (0 -161.29920136459512) / Math.Sqrt((6.203888216708392 / (30)) + (1.141454288981956 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.08684862042921883 = (148.40999779794058 - 161.29920136459512) / 148.40999779794058 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Perf_Convert.ToDateTime_String(value: "12/12/1999 11:59:59 PM")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.00788877189593 > 827.5976355602348.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -23.96904674285673 (T) = (0 -976.403542656839) / Math.Sqrt((105.92754599965141 / (29)) + (1079.1916257029548 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.2358885757935735 = (790.0417252662788 - 976.403542656839) / 790.0417252662788 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare	51a8dd5323721b363e61069575511f783e7ea6d3
Diff	Diff

Regressions in System.Xml.Tests.Perf_XmlConvert

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TimeSpan_ToString - Duration of single invocation	284.37 ns	485.50 ns	1.71	0.00	False
DateTime_ToString - Duration of single invocation	181.92 ns	252.63 ns	1.39	0.02	False
DateTime_ToString_Unspecified - Duration of single invocation	176.72 ns	244.43 ns	1.38	0.02	False
DateTime_ToString_RoundtripKind - Duration of single invocation	172.48 ns	240.52 ns	1.39	0.04	False
DateTime_ToString_Local - Duration of single invocation	762.05 ns	871.21 ns	1.14	0.31	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Tests.Perf_XmlConvert*'

Payloads

Baseline
Compare

Histogram

System.Xml.Tests.Perf_XmlConvert.TimeSpan_ToString

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 485.4991220688007 > 297.84097296389814.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -200.42861851608558 (T) = (0 -487.3358322746505) / Math.Sqrt((5.734782256402849 / (40)) + (7.938577923150585 / (9))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (9) - 2, .025) and -0.7137120986805264 = (284.374389753025 - 487.3358322746505) / 284.374389753025 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 252.62870721587785 > 189.47261588983218.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -141.05081732442426 (T) = (0 -252.27988123635077) / Math.Sqrt((2.7371070189527766 / (29)) + (3.000486150555129 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.3905055395152061 = (181.43033167944594 - 252.27988123635077) / 181.43033167944594 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_Unspecified

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 244.42988464845365 > 183.79087081921818.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -95.9307398731106 (T) = (0 -243.94438135600558) / Math.Sqrt((0.7824593953314449 / (29)) + (9.132859404608645 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.38925553871985924 = (175.59359999441867 - 243.94438135600558) / 175.59359999441867 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_RoundtripKind

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 240.52272731966212 > 180.65456883542745.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -49.88798574714676 (T) = (0 -243.8207835841296) / Math.Sqrt((2.9861668681837097 / (30)) + (36.46260472525284 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.40985114796313127 = (172.94079870515927 - 243.8207835841296) / 172.94079870515927 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_Local

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 871.2076015681679 > 793.6973051385118.
IsChangePoint: Marked as a change because one of 12/6/2022 11:13:59 AM, 12/6/2022 11:39:16 PM, 1/14/2023 12:48:25 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -3.161910824280395 (T) = (0 -814.2709490004061) / Math.Sqrt((10058.892833465266 / (30)) + (6141.050849243692 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -0.11234194876273867 = (732.032941764106 - 814.2709490004061) / 732.032941764106 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

performanceautofiler · 2023-01-17T14:01:41Z

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare	51a8dd5323721b363e61069575511f783e7ea6d3
Diff	Diff

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
FormatterInt32 - Duration of single invocation	52.62 ns	109.72 ns	2.09	0.01	False
FormatterUInt64 - Duration of single invocation	51.57 ns	109.12 ns	2.12	0.01	False
FormatterUInt64 - Duration of single invocation	163.62 ns	223.07 ns	1.36	0.00	False
FormatterUInt32 - Duration of single invocation	50.27 ns	109.42 ns	2.18	0.00	False
FormatterDouble - Duration of single invocation	498.35 ns	807.53 ns	1.62	0.01	False
FormatterUInt32 - Duration of single invocation	87.68 ns	145.36 ns	1.66	0.00	False
FormatterDouble - Duration of single invocation	493.32 ns	797.46 ns	1.62	0.01	False
FormatterDouble - Duration of single invocation	304.50 ns	627.61 ns	2.06	0.01	False
FormatterInt64 - Duration of single invocation	51.78 ns	110.81 ns	2.14	0.00	False
FormatterInt64 - Duration of single invocation	157.43 ns	215.79 ns	1.37	0.00	False
FormatterInt32 - Duration of single invocation	90.54 ns	147.14 ns	1.63	0.01	False
FormatterInt64 - Duration of single invocation	157.47 ns	215.41 ns	1.37	0.01	False
FormatterInt32 - Duration of single invocation	88.92 ns	151.17 ns	1.70	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 12345)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 109.71941728691117 > 54.42982900542209.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -36.392757259891944 (T) = (0 -111.96217404147657) / Math.Sqrt((0.039695480497061265 / (40)) + (24.64840379484608 / (9))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (9) - 2, .025) and -1.1645786495698969 = (51.72469665804175 - 111.96217404147657) / 51.72469665804175 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 109.11564556095219 > 53.39608003252147.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -233.25074646351936 (T) = (0 -110.11809777304838) / Math.Sqrt((0.031679102773248535 / (39)) + (0.5728444864371841 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -1.1635326725375508 = (50.897358367088465 - 110.11809777304838) / 50.897358367088465 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 18446744073709551615)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 223.07295958463317 > 171.76436233411013.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -243.19037934177652 (T) = (0 -222.06767500893818) / Math.Sqrt((0.4175202511900338 / (39)) + (0.41798766742870275 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.3546405327385452 = (163.931072223275 - 222.06767500893818) / 163.931072223275 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 109.42300703333198 > 53.45304738386099.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -449.56839737492595 (T) = (0 -108.6744715727103) / Math.Sqrt((0.03739958282409874 / (39)) + (0.14002228065931083 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -1.135199623528906 = (50.89663297762336 - 108.6744715727103) / 50.89663297762336 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: -1.7976931348623157E+308)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 807.5289644521714 > 521.3305147528226.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -100.60792630291313 (T) = (0 -805.048664314627) / Math.Sqrt((20.985659397843484 / (29)) + (159.75311699994575 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.6068049056752801 = (501.02452480146934 - 805.048664314627) / 501.02452480146934 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value: 4294967295)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 145.35709704081123 > 92.0981159352102.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -447.2667144672195 (T) = (0 -145.72541209303296) / Math.Sqrt((0.1399787944110303 / (39)) + (0.11768016036577678 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.656218882609853 = (87.98680755492917 - 145.72541209303296) / 87.98680755492917 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: 1.7976931348623157E+308)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 797.4550312059482 > 519.7061667492266.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -124.03507059108874 (T) = (0 -804.117274868269) / Math.Sqrt((13.825772896277638 / (29)) + (106.18977148194486 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.6126335207636067 = (498.6360909126502 - 804.117274868269) / 498.6360909126502 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 627.6142689774978 > 320.72561665217063.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -124.23915221077814 (T) = (0 -607.4026769415992) / Math.Sqrt((5.846392200910916 / (30)) + (107.70548172256686 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.9813536801435789 = (306.55944116831466 - 607.4026769415992) / 306.55944116831466 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 110.81004291748594 > 54.38602093632607.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -389.16980246037855 (T) = (0 -110.26024420079864) / Math.Sqrt((0.03450194529521647 / (40)) + (0.19533589091966025 / (9))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (9) - 2, .025) and -1.1286367880360304 = (51.798524210666 - 110.26024420079864) / 51.798524210666 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: -9223372036854775808)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 215.79109865855125 > 165.16680938686656.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -268.99510682252395 (T) = (0 -215.38113367756588) / Math.Sqrt((0.09404920476143175 / (39)) + (0.39664939599108845 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.36849332391200035 = (157.38559327558383 - 215.38113367756588) / 157.38559327558383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 2147483647)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 147.13748852660004 > 93.47670861701359.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -341.9124692058217 (T) = (0 -146.96046045960315) / Math.Sqrt((0.16548526143121162 / (39)) + (0.21884543046537813 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.6479285835558797 = (89.1789012740429 - 146.96046045960315) / 89.1789012740429 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 9223372036854775807)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 215.40785293343393 > 165.31467213089334.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -442.8437761273376 (T) = (0 -215.69298888213882) / Math.Sqrt((0.24026067757304137 / (39)) + (0.10019793402608279 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.3698558786719983 = (157.45670200812773 - 215.69298888213882) / 157.45670200812773 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: -2147483648)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 151.16794545747706 > 93.37375017872168.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -110.48560169343327 (T) = (0 -149.5777626115935) / Math.Sqrt((0.17836900390948693 / (40)) + (2.6963060272509978 / (9))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (9) - 2, .025) and -0.6871831904301842 = (88.65531820137171 - 149.5777626115935) / 88.65531820137171 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare	51a8dd5323721b363e61069575511f783e7ea6d3
Diff	Diff

Regressions in System.Text.Json.Tests.Perf_Basic

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
WriteBasicUtf8 - Duration of single invocation	7.71 ms	13.06 ms	1.69	0.00	False
WriteBasicUtf8 - Duration of single invocation	2.97 μs	3.60 μs	1.21	0.01	False
WriteBasicUtf8 - Duration of single invocation	2.61 μs	3.27 μs	1.25	0.00	False
WriteBasicUtf8 - Duration of single invocation	6.26 ms	11.32 ms	1.81	0.00	False
WriteBasicUtf16 - Duration of single invocation	7.70 ms	12.86 ms	1.67	0.02	False
WriteBasicUtf16 - Duration of single invocation	2.94 μs	3.63 μs	1.24	0.01	False
WriteBasicUtf8 - Duration of single invocation	7.59 ms	12.75 ms	1.68	0.01	False
WriteBasicUtf16 - Duration of single invocation	6.36 ms	11.62 ms	1.83	0.00	False
WriteBasicUtf8 - Duration of single invocation	2.56 μs	3.51 μs	1.37	0.01	False
WriteBasicUtf16 - Duration of single invocation	6.24 ms	11.35 ms	1.82	0.01	False
WriteBasicUtf16 - Duration of single invocation	2.91 μs	3.58 μs	1.23	0.01	False
WriteBasicUtf16 - Duration of single invocation	2.56 μs	3.24 μs	1.27	0.03	False
WriteBasicUtf8 - Duration of single invocation	6.36 ms	11.59 ms	1.82	0.01	False
WriteBasicUtf16 - Duration of single invocation	2.50 μs	3.21 μs	1.28	0.01	False
WriteBasicUtf16 - Duration of single invocation	7.56 ms	12.73 ms	1.68	0.02	False
WriteBasicUtf8 - Duration of single invocation	2.99 μs	3.64 μs	1.22	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Basic*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: False, DataSize: 100000)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.061180491071429 > 8.092457930937499.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -200.66636436305376 (T) = (0 -12982011.543541411) / Math.Sqrt((2533014947.045617 / (39)) + (5645904285.57615 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.6854792738527737 = (7702267.090989686 - 12982011.543541411) / 7702267.090989686 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: True, DataSize: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.5967988597763783 > 3.1013025339167566.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 12/21/2022 9:23:17 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -57.879975591887 (T) = (0 -3622.7192730883685) / Math.Sqrt((258.3594721436806 / (39)) + (1159.6534545284596 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.22844323889983076 = (2949.032693063461 - 3622.7192730883685) / 2949.032693063461 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: False, DataSize: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.2728380992894537 > 2.7419983051356507.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -38.3532541865435 (T) = (0 -3303.5338029809623) / Math.Sqrt((127.260048880849 / (39)) + (2914.453043670048 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.26577521444734703 = (2609.8897855440514 - 3303.5338029809623) / 2609.8897855440514 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: True, DataSize: 100000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.315466738839286 > 6.558469383229166.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -231.20215551122683 (T) = (0 -11369871.260244964) / Math.Sqrt((646499972.0461892 / (39)) + (4289842860.8616076 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.8235053022537588 = (6235173.128475353 - 11369871.260244964) / 6235173.128475353 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: False, DataSize: 100000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.864913991586539 > 8.101668126742789.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -205.83959786804883 (T) = (0 -12918780.656655729) / Math.Sqrt((2642858719.036015 / (39)) + (5189462914.024044 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.6791492674710026 = (7693646.364216887 - 12918780.656655729) / 7693646.364216887 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: False, DataSize: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.6343047025750224 > 3.0900626814474066.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -73.13389055555403 (T) = (0 -3640.322289645335) / Math.Sqrt((627.0661149841708 / (40)) + (618.2462592715501 / (9))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (9) - 2, .025) and -0.226291562894797 = (2968.561800304612 - 3640.322289645335) / 2968.561800304612 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: True, DataSize: 100000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.747994694196429 > 7.973078369711539.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -180.68262054810836 (T) = (0 -12823574.180489417) / Math.Sqrt((1782651170.3361697 / (39)) + (7191478364.20486 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.6935358404513049 = (7572071.3280399805 - 12823574.180489417) / 7572071.3280399805 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: False, DataSize: 100000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.623509020833334 > 6.6781294459375005.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -144.73459318418472 (T) = (0 -11559944.05672568) / Math.Sqrt((308591300.01542425 / (40)) + (11591532080.183668 / (9))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (9) - 2, .025) and -0.8204130023400115 = (6350176.603807045 - 11559944.05672568) / 6350176.603807045 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: True, DataSize: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.5057931786117384 > 2.691546066472877.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -22.665380539507574 (T) = (0 -3260.4564994694883) / Math.Sqrt((91.19837997816093 / (40)) + (8703.534136745186 / (9))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (9) - 2, .025) and -0.276214271293359 = (2554.787681668244 - 3260.4564994694883) / 2554.787681668244 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: True, DataSize: 100000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.34581332451923 > 6.561902010921473.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -233.51073139312206 (T) = (0 -11398846.768520428) / Math.Sqrt((663550734.0981238 / (39)) + (4242545078.052336 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.8272448353907021 = (6238270.070733637 - 11398846.768520428) / 6238270.070733637 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: True, DataSize: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.5838359879646555 > 3.053364702934296.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -147.06723802828128 (T) = (0 -3590.8010567916594) / Math.Sqrt((356.2655162548107 / (40)) + (104.86944758607004 / (9))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (9) - 2, .025) and -0.22805608266233182 = (2923.971557558737 - 3590.8010567916594) / 2923.971557558737 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: False, DataSize: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.2423958516084777 > 2.686630522778277.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 12/21/2022 9:23:17 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -36.76826070265822 (T) = (0 -3263.5334718524737) / Math.Sqrt((471.87242619450694 / (39)) + (3137.32316006606 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.27221578126412294 = (2565.235803481152 - 3263.5334718524737) / 2565.235803481152 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: False, DataSize: 100000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.588068265625 > 6.677096469831731.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -260.285313414402 (T) = (0 -11537596.18722146) / Math.Sqrt((322631729.31609136 / (39)) + (3492925891.043638 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.815363688733465 = (6355528.789534708 - 11537596.18722146) / 6355528.789534708 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: True, DataSize: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.2099137506228383 > 2.641370772265622.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 12/21/2022 9:23:17 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -71.47457999258084 (T) = (0 -3190.2451677691365) / Math.Sqrt((477.33712699653034 / (39)) + (702.9207597774719 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.27056262686072086 = (2510.8917107467 - 3190.2451677691365) / 2510.8917107467 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: True, DataSize: 100000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.730139908854166 > 7.976057794322917.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -56.84440646842707 (T) = (0 -12938880.038394764) / Math.Sqrt((30656301739.18669 / (39)) + (72260165749.42647 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.7020655425919261 = (7601869.44310692 - 12938880.038394764) / 7601869.44310692 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: False, DataSize: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.6421706237102254 > 3.1375063454709937.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 12/21/2022 9:23:17 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -90.85811692225904 (T) = (0 -3663.8712290763006) / Math.Sqrt((228.84510877127218 / (39)) + (439.58572157941273 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.2246279278273674 = (2991.824002884235 - 3663.8712290763006) / 2991.824002884235 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-01-17T14:01:41Z

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare	51a8dd5323721b363e61069575511f783e7ea6d3
Diff	Diff

Regressions in System.Tests.Perf_UInt16

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ToString - Duration of single invocation	43.13 ns	93.52 ns	2.17	0.01	False
ToString - Duration of single invocation	43.06 ns	95.22 ns	2.21	0.01	False
Parse - Duration of single invocation	36.06 ns	38.44 ns	1.07	0.04	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt16.ToString(value: 65535)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 93.52238555460103 > 45.26633865513693.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -55.12922399448615 (T) = (0 -93.83350015329374) / Math.Sqrt((0.008063339187956247 / (39)) + (7.6075935747026175 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -1.1750050642508048 = (43.141738700096006 - 93.83350015329374) / 43.141738700096006 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt16.ToString(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 95.21710004630434 > 45.26457867153237.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -87.91609685872565 (T) = (0 -94.43398878627794) / Math.Sqrt((0.03345900924742786 / (39)) + (3.0558299763704833 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -1.188971800116352 = (43.14079732834311 - 94.43398878627794) / 43.14079732834311 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt16.Parse(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.437705013524386 > 37.78265402776617.
IsChangePoint: Marked as a change because one of 11/22/2022 2:57:03 PM, 12/5/2022 3:14:07 PM, 12/10/2022 11:25:25 PM, 12/14/2022 12:26:48 AM, 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -8.265577113899537 (T) = (0 -37.84065419580893) / Math.Sqrt((1.1785164063824833 / (29)) + (0.2668685481621243 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.053828915146127074 = (35.90777748830495 - 37.84065419580893) / 35.90777748830495 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare	51a8dd5323721b363e61069575511f783e7ea6d3
Diff	Diff

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Append_Primitives - Duration of single invocation	2.60 μs	3.90 μs	1.50	0.01	False
Insert_Primitives - Duration of single invocation	44.00 μs	55.45 μs	1.26	0.01	False
Append_Char - Duration of single invocation	395.00 μs	438.61 μs	1.11	0.07	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_StringBuilder.Append_Primitives

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.896020987157251 > 2.7471366960779617.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -21.963104133998268 (T) = (0 -3895.046043942166) / Math.Sqrt((98018.98363748692 / (40)) + (456.8341882680346 / (9))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (9) - 2, .025) and -0.3927692148748785 = (2796.6198580086243 - 3895.046043942166) / 2796.6198580086243 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_StringBuilder.Insert_Primitives

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 55.45333727836879 > 46.149554892941076.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -18.97724583618843 (T) = (0 -53909.65039054993) / Math.Sqrt((528893.0985251203 / (30)) + (4779371.036386382 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.22344395120001614 = (44063.849706946196 - 53909.65039054993) / 44063.849706946196 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_StringBuilder.Append_Char(length: 100000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 438.6106958085318 > 415.1382776090746.
IsChangePoint: Marked as a change because one of 11/11/2022 5:53:56 PM, 11/25/2022 11:21:20 AM, 12/6/2022 2:12:43 AM, 12/14/2022 12:26:48 AM, 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -9.66333475182101 (T) = (0 -422693.33031731) / Math.Sqrt((23957554.122708473 / (23)) + (214318686.4750931 / (25))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (25) - 2, .025) and -0.0762945314295458 = (392730.16630111856 - 422693.33031731) / 392730.16630111856 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare	51a8dd5323721b363e61069575511f783e7ea6d3
Diff	Diff

Regressions in System.Tests.Perf_SByte

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	45.51 ns	99.27 ns	2.18	0.17	False
ToString - Duration of single invocation	35.56 ns	87.98 ns	2.47	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_SByte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_SByte.ToString(value: -128)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 99.26940879288067 > 48.088906374939974.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -158.81183970198964 (T) = (0 -99.90228401314756) / Math.Sqrt((1.5123057139264744 / (39)) + (0.6650185339414671 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -1.1440369520319607 = (46.5954114822823 - 99.90228401314756) / 46.5954114822823 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_SByte.ToString(value: 127)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 87.9775235282176 > 37.22617322416467.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -198.71901967408726 (T) = (0 -88.94629483876895) / Math.Sqrt((0.21067251612327986 / (39)) + (0.5959207204313651 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -1.4868230357367764 = (35.76703832985713 - 88.94629483876895) / 35.76703832985713 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-01-17T14:01:41Z

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare	51a8dd5323721b363e61069575511f783e7ea6d3
Diff	Diff

Regressions in System.Tests.Perf_Version

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ToString3 - Duration of single invocation	146.64 ns	292.01 ns	1.99	0.01	False
TryFormat2 - Duration of single invocation	76.06 ns	178.11 ns	2.34	0.03	False
ToString2 - Duration of single invocation	113.17 ns	202.29 ns	1.79	0.01	False
TryFormatL - Duration of single invocation	308.48 ns	494.73 ns	1.60	0.00	False
ToStringL - Duration of single invocation	381.29 ns	571.16 ns	1.50	0.00	False
TryFormat3 - Duration of single invocation	113.80 ns	259.18 ns	2.28	0.01	False
TryFormat4 - Duration of single invocation	147.95 ns	341.22 ns	2.31	0.01	False
ToString4 - Duration of single invocation	181.71 ns	370.40 ns	2.04	0.00	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Version*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Version.ToString3

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 292.0053214240883 > 153.96889596591564.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -78.38841784143354 (T) = (0 -291.1987646696364) / Math.Sqrt((7.920802976271504 / (39)) + (29.495872404625615 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -1.0088448745098075 = (144.95831328971772 - 291.1987646696364) / 144.95831328971772 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.TryFormat2

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 178.109260102156 > 79.8436745807725.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -120.31110579334712 (T) = (0 -174.34052918263964) / Math.Sqrt((0.0586730631671134 / (39)) + (5.9876415593349375 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -1.2910226893305758 = (76.09725123830222 - 174.34052918263964) / 76.09725123830222 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.ToString2

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 202.29497164941645 > 116.69329800612576.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -158.8119811825331 (T) = (0 -203.6124235866074) / Math.Sqrt((4.617759530977352 / (39)) + (2.1071764091277703 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.8625635012061359 = (109.31837945646126 - 203.6124235866074) / 109.31837945646126 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.TryFormatL

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 494.7341692689553 > 323.77482470487433.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -235.82942630742454 (T) = (0 -494.27034366119256) / Math.Sqrt((0.8004790811743898 / (39)) + (5.412167491158003 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.6032268919033829 = (308.2971887244138 - 494.27034366119256) / 308.2971887244138 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.ToStringL

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 571.1594305157032 > 397.1593984812333.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -122.02634075482878 (T) = (0 -568.0885076240696) / Math.Sqrt((1.7963967509863743 / (40)) + (21.594005225728583 / (9))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (9) - 2, .025) and -0.5056217166547757 = (377.31157922340645 - 568.0885076240696) / 377.31157922340645 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.TryFormat3

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 259.1836923106856 > 117.65711560344079.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -98.8032151471796 (T) = (0 -261.06517985435715) / Math.Sqrt((0.1956626346225749 / (39)) + (20.454825651990717 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -1.3320135645110527 = (111.94839679635142 - 261.06517985435715) / 111.94839679635142 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.TryFormat4

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 341.22101367712844 > 155.42230450631064.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -361.5111766899112 (T) = (0 -340.4874734313863) / Math.Sqrt((0.4773486948419272 / (39)) + (2.4421848557629495 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -1.3010559170587637 = (147.97009968649581 - 340.4874734313863) / 147.97009968649581 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.ToString4

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 370.40403571285975 > 191.1024886572141.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -187.1075488964661 (T) = (0 -370.68595382651984) / Math.Sqrt((4.050268754425069 / (40)) + (8.32085531505341 / (9))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (9) - 2, .025) and -1.045949083507155 = (181.18043934460576 - 370.68595382651984) / 181.18043934460576 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare	51a8dd5323721b363e61069575511f783e7ea6d3
Diff	Diff

Regressions in System.Tests.Perf_Int64

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TryFormat - Duration of single invocation	143.45 ns	197.99 ns	1.38	0.00	False
ToString - Duration of single invocation	130.62 ns	182.68 ns	1.40	0.00	False
ToString - Duration of single invocation	52.00 ns	116.55 ns	2.24	0.01	False
TryParse - Duration of single invocation	29.76 ns	33.55 ns	1.13	0.07	False
TryFormat - Duration of single invocation	53.94 ns	104.32 ns	1.93	0.00	False
TryFormat - Duration of single invocation	129.09 ns	180.30 ns	1.40	0.00	False
TryParseSpan - Duration of single invocation	31.82 ns	35.21 ns	1.11	0.08	False
ToString - Duration of single invocation	147.09 ns	199.59 ns	1.36	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int64.TryFormat(value: -9223372036854775808)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 197.9863892020704 > 151.8964696271409.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -100.88191451123181 (T) = (0 -196.27627024906673) / Math.Sqrt((1.0841343383292446 / (39)) + (2.1358668152442024 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.35988795616693975 = (144.33267782023938 - 196.27627024906673) / 144.33267782023938 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.ToString(value: 9223372036854775807)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 182.67660672731847 > 137.0241975761147.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -100.0573754509197 (T) = (0 -183.4373525110127) / Math.Sqrt((0.23293975090041288 / (39)) + (2.485675539836043 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.4079348840362841 = (130.28823604763053 - 183.4373525110127) / 130.28823604763053 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.ToString(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 116.54645590295756 > 54.71404245706407.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -44.856554005738 (T) = (0 -107.07881506345782) / Math.Sqrt((0.11215588972554959 / (39)) + (13.591351845907418 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -1.0630581163729802 = (51.902956205475625 - 107.07881506345782) / 51.902956205475625 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryParse(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.55076725527372 > 31.52671161102531.
IsChangePoint: Marked as a change because one of 11/11/2022 8:39:50 AM, 11/18/2022 6:07:58 AM, 12/21/2022 1:04:44 PM, 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -9.578750969153964 (T) = (0 -33.386556949866666) / Math.Sqrt((0.6304868755538665 / (29)) + (0.7844195748984857 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.07761801682148489 = (30.98181027850928 - 33.386556949866666) / 30.98181027850928 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryFormat(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 104.32463210936774 > 56.619970792063285.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -127.3928841881721 (T) = (0 -105.95816339300542) / Math.Sqrt((0.015720084689127525 / (39)) + (1.5002621866987151 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.9664545184780661 = (53.88284468181423 - 105.95816339300542) / 53.88284468181423 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryFormat(value: 9223372036854775807)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 180.29853840278824 > 135.47962122137844.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -112.1360887409201 (T) = (0 -180.53455318859085) / Math.Sqrt((0.07733556382762427 / (40)) + (1.8853090993944766 / (9))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (9) - 2, .025) and -0.39976555465108876 = (128.974850530303 - 180.53455318859085) / 128.974850530303 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryParseSpan(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.20662298527106 > 33.410466469976086.
IsChangePoint: Marked as a change because one of 11/11/2022 8:39:50 AM, 11/18/2022 6:07:58 AM, 12/15/2022 2:06:03 PM, 12/21/2022 3:18:57 AM, 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -14.036392450992802 (T) = (0 -34.945293138958874) / Math.Sqrt((0.3845552334649192 / (29)) + (0.318334520874065 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.07479293971512205 = (32.51351199629322 - 34.945293138958874) / 32.51351199629322 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.ToString(value: -9223372036854775808)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 199.5927574731434 > 154.145821775689.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -20.96519718924038 (T) = (0 -205.18239981757083) / Math.Sqrt((2.079215017799054 / (39)) + (68.14419836243604 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.39304314749918556 = (147.29077142076878 - 205.18239981757083) / 147.29077142076878 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare	51a8dd5323721b363e61069575511f783e7ea6d3
Diff	Diff

Regressions in System.Tests.Perf_Uri

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
BuilderToString - Duration of single invocation	470.87 ns	602.65 ns	1.28	0.08	False
PathAndQuery - Duration of single invocation	3.96 ns	7.13 ns	1.80	0.15	False
ParseAbsoluteUri - Duration of single invocation	727.89 ns	943.44 ns	1.30	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Uri*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Uri.BuilderToString

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 602.646478780456 > 492.2334729127517.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -17.29311186872123 (T) = (0 -588.9083891654714) / Math.Sqrt((76.00763116476512 / (40)) + (367.9385664375885 / (9))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (9) - 2, .025) and -0.237729435274091 = (475.79735310654513 - 588.9083891654714) / 475.79735310654513 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Uri.PathAndQuery

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.129040150217078 > 4.144802674703842.
IsChangePoint: Marked as a change because one of 1/9/2023 10:41:51 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -45.78709664963521 (T) = (0 -7.070681624873844) / Math.Sqrt((0.07513389470322247 / (19)) + (0.014842827486885006 / (29))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (29) - 2, .025) and -0.7629395660320734 = (4.010733981532981 - 7.070681624873844) / 4.010733981532981 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Uri.ParseAbsoluteUri

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 943.4376550324816 > 763.7829049038537.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -35.086514692256245 (T) = (0 -934.8840704642118) / Math.Sqrt((38.7134308128162 / (39)) + (310.7000094737323 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.2880948212052106 = (725.788237848425 - 934.8840704642118) / 725.788237848425 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare	51a8dd5323721b363e61069575511f783e7ea6d3
Diff	Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
FirstWithPredicate_LastElementMatches - Duration of single invocation	4.63 μs	5.17 μs	1.12	0.01	False
AnyWithPredicate_LastElementMatches - Duration of single invocation	4.74 μs	5.07 μs	1.07	0.02	False
SelectToArray - Duration of single invocation	292.32 ns	315.94 ns	1.08	0.31	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IOrderedEnumerable)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.173380851693911 > 4.866554273631492.
IsChangePoint: Marked as a change because one of 11/4/2022 2:24:58 AM, 11/8/2022 11:25:18 AM, 12/12/2022 5:25:39 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -7.023604709108864 (T) = (0 -5036.468928584996) / Math.Sqrt((7379.467244086969 / (39)) + (15850.922303671789 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.0656305150563021 = (4726.280692439534 - 5036.468928584996) / 4726.280692439534 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IOrderedEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.074357734814094 > 4.944982563062363.
IsChangePoint: Marked as a change because one of 10/26/2022 7:11:30 AM, 12/10/2022 11:25:25 PM, 12/30/2022 6:06:02 AM, 1/2/2023 2:49:36 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -18.081990888140297 (T) = (0 -5129.1615298569295) / Math.Sqrt((4982.860194347438 / (39)) + (3092.8201533337406 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.08288639183049486 = (4736.564766675729 - 5129.1615298569295) / 4736.564766675729 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Array)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 315.9406458772855 > 306.86183443081944.
IsChangePoint: Marked as a change because one of 12/6/2022 2:12:43 AM, 12/20/2022 9:29:18 AM, 1/10/2023 1:28:02 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -14.908789034219835 (T) = (0 -318.86625589911625) / Math.Sqrt((6.994020137440737 / (22)) + (67.48661772042186 / (26))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (26) - 2, .025) and -0.08672941212778182 = (293.4182624861383 - 318.86625589911625) / 293.4182624861383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare	51a8dd5323721b363e61069575511f783e7ea6d3
Diff	Diff

Regressions in System.Tests.Perf_Int16

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	44.23 ns	98.36 ns	2.22	0.00	False
ToString - Duration of single invocation	53.97 ns	106.79 ns	1.98	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int16.ToString(value: 32767)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 98.36074991231403 > 46.45215194738627.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -126.59460256493378 (T) = (0 -95.44114491429787) / Math.Sqrt((0.017916089952324337 / (40)) + (1.4690572374342914 / (9))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (9) - 2, .025) and -1.1580896192425052 = (44.22482924865648 - 95.44114491429787) / 44.22482924865648 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int16.ToString(value: -32768)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 106.79379919723974 > 57.82310556924916.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -89.52062816951724 (T) = (0 -107.4236847386128) / Math.Sqrt((0.8740814798573825 / (39)) + (2.9341616641663375 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.9681350095635322 = (54.5814612395091 - 107.4236847386128) / 54.5814612395091 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare	51a8dd5323721b363e61069575511f783e7ea6d3
Diff	Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IntegerFormatting - Duration of single invocation	6.96 ms	11.83 ms	1.70	0.00	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.LowLevelPerf.IntegerFormatting

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.829006955208333 > 7.271670164192709.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -171.421892410502 (T) = (0 -11843992.449997457) / Math.Sqrt((357247652.62663573 / (39)) + (7310777066.466294 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.708887064316908 = (6930822.227700487 - 11843992.449997457) / 6930822.227700487 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-01-17T14:01:41Z

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare	51a8dd5323721b363e61069575511f783e7ea6d3
Diff	Diff

Regressions in System.Tests.Perf_Int32

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TryFormat - Duration of single invocation	76.15 ns	122.33 ns	1.61	0.03	False
ToString - Duration of single invocation	44.12 ns	95.92 ns	2.17	0.02	False
TryFormat - Duration of single invocation	28.86 ns	77.50 ns	2.69	0.02	False
ToString - Duration of single invocation	69.68 ns	119.11 ns	1.71	0.01	False
TryFormat - Duration of single invocation	46.04 ns	91.56 ns	1.99	0.03	False
TryParseSpan - Duration of single invocation	24.27 ns	26.88 ns	1.11	0.08	False
ToString - Duration of single invocation	75.31 ns	126.59 ns	1.68	0.03	False
TryFormat - Duration of single invocation	69.56 ns	115.09 ns	1.65	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int32.TryFormat(value: -2147483648)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 122.32885399900096 > 79.82047281031858.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -136.9790191874176 (T) = (0 -122.97483134731075) / Math.Sqrt((1.5067038457883541 / (39)) + (0.7069759408136425 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.6163139211531616 = (76.08350688433976 - 122.97483134731075) / 76.08350688433976 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToString(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 95.91547807774568 > 46.40092130014793.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -47.66216842922447 (T) = (0 -97.03412214351138) / Math.Sqrt((0.030101058135813437 / (39)) + (11.061282595419646 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -1.1964130677098486 = (44.178448749026394 - 97.03412214351138) / 44.178448749026394 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.TryFormat(value: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 77.50364607322983 > 30.298298498824582.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -157.90417827006624 (T) = (0 -78.40337758378564) / Math.Sqrt((0.03579573577282941 / (40)) + (0.8735694429787783 / (9))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (9) - 2, .025) and -1.7052342510377831 = (28.982102956041054 - 78.40337758378564) / 28.982102956041054 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToString(value: 2147483647)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 119.11165968503205 > 72.65626921602292.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -105.8674283663133 (T) = (0 -120.04639779272185) / Math.Sqrt((0.03919892713755859 / (39)) + (2.075330117647851 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.7373312541056446 = (69.09816277640165 - 120.04639779272185) / 69.09816277640165 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.TryFormat(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 91.55907071187205 > 48.30075259324551.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -151.00863442549658 (T) = (0 -91.82176988224) / Math.Sqrt((0.2071534972497684 / (39)) + (0.7873073658131604 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -1.00386676064662 = (45.822293021423434 - 91.82176988224) / 45.822293021423434 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.TryParseSpan(value: "4")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.88391565400078 > 25.507063600553145.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 12/17/2022 10:44:50 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -13.033963063579126 (T) = (0 -26.820594667928944) / Math.Sqrt((0.380268067286025 / (40)) + (0.0411851935488755 / (9))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (9) - 2, .025) and -0.0611998410263449 = (25.273839696384865 - 26.820594667928944) / 25.273839696384865 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToString(value: -2147483648)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 126.5888661763779 > 79.17407946012858.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -123.82758879386076 (T) = (0 -125.6824751284138) / Math.Sqrt((0.9977401687889945 / (39)) + (1.2579164630850348 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.6684272244856588 = (75.32991147825406 - 125.6824751284138) / 75.32991147825406 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.TryFormat(value: 2147483647)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 115.08704042108766 > 73.0844658229811.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -278.89520886107795 (T) = (0 -114.84276853722938) / Math.Sqrt((0.671726523133363 / (39)) + (0.08231602437855678 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.6511445539907168 = (69.55343083660442 - 114.84276853722938) / 69.55343083660442 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare	51a8dd5323721b363e61069575511f783e7ea6d3
Diff	Diff

Regressions in System.Text.Json.Tests.Perf_Deep

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
WriteDeepUtf16 - Duration of single invocation	38.07 ms	43.56 ms	1.14	0.03	False
WriteDeepUtf16 - Duration of single invocation	16.68 ms	22.04 ms	1.32	0.02	False
WriteDeepUtf8 - Duration of single invocation	38.06 ms	43.72 ms	1.15	0.08	False
WriteDeepUtf8 - Duration of single invocation	37.72 ms	43.54 ms	1.15	0.09	False
WriteDeepUtf16 - Duration of single invocation	37.55 ms	44.95 ms	1.20	0.07	False
WriteDeepUtf8 - Duration of single invocation	16.66 ms	22.32 ms	1.34	0.02	False
WriteDeepUtf16 - Duration of single invocation	16.31 ms	22.42 ms	1.37	0.02	False
WriteDeepUtf8 - Duration of single invocation	16.35 ms	21.76 ms	1.33	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Deep*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: True, SkipValidation: False)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 43.563193722222216 > 40.12663703416667.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -47.16572847069028 (T) = (0 -43882063.06968525) / Math.Sqrt((199940985028.32224 / (39)) + (95998108047.06354 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.15616914889785183 = (37954708.540283196 - 43882063.06968525) / 37954708.540283196 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.03619524242424 > 17.512763683365385.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -102.99679410773679 (T) = (0 -22129010.68635069) / Math.Sqrt((16667330559.15118 / (39)) + (22566003207.359962 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.3371506368668238 = (16549377.51680903 - 22129010.68635069) / 16549377.51680903 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: True, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 43.717205722222225 > 40.06933508730769.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -57.33601456868164 (T) = (0 -43930807.54420703) / Math.Sqrt((149852487933.64014 / (40)) + (63709005302.158 / (9))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (9) - 2, .025) and -0.1571287973146338 = (37965356.705457434 - 43930807.54420703) / 37965356.705457434 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: True, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 43.54228723333333 > 39.564740506250004.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -19.736579984822185 (T) = (0 -44093901.18321124) / Math.Sqrt((424786763575.7694 / (40)) + (854753240932.4954 / (9))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (9) - 2, .025) and -0.170204557405457 = (37680507.14225121 - 44093901.18321124) / 37680507.14225121 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: True, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 44.9513435952381 > 39.534981679999994.
IsChangePoint: Marked as a change because one of 12/20/2022 1:34:12 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -29.60843936423054 (T) = (0 -43808217.06864741) / Math.Sqrt((117049774808.64949 / (39)) + (376220198262.84717 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.16694185093162894 = (37541045.4545555 - 43808217.06864741) / 37541045.4545555 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: False, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.323676993506496 > 17.45892462730769.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 12/21/2022 9:23:17 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -133.1278787162245 (T) = (0 -22145822.637384836) / Math.Sqrt((6866240964.815075 / (39)) + (14292210931.24869 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.33776669327209546 = (16554323.522001814 - 22145822.637384836) / 16554323.522001814 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.424703032467534 > 17.154001830416668.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 12/21/2022 9:23:17 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -39.20749545430782 (T) = (0 -21951443.495097496) / Math.Sqrt((7351527257.89311 / (40)) + (187604242986.47263 / (9))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (9) - 2, .025) and -0.3495412500896024 = (16265855.892615387 - 21951443.495097496) / 16265855.892615387 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: False, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.75978978321678 > 17.157650616153845.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 12/21/2022 9:23:17 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -99.83628599512195 (T) = (0 -21885960.194942098) / Math.Sqrt((77243859167.01202 / (39)) + (10133467443.399265 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.34093321683404004 = (16321439.367886733 - 21885960.194942098) / 16321439.367886733 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2eca2d319bf8d1247edf38facfdd501e0e2f0030
Compare	51a8dd5323721b363e61069575511f783e7ea6d3
Diff	Diff

Regressions in System.Tests.Perf_UInt32

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ToString - Duration of single invocation	43.08 ns	93.22 ns	2.16	0.00	False
TryFormat - Duration of single invocation	69.17 ns	115.26 ns	1.67	0.02	False
TryFormat - Duration of single invocation	28.74 ns	78.95 ns	2.75	0.02	False
Parse - Duration of single invocation	45.59 ns	48.98 ns	1.07	0.06	False
TryParse - Duration of single invocation	32.50 ns	35.35 ns	1.09	0.07	False
TryParse - Duration of single invocation	42.86 ns	45.42 ns	1.06	0.04	False
TryFormat - Duration of single invocation	45.22 ns	92.66 ns	2.05	0.01	False
Parse - Duration of single invocation	35.11 ns	37.00 ns	1.05	0.08	False
ToString - Duration of single invocation	67.92 ns	117.83 ns	1.73	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.ToString(value: 12345)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 93.220664396323 > 45.222740333014045.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -155.22064810013362 (T) = (0 -93.97805657559888) / Math.Sqrt((0.018173182209468754 / (39)) + (0.9632431285175578 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -1.1811119776243393 = (43.08722226997236 - 93.97805657559888) / 43.08722226997236 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryFormat(value: 4294967295)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 115.26075433243855 > 72.43904884038861.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -264.13624106763916 (T) = (0 -115.29671120185819) / Math.Sqrt((0.15569768910313828 / (39)) + (0.24349752794384438 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.6769189422620165 = (68.75508904821186 - 115.29671120185819) / 68.75508904821186 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryFormat(value: 0)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 78.95100589916447 > 30.19332177732932.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -111.88526257327347 (T) = (0 -78.56628563647409) / Math.Sqrt((0.059386599248268127 / (39)) + (1.767162296746951 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -1.7283448722169799 = (28.79631766369522 - 78.56628563647409) / 28.79631766369522 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.Parse(value: "4294967295")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.98133628044679 > 47.937817559959164.
IsChangePoint: Marked as a change because one of 11/11/2022 8:39:50 AM, 11/18/2022 4:41:14 PM, 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -12.962251287682406 (T) = (0 -48.72546064467585) / Math.Sqrt((0.35316543003010387 / (30)) + (0.6732196774341038 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.06134424309238525 = (45.90919577865418 - 48.72546064467585) / 45.90919577865418 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryParse(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.35186043824988 > 33.40195816612822.
IsChangePoint: Marked as a change because one of 11/11/2022 8:39:50 AM, 12/17/2022 10:44:50 PM, 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -11.131038795628637 (T) = (0 -34.640893070826074) / Math.Sqrt((0.25791309239015514 / (30)) + (0.777531070466115 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.07701164821805553 = (32.16389825322721 - 34.640893070826074) / 32.16389825322721 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryParse(value: "4294967295")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 45.418535137587725 > 44.63836716486543.
IsChangePoint: Marked as a change because one of 11/4/2022 4:27:34 PM, 11/11/2022 8:39:50 AM, 12/17/2022 10:44:50 PM, 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -11.925665091010737 (T) = (0 -44.935457625502984) / Math.Sqrt((0.3019643831886204 / (29)) + (0.682853190846856 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.060601131439256294 = (42.36791409464621 - 44.935457625502984) / 42.36791409464621 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryFormat(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 92.65847085830944 > 47.41601337985222.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -135.6452263475296 (T) = (0 -92.08780739627132) / Math.Sqrt((0.6078952435819791 / (40)) + (0.9348595785109052 / (9))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (9) - 2, .025) and -1.0336861980366285 = (45.2812274996876 - 92.08780739627132) / 45.2812274996876 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.Parse(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.00025900178148 > 36.86586432303018.
IsChangePoint: Marked as a change because one of 11/10/2022 11:11:41 PM, 12/17/2022 10:44:50 PM, 1/15/2023 6:48:05 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -16.698798501192098 (T) = (0 -37.17513847691453) / Math.Sqrt((0.487356518317323 / (40)) + (0.010854991554947157 / (8))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (8) - 2, .025) and -0.05515311773228114 = (35.23198467802547 - 37.17513847691453) / 35.23198467802547 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.ToString(value: 4294967295)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 117.82621050565767 > 71.3256308314694.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -61.18107239443043 (T) = (0 -118.60017368741102) / Math.Sqrt((0.03171470740205005 / (39)) + (6.1759350771212675 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -0.7469739993398788 = (67.8889174837324 - 118.60017368741102) / 67.8889174837324 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

naricc · 2023-01-31T19:07:16Z

@jandupej I think this is also related to your issue? Something in ehre: dotnet/runtime@191b98d...1555786

kotlarmilos · 2023-03-01T10:34:11Z

Mitigated by dotnet/runtime#81695.

jandupej · 2023-03-13T12:02:40Z

Remaining regressions: dotnet/runtime#83330

performanceautofiler bot added LLVM=false untriaged labels Jan 17, 2023

naricc removed the untriaged label Jan 31, 2023

naricc assigned jandupej Jan 31, 2023

naricc added the LinkedFromMaster Is this issue linked from the master perf issue label Jan 31, 2023

naricc mentioned this issue Jan 31, 2023

.NET 8 per-preview performance overview dotnet/runtime#77490

Closed

kotlarmilos closed this as completed Mar 1, 2023

performanceautofiler bot removed refs/heads/main labels Mar 24, 2023

performanceautofiler bot added perf-regression arch-x64 os-linux branch-refs/heads/main runtime-mono mono-aot kind-micro_mono and removed Mono labels Mar 24, 2023

[Perf] Linux/x64: 111 Regressions on 1/15/2023 6:48:05 PM #11780

[Perf] Linux/x64: 111 Regressions on 1/15/2023 6:48:05 PM #11780

Comments

performanceautofiler bot commented Jan 17, 2023 • edited Loading

Run Information

Regressions in System.Numerics.Tests.Perf_BigInteger

Repro

Payloads

Histogram

System.Numerics.Tests.Perf_BigInteger.ToStringD(numberString: 123)

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

Regressions in System.Tests.Perf_UInt64

Repro

Payloads

Histogram

System.Tests.Perf_UInt64.TryFormat(value: 0)

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

Run Information

Regressions in System.Perf_Convert

Repro

Payloads

Histogram

System.Perf_Convert.ChangeType

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

Run Information

Regressions in System.Xml.Tests.Perf_XmlConvert

Repro

Payloads

Histogram

System.Xml.Tests.Perf_XmlConvert.TimeSpan_ToString

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

performanceautofiler bot commented Jan 17, 2023 • edited Loading

Run Information

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Repro

Payloads

Histogram

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 12345)

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

Regressions in System.Text.Json.Tests.Perf_Basic

Repro

Payloads

Histogram

performanceautofiler bot commented Jan 17, 2023 •

edited

Loading

performanceautofiler bot commented Jan 17, 2023 •

edited

Loading

performanceautofiler bot commented Jan 17, 2023 •

edited

Loading

performanceautofiler bot commented Jan 17, 2023 •

edited

Loading