[Perf] Linux/arm64: 73 Regressions on 6/8/2023 3:52:42 PM

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryParseDecimal - Duration of single invocation	41.03 ns	47.66 ns	1.16	0.01	False
TryParseDateTimeOffset - Duration of single invocation	4.35 ns	7.06 ns	1.62	0.25	False
TryParseUInt32 - Duration of single invocation	0.35 ns	1.65 ns	4.68	0.63	False
TryParseUInt64 - Duration of single invocation	28.43 ns	30.91 ns	1.09	0.03	False
TryParseInt16 - Duration of single invocation	1.91 ns	3.05 ns	1.60	0.51	False
TryParseUInt32Hex - Duration of single invocation	2.33 ns	3.43 ns	1.47	0.56	False
TryParseUInt16 - Duration of single invocation	2.31 ns	3.40 ns	1.47	0.50	False
TryParseInt16 - Duration of single invocation	4.07 ns	5.19 ns	1.28	0.35	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseDecimal(value: 123456.789)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 47.660806412672905 > 43.12461232794377.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -37.89858481016644 (T) = (0 -47.85550178643013) / Math.Sqrt((0.30727609213412327 / (11)) + (0.08517766260064298 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.17242470643142851 = (40.81754804714963 - 47.85550178643013) / 40.81754804714963 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseDateTimeOffset(value: 12/30/2017 3:45:22 AM -08:00)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.055156524002294 > 4.652023214687725.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -22.617233995446007 (T) = (0 -6.802464221235504) / Math.Sqrt((0.04055860838073315 / (11)) + (0.10354255846675964 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.559817511798753 = (4.361064143581147 - 6.802464221235504) / 4.361064143581147 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32(value: 0)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6520437245985364 > 0.6103537384918126.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -9.925395584655949 (T) = (0 -1.7470253900005799) / Math.Sqrt((0.07612360098429191 / (11)) + (0.07509596422371864 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -1.779122272028069 = (0.6286248746895491 - 1.7470253900005799) / 0.6286248746895491 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value: 18446744073709551615)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.90916186034061 > 29.36058483498697.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -14.316676849529188 (T) = (0 -30.824090657658466) / Math.Sqrt((0.46424115436003 / (11)) + (0.0351787672205698 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.10917482791081597 = (27.790110162991276 - 30.824090657658466) / 27.790110162991276 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt16(value: 0)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.054266390035134 > 2.5419465835447084.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -7.900331664281703 (T) = (0 -2.9928171380941864) / Math.Sqrt((0.046739848465401926 / (11)) + (0.055868438259977345 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.32282334610063385 = (2.2624465669707856 - 2.9928171380941864) / 2.2624465669707856 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32Hex(value: 3039)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.432395927489877 > 2.608573315195493.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -3.4250832918732455 (T) = (0 -3.0668658254443746) / Math.Sqrt((0.02332270132459701 / (11)) + (0.4176402546768138 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.2605144162245754 = (2.4330271720573298 - 3.0668658254443746) / 2.4330271720573298 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt16(value: 0)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.4041492355139478 > 2.1885379122903377.
IsChangePoint: Marked as a change because one of 3/8/2023 10:34:21 AM, 3/9/2023 2:25:45 AM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -10.826742384395054 (T) = (0 -3.4368882588550633) / Math.Sqrt((0.070288019721046 / (11)) + (0.0767128384697058 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.5366622029450498 = (2.236593216302311 - 3.4368882588550633) / 2.236593216302311 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt16(value: 32767)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.190967129914621 > 4.728750225191263.
IsChangePoint: Marked as a change because one of 4/29/2023 4:36:15 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -9.37102944969917 (T) = (0 -5.091914169174353) / Math.Sqrt((0.0426944529963249 / (11)) + (0.027937361151483667 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.1667451419943408 = (4.364204302981405 - 5.091914169174353) / 4.364204302981405 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_Path

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetFileNameWithoutExtension - Duration of single invocation	31.73 ns	38.28 ns	1.21	0.20	False
GetExtension - Duration of single invocation	16.71 ns	20.43 ns	1.22	0.39	False
GetFileName - Duration of single invocation	20.79 ns	29.47 ns	1.42	0.21	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Path.GetFileNameWithoutExtension

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.278556980603575 > 34.17169716176137.
IsChangePoint: Marked as a change because one of 4/17/2023 5:51:51 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -10.221724043610893 (T) = (0 -39.30077272594382) / Math.Sqrt((3.493275974267829 / (11)) + (1.0494637622272727 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.1963796214003713 = (32.8497510513778 - 39.30077272594382) / 32.8497510513778 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_Path.GetExtension

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.4326549421891 > 17.434769299008128.
IsChangePoint: Marked as a change because one of 4/17/2023 5:51:51 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -8.604360616197114 (T) = (0 -20.325742538034856) / Math.Sqrt((1.289780620332479 / (11)) + (0.4616545523094346 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.19825978677553172 = (16.962717736468985 - 20.325742538034856) / 16.962717736468985 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_Path.GetFileName

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.469376340074447 > 25.03347697270492.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -7.105221640956451 (T) = (0 -29.10641640117524) / Math.Sqrt((6.040298027633847 / (11)) + (0.1558046134893716 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.22377511835975153 = (23.784121743043045 - 29.10641640117524) / 23.784121743043045 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Ascii

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
FromUtf16 - Duration of single invocation	3.96 ns	6.87 ns	1.73	0.16	False
ToUtf16 - Duration of single invocation	5.34 ns	7.04 ns	1.32	0.22	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Ascii*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Ascii.FromUtf16(Size: 6)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.867970387709472 > 5.660990138036966.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -9.226923201039526 (T) = (0 -6.853356546953053) / Math.Sqrt((0.18619197172942323 / (11)) + (0.16893445063708076 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.30358892705162094 = (5.2572988345747635 - 6.853356546953053) / 5.2572988345747635 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.ToUtf16(Size: 6)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.04237486796016 > 5.56696282374153.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -6.725051817934144 (T) = (0 -6.573081295565621) / Math.Sqrt((0.15729897068421445 / (11)) + (0.3546097163608854 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.263615669867114 = (5.201804197519067 - 6.573081295565621) / 5.201804197519067 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_String

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Replace_String - Duration of single invocation	64.50 ns	75.91 ns	1.18	0.02	False
Trim_CharArr - Duration of single invocation	19.71 ns	21.97 ns	1.11	0.24	False
CtorCharCount - Duration of single invocation	21.97 ns	24.17 ns	1.10	0.28	False
Replace_String - Duration of single invocation	13.41 ns	15.04 ns	1.12	0.42	False
Replace_String - Duration of single invocation	30.56 ns	34.77 ns	1.14	0.07	False
Replace_String - Duration of single invocation	54.51 ns	63.97 ns	1.17	0.01	False
Join_Array - Duration of single invocation	52.80 ns	70.55 ns	1.34	0.02	False
Remove_IntInt - Duration of single invocation	12.37 ns	15.11 ns	1.22	0.47	False
TrimEnd_CharArr - Duration of single invocation	4.35 ns	6.34 ns	1.46	0.59	False
Remove_IntInt - Duration of single invocation	12.32 ns	14.67 ns	1.19	0.44	False
Replace_Char - Duration of single invocation	20.69 ns	22.35 ns	1.08	0.16	False
Join_List - Duration of single invocation	57.30 ns	75.23 ns	1.31	0.03	False
Insert - Duration of single invocation	15.14 ns	16.83 ns	1.11	0.31	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 75.91032228344378 > 67.69740818129758.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -30.509764020553238 (T) = (0 -76.50084764185068) / Math.Sqrt((1.4164103661852516 / (11)) + (0.2550527387815935 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.18151148781438017 = (64.74828931487228 - 76.50084764185068) / 64.74828931487228 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Trim_CharArr(s: " Te st ", c: [' ', ' '])

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.968179215787664 > 19.155204213406297.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -16.467940590671006 (T) = (0 -22.046092789623085) / Math.Sqrt((0.31137464849073887 / (11)) + (0.23118943251710555 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.19099590533815686 = (18.510636930664834 - 22.046092789623085) / 18.510636930664834 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.CtorCharCount(size: 100)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.171572478106853 > 22.694256324423595.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -8.823683548082322 (T) = (0 -24.873331322269628) / Math.Sqrt((0.695252027739221 / (11)) + (0.7091031847396577 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.13860234658332202 = (21.845494519582406 - 24.873331322269628) / 21.845494519582406 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "bad", newValue: "nice")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.038745029771809 > 13.62036462552821.
IsChangePoint: Marked as a change because one of 3/12/2023 3:57:11 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -10.851673884494117 (T) = (0 -15.403378662046956) / Math.Sqrt((0.24670912078250976 / (11)) + (0.26237208984105415 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.1701724652077257 = (13.163340550243243 - 15.403378662046956) / 13.163340550243243 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "b")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.7667506932578 > 32.45060555164664.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -15.050663628665218 (T) = (0 -35.17180028627591) / Math.Sqrt((0.5466332165555555 / (11)) + (0.41149254510058114 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.1390150507040911 = (30.879135674751783 - 35.17180028627591) / 30.879135674751783 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "nice", newValue: "bad")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 63.965651357404745 > 56.82080072091466.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -25.27194121283916 (T) = (0 -64.487902511986) / Math.Sqrt((1.993500387461184 / (11)) + (0.20771083081074013 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.210694942459518 = (53.265195261309415 - 64.487902511986) / 53.265195261309415 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Join_Array

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 70.551134514935 > 55.454421271786394.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/10/2023 2:36:50 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -57.53946400611749 (T) = (0 -71.53697827686838) / Math.Sqrt((0.09241640112081184 / (11)) + (1.2572909128703673 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.35277254575745265 = (52.881749042898385 - 71.53697827686838) / 52.881749042898385 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 10, i2: 1)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.107141302965218 > 13.97605623968093.
IsChangePoint: Marked as a change because one of 5/25/2023 8:00:22 PM, 6/6/2023 5:17:50 PM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -4.685699611901317 (T) = (0 -15.572109432720122) / Math.Sqrt((1.0960493423845492 / (8)) + (0.5805572125381241 / (16))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (16) - 2, .025) and -0.14319781007170757 = (13.621535394424308 - 15.572109432720122) / 13.621535394424308 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.TrimEnd_CharArr(s: "Test", c: [' ', ' '])

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.338814034318995 > 4.562260014264121.
IsChangePoint: Marked as a change because one of 3/13/2023 3:41:35 AM, 4/6/2023 1:55:21 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -25.837056506015 (T) = (0 -6.1677978818813095) / Math.Sqrt((0.03199335741768614 / (11)) + (0.03136438279914467 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.4400360780919207 = (4.283085664113205 - 6.1677978818813095) / 4.283085664113205 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 7, i2: 4)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.672120899435495 > 13.630885458064505.
IsChangePoint: Marked as a change because one of 5/5/2023 12:48:01 AM, 5/27/2023 6:38:16 PM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -5.929021697102252 (T) = (0 -15.63662715543539) / Math.Sqrt((1.0266599837327952 / (11)) + (0.6018270824922828 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.16507385801616 = (13.421146691988092 - 15.63662715543539) / 13.421146691988092 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'i', newChar: 'I')

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.353622797387718 > 21.111138446985212.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -10.615134963269318 (T) = (0 -22.006081663065668) / Math.Sqrt((0.18796992033101428 / (11)) + (0.25867415830525525 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.10225502687906146 = (19.964600864986718 - 22.006081663065668) / 19.964600864986718 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Join_List

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 75.23079881759091 > 60.09401947376336.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/10/2023 2:36:50 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -32.74632430517555 (T) = (0 -76.47344265222455) / Math.Sqrt((2.8848181786841733 / (11)) + (0.7665371867242717 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.3204649020557574 = (57.914029015968055 - 76.47344265222455) / 57.914029015968055 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Insert(s1: "Test", i: 2, s2: " Test")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.834476783296836 > 15.432680151104487.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -10.241899348763466 (T) = (0 -17.10747127956353) / Math.Sqrt((0.7163697840765824 / (11)) + (0.24747512955870465 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.2101852141769907 = (14.136242187686774 - 17.10747127956353) / 14.136242187686774 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Get

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetInt64 - Duration of single invocation	795.08 ns	876.19 ns	1.10	0.01	False
GetUInt16 - Duration of single invocation	874.92 ns	1.21 μs	1.38	0.01	False
GetByte - Duration of single invocation	867.61 ns	1.20 μs	1.38	0.01	False
GetUInt32 - Duration of single invocation	620.86 ns	779.51 ns	1.26	0.01	False
GetGuid - Duration of single invocation	5.33 μs	5.60 μs	1.05	0.01	False
GetDecimal - Duration of single invocation	4.67 μs	5.20 μs	1.11	0.01	False
GetUInt64 - Duration of single invocation	744.39 ns	823.24 ns	1.11	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Get.GetInt64

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 876.1868350367732 > 821.9563775841408.
IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -34.41355477580237 (T) = (0 -871.1522425505261) / Math.Sqrt((57.41054482101593 / (11)) + (17.554021691629504 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.11265789430204087 = (782.9470738595633 - 871.1522425505261) / 782.9470738595633 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetUInt16

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.20865707995365 > 917.2387453664218.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -69.78208072226661 (T) = (0 -1198.4017430855629) / Math.Sqrt((216.16806818980098 / (11)) + (20.598237681985573 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.36674775237671065 = (876.8273011618992 - 1198.4017430855629) / 876.8273011618992 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetByte

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.198469486996113 > 913.0494035138629.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -116.5212191341499 (T) = (0 -1195.6313969712444) / Math.Sqrt((35.70370220351162 / (11)) + (56.96264199466029 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.3682760496720627 = (873.8232297918272 - 1195.6313969712444) / 873.8232297918272 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetUInt32

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 779.5102550242899 > 654.0994274174709.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/22/2023 11:33:11 PM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -267.90425397915396 (T) = (0 -782.4722501850831) / Math.Sqrt((2.4354595557367418 / (11)) + (1.7502925670624012 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.2567503848727855 = (622.6154848277916 - 782.4722501850831) / 622.6154848277916 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetGuid

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.603025468174376 > 5.591184619179778.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -76.76222943065198 (T) = (0 -5625.034144043242) / Math.Sqrt((15.764471758750139 / (11)) + (180.74887006887604 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.05646045260983311 = (5324.415249191209 - 5625.034144043242) / 5324.415249191209 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetDecimal

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.199888614711422 > 4.90385620774446.
IsChangePoint: Marked as a change because one of 3/31/2023 7:02:21 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -169.28016396019135 (T) = (0 -5196.28917081464) / Math.Sqrt((60.7166572787423 / (11)) + (52.24224939322281 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.11186694164914467 = (4673.481129951929 - 5196.28917081464) / 4673.481129951929 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetUInt64

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 823.2364765000176 > 776.0041151385143.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -19.232978315890076 (T) = (0 -819.2057043796968) / Math.Sqrt((176.38364962988155 / (11)) + (23.575057770191908 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.11010768669283635 = (737.9515646992982 - 819.2057043796968) / 737.9515646992982 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Tests.DictionarySequentialKeys

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryGetValue_3k_Int_32ByteRefsValue - Duration of single invocation	4.21 ns	6.93 ns	1.65	0.01	False
TryGetValue_3k_Int_Int - Duration of single invocation	3.94 ns	6.64 ns	1.69	0.05	False
TryGetValue_3k_Int_32ByteValue - Duration of single invocation	4.20 ns	6.86 ns	1.63	0.01	False
TryGetValue_17_Int_Int - Duration of single invocation	3.88 ns	6.40 ns	1.65	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.DictionarySequentialKeys*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_32ByteRefsValue

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.925765112477837 > 4.41961458459647.
IsChangePoint: Marked as a change because one of 5/4/2023 12:54:08 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -173.11215246790943 (T) = (0 -6.856185245683918) / Math.Sqrt((7.064661031369072E-07 / (11)) + (0.0030386533875426587 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.6288679303338204 = (4.209171976440601 - 6.856185245683918) / 4.209171976440601 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_Int

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.64156762656986 > 4.140900719842194.
IsChangePoint: Marked as a change because one of 5/4/2023 12:54:08 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -89.02926342507101 (T) = (0 -6.550785186096254) / Math.Sqrt((2.2636859161895397E-05 / (11)) + (0.011128961029064766 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.6614669058455386 = (3.942771994463825 - 6.550785186096254) / 3.942771994463825 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_32ByteValue

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.8615419210218125 > 4.4138268725198415.
IsChangePoint: Marked as a change because one of 5/4/2023 12:54:08 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -151.92493446674086 (T) = (0 -6.8352266736848195) / Math.Sqrt((5.566780992726099E-06 / (11)) + (0.003893873178966821 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.6260190847192143 = (4.203657102133673 - 6.8352266736848195) / 4.203657102133673 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_Int

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.397302324183512 > 4.112509238818076.
IsChangePoint: Marked as a change because one of 5/4/2023 12:54:08 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -106.26809135705126 (T) = (0 -6.412439473707552) / Math.Sqrt((0.0026263629205096287 / (11)) + (0.004015843673940449 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.6335288787829889 = (3.9255133820988495 - 6.412439473707552) / 3.9255133820988495 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Insert_Strings - Duration of single invocation	612.41 ns	854.22 ns	1.39	0.55	False
Insert_Primitives - Duration of single invocation	27.03 μs	38.20 μs	1.41	0.19	False
ToString_MultipleSegments - Duration of single invocation	30.93 ns	35.33 ns	1.14	0.20	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_StringBuilder.Insert_Strings

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 854.2163662354409 > 663.9922752324635.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -12.376421209552962 (T) = (0 -786.7329789741761) / Math.Sqrt((504.2442560236044 / (11)) + (1637.5053355882962 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.2597583397936915 = (624.5110305068654 - 786.7329789741761) / 624.5110305068654 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_StringBuilder.Insert_Primitives

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.201820562039316 > 28.479541090725938.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -39.55746674501332 (T) = (0 -37605.50462023124) / Math.Sqrt((96479.28954479723 / (11)) + (779002.4708906175 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.3806434283560237 = (27237.666038804324 - 37605.50462023124) / 27237.666038804324 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_StringBuilder.ToString_MultipleSegments(length: 100)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.33046481595832 > 33.53048517486927.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -8.164167190558512 (T) = (0 -35.924129824674395) / Math.Sqrt((1.7027675999445542 / (11)) + (0.9355243066121615 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.12135335473330401 = (32.036404647149226 - 35.924129824674395) / 32.036404647149226 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Enum

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Parse_Flags - Duration of single invocation	199.13 ns	233.76 ns	1.17	0.02	False
TryParseGeneric_Flags - Duration of single invocation	98.27 ns	111.59 ns	1.14	0.01	False
ToString_Flags - Duration of single invocation	61.40 ns	71.38 ns	1.16	0.05	False
TryParseGeneric_Flags - Duration of single invocation	13.65 ns	18.04 ns	1.32	0.05	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.Parse_Flags(text: "Red, Orange, Yellow, Green, Blue")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 233.76103476891132 > 211.48596541623687.
IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -21.403401622469172 (T) = (0 -225.2955014675422) / Math.Sqrt((5.210696282354547 / (11)) + (10.67225810173681 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.12119552423483713 = (200.94220552770733 - 225.2955014675422) / 200.94220552770733 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.TryParseGeneric_Flags(text: "Red, Orange, Yellow, Green, Blue")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 111.59146341000964 > 103.47464118181739.
IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -89.957410716652 (T) = (0 -111.65070825381325) / Math.Sqrt((0.21328138471772454 / (11)) + (0.017149605924349585 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.13115028094362874 = (98.7054595085915 - 111.65070825381325) / 98.7054595085915 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_Flags(value: Red, Orange, Yellow, Green, Blue)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 71.37748584804233 > 65.1234596451796.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -17.72222218335717 (T) = (0 -70.51215255375118) / Math.Sqrt((1.5113222699917075 / (11)) + (0.7521958600291748 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.12493424046084606 = (62.681132832142104 - 70.51215255375118) / 62.681132832142104 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.TryParseGeneric_Flags(text: "Red")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.039376914256113 > 14.427384310207051.
IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -18.011948851782865 (T) = (0 -17.892030385411847) / Math.Sqrt((0.617121838476879 / (11)) + (0.033590941682971553 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.3225333323232584 = (13.528604495722918 - 17.892030385411847) / 13.528604495722918 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.ReadOnlySpan

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IndexOfString - Duration of single invocation	14.58 ns	16.29 ns	1.12	0.01	False
IndexOfString - Duration of single invocation	9.86 ns	12.06 ns	1.22	0.01	False
IndexOfString - Duration of single invocation	11.73 ns	13.61 ns	1.16	0.01	False
IndexOfString - Duration of single invocation	21.96 ns	23.62 ns	1.08	0.01	False
IndexOfString - Duration of single invocation	10.89 ns	12.84 ns	1.18	0.01	False
IndexOfString - Duration of single invocation	23.83 ns	25.78 ns	1.08	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "StrIng", value: "string", comparisonType: OrdinalIgnoreCase)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.286988495671267 > 13.983113552729288.
IsChangePoint: Marked as a change because one of 5/1/2023 10:17:01 AM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -12.133480472642516 (T) = (0 -16.275756629880785) / Math.Sqrt((0.4269483175296206 / (11)) + (0.00025050457779772126 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.1722055163230709 = (13.884729600090905 - 16.275756629880785) / 13.884729600090905 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!", value: "y", comparisonType: Ordinal)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.058406981939644 > 10.382679389196479.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -246.97226380787552 (T) = (0 -12.041032810152464) / Math.Sqrt((0.0006967430820934968 / (11)) + (0.00015043423187038408 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.21584305250585367 = (9.903443364121614 - 12.041032810152464) / 9.903443364121614 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだすだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "す", comparisonType: Ordinal)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.609059348750794 > 12.311911474635638.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -53.51093748030656 (T) = (0 -13.604444764052232) / Math.Sqrt((0.014612473251864669 / (11)) + (0.0003261157342546361 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.16919002920185655 = (11.635785821180205 - 13.604444764052232) / 11.635785821180205 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "More Test's", value: "Tests", comparisonType: OrdinalIgnoreCase)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.620953071328923 > 21.636965132647063.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -11.049728095859797 (T) = (0 -23.609545143717636) / Math.Sqrt((0.5144431916395187 / (11)) + (0.0003455175949070959 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.11264627352262976 = (21.21927310192663 - 23.609545143717636) / 21.21927310192663 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "w", comparisonType: OrdinalIgnoreCase)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.843763911846429 > 11.47204492557835.
IsChangePoint: Marked as a change because one of 5/1/2023 10:17:01 AM, 5/4/2023 7:16:27 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -69.84548372287243 (T) = (0 -12.844973482383937) / Math.Sqrt((0.0058106078805219685 / (11)) + (0.0028515573985544787 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.17463869604294002 = (10.935254836789726 - 12.844973482383937) / 10.935254836789726 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.776634248017785 > 25.02659396863801.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -42.21679701502951 (T) = (0 -25.805211419185067) / Math.Sqrt((0.014977032959007024 / (11)) + (0.009323669974468237 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.08060186746388727 = (23.88040609234599 - 25.805211419185067) / 23.88040609234599 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Double

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToStringWithFormat - Duration of single invocation	115.38 ns	128.11 ns	1.11	0.01	False
ToStringWithFormat - Duration of single invocation	124.53 ns	144.19 ns	1.16	0.02	False
ToStringWithCultureInfo - Duration of single invocation	109.94 ns	122.09 ns	1.11	0.01	False
ToStringWithFormat - Duration of single invocation	125.04 ns	140.95 ns	1.13	0.01	False
ToStringWithFormat - Duration of single invocation	124.06 ns	144.36 ns	1.16	0.01	False
ToStringWithFormat - Duration of single invocation	183.59 ns	198.03 ns	1.08	0.01	False
ToStringWithFormat - Duration of single invocation	115.18 ns	128.33 ns	1.11	0.01	False
ToString - Duration of single invocation	117.03 ns	128.94 ns	1.10	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "R")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 128.10958797590004 > 121.5153318640843.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -33.57020496329302 (T) = (0 -128.94810527292714) / Math.Sqrt((1.270503741208978 / (11)) + (0.4219050183066903 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.11128273486302451 = (116.03537176236355 - 128.94810527292714) / 116.03537176236355 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "E")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 144.18980229015702 > 131.79905102173035.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -24.403157462078074 (T) = (0 -145.17740144944162) / Math.Sqrt((0.9947123487136971 / (11)) + (7.4260959427682955 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.15838731002487316 = (125.32716837715041 - 145.17740144944162) / 125.32716837715041 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithCultureInfo(value: 12345, culture: zh)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 122.0886867974638 > 115.22525800663493.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -25.747152784753638 (T) = (0 -122.25681839802223) / Math.Sqrt((2.3527978227784887 / (11)) + (0.3330871319322645 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.114910728366016 = (109.6561502975208 - 122.25681839802223) / 109.6561502975208 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "E")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 140.94695529567164 > 128.91926725766317.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -34.13475572055457 (T) = (0 -143.07461296666585) / Math.Sqrt((1.1013403647998201 / (11)) + (3.0424148657392003 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.15997667016918202 = (123.342664250134 - 143.07461296666585) / 123.342664250134 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "E")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 144.35815145943678 > 130.4015217053047.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -50.56728885591235 (T) = (0 -144.5920623011286) / Math.Sqrt((1.5218326483757352 / (11)) + (0.09872865583057945 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.15420551255140982 = (125.27410476623268 - 144.5920623011286) / 125.27410476623268 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G17")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 198.0278440186184 > 190.2858321846546.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -30.730950965563412 (T) = (0 -198.3686313003622) / Math.Sqrt((0.5819937540914268 / (11)) + (3.1549236330196595 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.09197370768610795 = (181.66062965078646 - 198.3686313003622) / 181.66062965078646 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "G")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 128.32615291642242 > 122.75535751270775.
IsChangePoint: Marked as a change because one of 4/11/2023 5:52:15 AM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -35.54444856682341 (T) = (0 -129.02282497718073) / Math.Sqrt((1.2274004518014512 / (11)) + (0.24410938716152522 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.11045278827837468 = (116.18938359118835 - 129.02282497718073) / 116.18938359118835 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToString(value: 12345)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 128.93683288135637 > 121.70276194003688.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -22.66753887061889 (T) = (0 -129.08962653967112) / Math.Sqrt((3.046324377537492 / (11)) + (0.1995380060609665 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.10489085936520745 = (116.83473118226078 - 129.08962653967112) / 116.83473118226078 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Sort<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Array - Duration of single invocation	5.71 μs	6.09 μs	1.07	0.18	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<Int32>.Array(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.0854521578947365 > 5.938302972.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -7.872932982973314 (T) = (0 -6343.607521438506) / Math.Sqrt((10863.00008750731 / (11)) + (107396.34557188534 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.1355321912518536 = (5586.462075060218 - 6343.607521438506) / 5586.462075060218 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Strings

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
WriteStringsUtf8 - Duration of single invocation	4.42 ms	4.78 ms	1.08	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Strings*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: False, SkipValidation: True, Escaped: NoneEscaped)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.775847438186814 > 4.649525541636969.
IsChangePoint: Marked as a change because one of 5/10/2023 2:36:50 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -27.051533355675875 (T) = (0 -4758080.909970307) / Math.Sqrt((961185129.2248514 / (11)) + (610885186.0519683 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.07055413947909177 = (4444502.836900416 - 4758080.909970307) / 4444502.836900416 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_Matrix4x4

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
DecomposeBenchmark - Duration of single invocation	73.28 ns	77.67 ns	1.06	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Matrix4x4.DecomposeBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 77.66710360730693 > 77.03448112517621.
IsChangePoint: Marked as a change because one of 2/23/2023 8:54:44 PM, 5/3/2023 4:46:09 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -21.729590533681115 (T) = (0 -77.60364920447634) / Math.Sqrt((0.24189072840610937 / (11)) + (0.17786510768047048 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.05583807435868048 = (73.49957449830842 - 77.60364920447634) / 73.49957449830842 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Depth

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ReadSpanEmptyLoop - Duration of single invocation	23.09 μs	26.15 μs	1.13	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Depth*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Depth.ReadSpanEmptyLoop(Depth: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.15123517219388 > 24.344708258765074.
IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -48.780913460155546 (T) = (0 -26208.20773589109) / Math.Sqrt((30139.59617045419 / (11)) + (13248.603157875701 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.1288183836723921 = (23217.3821005889 - 26208.20773589109) / 23217.3821005889 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Single

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToStringWithFormat - Duration of single invocation	116.21 ns	127.52 ns	1.10	0.02	False
ToStringWithCultureInfo - Duration of single invocation	109.82 ns	121.51 ns	1.11	0.03	False
ToString - Duration of single invocation	118.01 ns	127.89 ns	1.08	0.02	False
ToStringWithFormat - Duration of single invocation	116.64 ns	127.68 ns	1.09	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "R")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 127.51792814193102 > 122.23942871706831.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -17.180640860447916 (T) = (0 -129.502435538448) / Math.Sqrt((1.5596605025026506 / (11)) + (5.11719628979264 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.10751181533768724 = (116.93097422979808 - 129.502435538448) / 116.93097422979808 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithCultureInfo(value: 12345, culture: zh)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 121.50886384365675 > 114.11259914897182.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -15.529888362359456 (T) = (0 -122.77737661400289) / Math.Sqrt((4.444072239624857 / (11)) + (3.893021143893208 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.11868031435832528 = (109.75197743103932 - 122.77737661400289) / 109.75197743103932 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToString(value: 12345)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 127.8899721179637 > 122.4049418169037.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -22.230199323746344 (T) = (0 -128.8855414023098) / Math.Sqrt((2.806151739672803 / (11)) + (0.5366082671763269 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.10363085861062385 = (116.78319829201368 - 128.8855414023098) / 116.78319829201368 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "G")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 127.68127267752325 > 121.71108878721024.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -48.117138240442216 (T) = (0 -129.46332552668994) / Math.Sqrt((0.24840572816308434 / (11)) + (0.7051740906468368 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.11484822912729853 = (116.12641267595109 - 129.46332552668994) / 116.12641267595109 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Decimal

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	68.71 ns	75.64 ns	1.10	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Decimal*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Decimal.ToString(value: 123456.789)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 75.63913698164221 > 72.05329929103512.
IsChangePoint: Marked as a change because one of 4/29/2023 8:32:21 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -32.63920695335976 (T) = (0 -76.01545829331391) / Math.Sqrt((0.11541824176990083 / (11)) + (0.5809925117768433 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.11218146245656646 = (68.3480716585694 - 76.01545829331391) / 68.3480716585694 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetChars - Duration of single invocation	28.65 ns	30.78 ns	1.07	0.18	False
GetString - Duration of single invocation	20.73 ns	25.61 ns	1.24	0.25	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetChars(size: 16, encName: "utf-8")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.777830533777333 > 30.712757131223942.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -6.528320996577533 (T) = (0 -32.20423460709815) / Math.Sqrt((0.9419472178843791 / (11)) + (1.107469096403043 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.09144538384759172 = (29.506043164130617 - 32.20423460709815) / 29.506043164130617 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_Encoding.GetString(size: 16, encName: "utf-8")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.606416846972788 > 21.526372546258692.
IsChangePoint: Marked as a change because one of 4/26/2023 5:34:01 AM, 5/25/2023 4:37:39 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -10.842547757595538 (T) = (0 -23.717353139489227) / Math.Sqrt((0.1914488841679191 / (11)) + (0.889221801298473 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.1546190403586385 = (20.541280119650835 - 23.717353139489227) / 20.541280119650835 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in BenchmarksGame.ReverseComplement_1

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
RunBench - Duration of single invocation	666.06 μs	711.08 μs	1.07	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BenchmarksGame.ReverseComplement_1*'

Payloads

Baseline
Compare

Histogram

BenchmarksGame.ReverseComplement_1.RunBench

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 711.0806727272726 > 699.5436659405049.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -46.199582925221826 (T) = (0 -711150.8244372613) / Math.Sqrt((5155732.999994658 / (11)) + (6561979.8941600965 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.06848681052631132 = (665568.1824345265 - 711150.8244372613) / 665568.1824345265 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in BenchmarksGame.KNucleotide_1

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
RunBench - Duration of single invocation	320.28 ms	367.41 ms	1.15	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BenchmarksGame.KNucleotide_1*'

Payloads

Baseline
Compare

Histogram

BenchmarksGame.KNucleotide_1.RunBench

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 367.4140510666667 > 335.16523215000007.
IsChangePoint: Marked as a change because one of 5/3/2023 1:18:01 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -28.82233871162661 (T) = (0 -367145763.02820516) / Math.Sqrt((5975255390463.162 / (11)) + (27365486502226.58 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.14646365214168044 = (320241956.5089126 - 367145763.02820516) / 320241956.5089126 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Booleans

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
WriteBooleans - Duration of single invocation	1.81 ms	2.11 ms	1.17	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Booleans*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Booleans.WriteBooleans(Formatted: True, SkipValidation: True)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.11373586453202 > 1.8965515837769784.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -28.85793844538245 (T) = (0 -2091449.68139853) / Math.Sqrt((349701193.400844 / (11)) + (874688275.8278222 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.15920622224485267 = (1804208.4671943425 - 2091449.68139853) / 1804208.4671943425 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Basic

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
WriteBasicUtf8 - Duration of single invocation	669.47 ns	723.41 ns	1.08	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Basic*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: False, DataSize: 10)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 723.4088336967769 > 704.1727600206511.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/24/2023 7:54:29 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:09:45 PM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -21.771522248091923 (T) = (0 -730.0522455890441) / Math.Sqrt((15.074439753653888 / (11)) + (78.11077369553945 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.08814952349243196 = (670.9116990153436 - 730.0522455890441) / 670.9116990153436 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[AndyAyersMS]

Guessing this was dotnet/runtime#87115.

@EgorBo we expected some regressions from this, right?