[Perf] Windows/arm64: 6 Regressions on 4/17/2023 8:43:19 PM

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	7ce83de7a397cb9331242c178c687307c0e31f88
Compare	ac3b110a14820325ca7b0f2773dba9bd4de97e3a
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
DeserializeFromString - Duration of single invocation	37.60 μs	42.41 μs	1.13	0.38	False
DeserializeFromStream - Duration of single invocation	42.31 μs	49.89 μs	1.18	0.33	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 42.41390753927886 > 38.45945574201291.
IsChangePoint: Marked as a change because one of 3/4/2023 1:07:20 AM, 4/17/2023 2:42:45 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsRegressionStdDev: Marked as regression because -6.6202222946715255 (T) = (0 -42238.821002713914) / Math.Sqrt((3700322.152242307 / (18)) + (1952956.7590003207 / (7))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (7) - 2, .025) and -0.1224645717482006 = (37630.42688904504 - 42238.821002713914) / 37630.42688904504 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 49.894279745324326 > 43.39422911614791.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 4/17/2023 5:51:51 AM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsRegressionStdDev: Marked as regression because -8.284943972502724 (T) = (0 -47576.2253446473) / Math.Sqrt((2626679.219713787 / (17)) + (2101964.0723833893 / (8))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (8) - 2, .025) and -0.12674359229674265 = (42224.53597243753 - 47576.2253446473) / 42224.53597243753 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	7ce83de7a397cb9331242c178c687307c0e31f88
Compare	ac3b110a14820325ca7b0f2773dba9bd4de97e3a
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int16

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryParse - Duration of single invocation	15.65 ns	17.72 ns	1.13	0.10	False
TryParse - Duration of single invocation	14.29 ns	16.54 ns	1.16	0.10	False
Parse - Duration of single invocation	15.44 ns	17.96 ns	1.16	0.10	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int16.TryParse(value: "-32768")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.72398905478734 > 16.390348966313212.
IsChangePoint: Marked as a change because one of 4/17/2023 2:42:45 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsRegressionStdDev: Marked as regression because -17.09102014372371 (T) = (0 -17.66059782596894) / Math.Sqrt((0.09429871335458624 / (18)) + (0.058841908935942114 / (6))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (6) - 2, .025) and -0.13469427023710076 = (15.564190539429319 - 17.66059782596894) / 15.564190539429319 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int16.TryParse(value: "32767")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.53729955058293 > 15.423870270016762.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsRegressionStdDev: Marked as regression because -13.424701322846975 (T) = (0 -16.443083990658764) / Math.Sqrt((0.21365657788526118 / (19)) + (0.025403038283471215 / (6))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (6) - 2, .025) and -0.1130603819781804 = (14.772858918431167 - 16.443083990658764) / 14.772858918431167 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int16.Parse(value: "-32768")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.963287438228072 > 16.295007727592953.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsRegressionStdDev: Marked as regression because -19.100057529011227 (T) = (0 -17.91726242659026) / Math.Sqrt((0.2615820405765825 / (19)) + (0.005808570831803422 / (6))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (6) - 2, .025) and -0.14863793375286188 = (15.598703386062203 - 17.91726242659026) / 15.598703386062203 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	7ce83de7a397cb9331242c178c687307c0e31f88
Compare	ac3b110a14820325ca7b0f2773dba9bd4de97e3a
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<ClassImplementingIXmlSerialiable>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
XmlSerializer_ - Duration of single invocation	1.30 μs	1.38 μs	1.06	0.17	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream<ClassImplementingIXmlSerialiable>*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<ClassImplementingIXmlSerialiable>.XmlSerializer_

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3798525906964427 > 1.365640732738552.
IsChangePoint: Marked as a change because one of 12/28/2022 7:19:51 AM, 3/31/2023 7:02:21 PM, 4/17/2023 2:42:45 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsRegressionStdDev: Marked as regression because -6.674108591879194 (T) = (0 -1399.4277166160537) / Math.Sqrt((711.5225135684504 / (18)) + (907.7515101938616 / (7))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (7) - 2, .025) and -0.06614049590396355 = (1312.6109757509034 - 1399.4277166160537) / 1312.6109757509034 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[EgorBo]

Small regression from dotnet/runtime#84582