[Perf] Linux/x64: 2 Regressions on 10/8/2023 11:25:37 PM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	a2d0a12769879de68bdfa75e4401d330e2925a1e
Compare	d3a782e3bb6ad0c1cb590b41c3f03e733f7d0d61
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.CreateAddAndRemove<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Queue - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	20.47 μs	23.54 μs	1.15	0.45	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove<String>*'

Payloads

Baseline
Compare

System.Collections.CreateAddAndRemove<String>.Queue(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.541069639265963 > 21.850061366697023.
IsChangePoint: Marked as a change because one of 10/8/2023 9:33:54 PM, 10/9/2023 8:09:24 PM falls between 10/1/2023 7:44:15 AM and 10/9/2023 8:09:24 PM.
IsRegressionStdDev: Marked as regression because -5.981185304616121 (T) = (0 -23935.044414731976) / Math.Sqrt((1676459.2468989727 / (46)) + (310432.2474069906 / (2))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (2) - 2, .025) and -0.12283931331132618 = (21316.535795443404 - 23935.044414731976) / 21316.535795443404 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	a2d0a12769879de68bdfa75e4401d330e2925a1e
Compare	d3a782e3bb6ad0c1cb590b41c3f03e733f7d0d61
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Reader

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	14.65 μs	16.20 μs	1.11	0.39	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Reader*'

Payloads

Baseline
Compare

System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json4KB)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.201876428693275 > 15.124242038317279.
IsChangePoint: Marked as a change because one of 10/8/2023 9:33:54 PM, 10/9/2023 8:09:24 PM falls between 10/1/2023 7:44:15 AM and 10/9/2023 8:09:24 PM.
IsRegressionStdDev: Marked as regression because -10.562148014426091 (T) = (0 -16346.564335638086) / Math.Sqrt((415983.5560159283 / (46)) + (41869.18083214028 / (2))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (2) - 2, .025) and -0.12596499894638405 = (14517.82635422442 - 16346.564335638086) / 14517.82635422442 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository