[Perf] Windows/x86: 3 Regressions on 6/30/2023 11:17:52 PM #19466
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x86 |
| OS | Windows 10.0.18362 |
| Queue | TigerWindows |
| Baseline | 299a8c9c178c38c6e7ca62b1bcce8f6e0d895ebe |
| Compare | e80ef8638b6d3f42add1bb046e032b2eff417473 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Tests.Perf_Guid
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ctor_str - Duration of single invocation | 36.09 ns | 42.66 ns | 1.18 | 0.03 | False | |||||
| Parse - Duration of single invocation | 36.85 ns | 43.18 ns | 1.17 | 0.03 | False | |||||
| ParseExactD - Duration of single invocation | 36.89 ns | 45.46 ns | 1.23 | 0.02 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Guid*'Payloads
System.Tests.Perf_Guid.ctor_str
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 42.65530303283029 > 37.75052110670917.
IsChangePoint: Marked as a change because one of 4/1/2023 1:31:30 PM, 4/10/2023 1:33:42 PM, 4/25/2023 3:46:39 AM, 5/26/2023 5:44:43 PM, 6/9/2023 2:21:24 AM, 6/30/2023 4:39:46 PM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsRegressionStdDev: Marked as regression because -26.504747433989873 (T) = (0 -42.745504492165615) / Math.Sqrt((0.5128246681224163 / (36)) + (0.35702960954982044 / (7))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (7) - 2, .025) and -0.18819550950712377 = (35.97514394739373 - 42.745504492165615) / 35.97514394739373 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Guid.Parse
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 43.17879279868558 > 38.0330096592404.
IsChangePoint: Marked as a change because one of 4/1/2023 1:31:30 PM, 4/10/2023 1:33:42 PM, 4/25/2023 3:46:39 AM, 5/26/2023 5:44:43 PM, 6/9/2023 2:21:24 AM, 6/30/2023 4:39:46 PM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsRegressionStdDev: Marked as regression because -69.78256340019591 (T) = (0 -43.10109296037118) / Math.Sqrt((0.351092401549357 / (36)) + (0.0047565995232243296 / (7))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (7) - 2, .025) and -0.1981288042941756 = (35.97367228456066 - 43.10109296037118) / 35.97367228456066 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Guid.ParseExactD
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 45.4624608663326 > 38.6690095108119.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 3:46:39 AM, 6/30/2023 4:39:46 PM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsRegressionStdDev: Marked as regression because -84.14418105871742 (T) = (0 -45.64060471282085) / Math.Sqrt((0.10753998907459192 / (36)) + (0.055660481524020144 / (7))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (7) - 2, .025) and -0.23888367170649388 = (36.84010513267435 - 45.64060471282085) / 36.84010513267435 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository