[Perf] Linux/x64: 4 Regressions on 11/23/2022 5:21:28 AM

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	3d4f51254ab9993276c743c8f0fbe6879179daa6
Compare	0fffb89ff019bff325cea3ca90a91eec174d5968
Diff	Diff

Regressions in System.Net.Primitives.Tests.CredentialCacheTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetCredential_Uri - Duration of single invocation	2.21 μs	2.61 μs	1.18	0.02	False
GetCredential_Uri - Duration of single invocation	14.70 μs	15.98 μs	1.09	0.01	False
GetCredential_Uri - Duration of single invocation	18.75 μs	19.79 μs	1.06	0.00	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Primitives.Tests.CredentialCacheTests*'

Payloads

Baseline
Compare

Histogram

System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString: "http://notfound", uriCount: 0)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.6114888172361304 > 2.3284576450260634.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/23/2022 1:02:40 AM, 11/29/2022 1:43:29 AM falls between 11/20/2022 12:28:28 PM and 11/29/2022 1:43:29 AM.
IsRegressionStdDev: Marked as regression because -27.44301787712773 (T) = (0 -2793.021148448572) / Math.Sqrt((1200.326062223437 / (29)) + (9595.5665209751 / (24))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (24) - 2, .025) and -0.2600569294708336 = (2216.5833012175995 - 2793.021148448572) / 2216.5833012175995 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString: "http://notfound", uriCount: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.983204209489552 > 15.27612907764446.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/23/2022 1:02:40 AM, 11/29/2022 1:43:29 AM falls between 11/20/2022 12:28:28 PM and 11/29/2022 1:43:29 AM.
IsRegressionStdDev: Marked as regression because -35.8996447255514 (T) = (0 -16063.342859093187) / Math.Sqrt((18147.871439066006 / (28)) + (21548.891101165653 / (24))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (24) - 2, .025) and -0.0963397105795481 = (14651.793330191213 - 16063.342859093187) / 14651.793330191213 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString: "http://name5", uriCount: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.79331965961294 > 19.5133717390221.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/23/2022 1:02:40 AM, 11/29/2022 1:43:29 AM falls between 11/20/2022 12:28:28 PM and 11/29/2022 1:43:29 AM.
IsRegressionStdDev: Marked as regression because -37.44571416467785 (T) = (0 -19883.180553011665) / Math.Sqrt((13235.735747349352 / (29)) + (18083.702730598903 / (24))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (24) - 2, .025) and -0.07009930273989946 = (18580.68732696344 - 19883.180553011665) / 18580.68732696344 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	3d4f51254ab9993276c743c8f0fbe6879179daa6
Compare	0fffb89ff019bff325cea3ca90a91eec174d5968
Diff	Diff

Regressions in System.Tests.Perf_Uri

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Ctor - Duration of single invocation	1.23 μs	1.83 μs	1.48	0.03	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Uri*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Uri.Ctor

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.8283604871316381 > 1.297379711143715.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/23/2022 1:02:40 AM, 11/29/2022 1:43:29 AM falls between 11/20/2022 12:28:28 PM and 11/29/2022 1:43:29 AM.
IsRegressionStdDev: Marked as regression because -83.95980415606728 (T) = (0 -1792.2527126280538) / Math.Sqrt((465.57978805456963 / (29)) + (667.0179948168272 / (24))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (24) - 2, .025) and -0.4496952395128483 = (1236.296197834186 - 1792.2527126280538) / 1236.296197834186 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[kotlarmilos]

Regression in d7aabd5. @MihaZupan Could you check it?

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[MihaZupan]

Does the Mono interpreter not support Vector128 and friends yet?

[kotlarmilos]

Mono interpreter doesn't support Vector128 yet. If that is a cause of the regression I suggest we close the issue and review it once it is implemented.

[MihaZupan]

Yes, it's due to vectorization - I changed the algorithm such that it now favors platforms with Vector128 support.
More info here: #78975 (comment)