[Perf] Linux/arm64: 398 Improvements on 4/4/2023 3:37:20 PM #15797

performanceautofiler · 2023-04-11T18:53:31Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ExclusiveOrOperatorBenchmark - Duration of single invocation	13.19 ns	1.00 ns	0.08	0.14	False
AddOperatorBenchmark - Duration of single invocation	36.03 ns	1.01 ns	0.03	0.11	False
UnaryNegateOperatorBenchmark - Duration of single invocation	32.81 ns	1.13 ns	0.03	0.08	False
AddBenchmark - Duration of single invocation	36.57 ns	1.06 ns	0.03	0.10	False
SubtractionOperatorBenchmark - Duration of single invocation	35.05 ns	1.07 ns	0.03	0.10	False
AllBitsSetBenchmark - Duration of single invocation	6.50 ns	1.16 ns	0.18	0.18	True
MultiplyBenchmark - Duration of single invocation	36.23 ns	1.00 ns	0.03	0.04	True
XorBenchmark - Duration of single invocation	13.18 ns	1.04 ns	0.08	0.13	False
SubtractBenchmark - Duration of single invocation	35.98 ns	1.01 ns	0.03	0.10	False
BitwiseAndOperatorBenchmark - Duration of single invocation	13.19 ns	1.15 ns	0.09	0.12	False
BitwiseOrBenchmark - Duration of single invocation	13.18 ns	1.03 ns	0.08	0.12	False
OnesComplementBenchmark - Duration of single invocation	11.13 ns	1.62 ns	0.15	0.15	False
MultiplyOperatorBenchmark - Duration of single invocation	36.11 ns	1.17 ns	0.03	0.05	True
EqualityOperatorBenchmark - Duration of single invocation	3.29 ns	0.02 ns	0.01	0.27	False
NegateBenchmark - Duration of single invocation	32.64 ns	0.99 ns	0.03	0.10	False
OnesComplementOperatorBenchmark - Duration of single invocation	11.12 ns	1.01 ns	0.09	0.16	False
BitwiseOrOperatorBenchmark - Duration of single invocation	13.30 ns	1.01 ns	0.08	0.12	False
BitwiseAndBenchmark - Duration of single invocation	13.25 ns	1.05 ns	0.08	0.12	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0022624489707959 < 12.701813462443234.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.840299985435685 (T) = (0 -1.1844835047783475) / Math.Sqrt((16.292517613443774 / (299)) + (0.07454169342601547 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9065805905910096 = (12.679201380868038 - 1.1844835047783475) / 12.679201380868038 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.AddOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0060193735811305 < 34.10312985536552.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.59330650728501 (T) = (0 -1.1865091649494572) / Math.Sqrt((116.46976721118538 / (299)) + (0.06924976234380173 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9632184651620007 = (32.25828313514708 - 1.1865091649494572) / 32.25828313514708 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.UnaryNegateOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1318773785890635 < 31.027625902435197.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.45156614555222 (T) = (0 -1.149598146611813) / Math.Sqrt((97.7395303178968 / (299)) + (0.008408344136209212 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9609496964082649 = (29.43890420496301 - 1.149598146611813) / 29.43890420496301 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.AddBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0557354548863345 < 34.22802700074242.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.57750035706735 (T) = (0 -1.1284530947443527) / Math.Sqrt((128.06427283963038 / (299)) + (0.007946498438696124 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9664027635865112 = (33.5876761069341 - 1.1284530947443527) / 33.5876761069341 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.SubtractionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0732849338760786 < 33.69614994489321.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.85360277955107 (T) = (0 -1.1455377816359753) / Math.Sqrt((116.46736789640333 / (299)) + (0.012156539855889823 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9645137342771476 = (32.28115887376321 - 1.1455377816359753) / 32.28115887376321 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.AllBitsSetBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1633291000913277 < 6.292653858773745.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 169.70192943109006 (T) = (0 -1.140134740094411) / Math.Sqrt((0.0770752552235925 / (299)) + (0.01266486719621311 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.811215900635641 = (6.039357890485874 - 1.140134740094411) / 6.039357890485874 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.MultiplyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9967726833054023 < 34.41745566387768.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 416.1457305431207 (T) = (0 -1.1109224090910965) / Math.Sqrt((2.2596444891166274 / (299)) + (0.004028054835899834 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9705360630538044 = (37.70447958532359 - 1.1109224090910965) / 37.70447958532359 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.XorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.042123350821531 < 12.710590940384959.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.37280885000038 (T) = (0 -1.1804240263027637) / Math.Sqrt((18.990382369993974 / (299)) + (0.017583370653374363 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9121994708792142 = (13.44438397038442 - 1.1804240263027637) / 13.44438397038442 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.SubtractBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.014319243403874 < 34.186234837672494.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.600593211119055 (T) = (0 -1.139294831972795) / Math.Sqrt((128.08340978960305 / (299)) + (0.018976539384028806 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9661269145719281 = (33.634220726424374 - 1.139294831972795) / 33.634220726424374 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1472266146266605 < 12.665446327046501.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.8613955177741 (T) = (0 -1.1159863723701615) / Math.Sqrt((15.993768645238811 / (299)) + (0.00537311477320508 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9119407505276548 = (12.673130637124428 - 1.1159863723701615) / 12.673130637124428 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.BitwiseOrBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0311581865552344 < 12.72488596932493.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.60689258125941 (T) = (0 -1.173591579298094) / Math.Sqrt((19.031355023954745 / (299)) + (0.012137222085235239 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9129856410430816 = (13.487332359468963 - 1.173591579298094) / 13.487332359468963 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.OnesComplementBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.622907479291253 < 10.811987157318452.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 44.09016561644735 (T) = (0 -1.215328254649883) / Math.Sqrt((7.355882901648393 / (299)) + (0.0681489179336521 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8576138596871978 = (8.535439277867768 - 1.215328254649883) / 8.535439277867768 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.MultiplyOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.16640422100869 < 34.299189921003844.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 835.7477758560349 (T) = (0 -1.125270698581238) / Math.Sqrt((0.4305552495600273 / (299)) + (0.007745578915115041 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9690462467363753 = (36.35328772564724 - 1.125270698581238) / 36.35328772564724 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.EqualityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.017534892034348215 < 2.9792918495512533.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 54.948744446947146 (T) = (0 -0.08924016147737894) / Math.Sqrt((2.1267019562167984 / (299)) + (0.05851412603435975 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9837420047931009 = (5.489001586094096 - 0.08924016147737894) / 5.489001586094096 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.NegateBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9940815687466537 < 31.12579567482411.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.48638523622576 (T) = (0 -1.1392221558608038) / Math.Sqrt((97.79690315079178 / (299)) + (0.00815451452957481 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9613249661868435 = (29.456267869461108 - 1.1392221558608038) / 29.456267869461108 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0102889019919383 < 10.632081149810517.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 46.18995585437535 (T) = (0 -1.166065404781679) / Math.Sqrt((7.326714811197396 / (299)) + (0.01531041012452959 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8627203666082242 = (8.494088860609791 - 1.166065404781679) / 8.494088860609791 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0099358978509103 < 12.641777804482743.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.50390097218929 (T) = (0 -1.1227012577776767) / Math.Sqrt((16.15074330053114 / (299)) + (0.00998030306591847 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.9114328419384263 = (12.67627055389032 - 1.1227012577776767) / 12.67627055389032 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.BitwiseAndBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0469125428280142 < 12.736689821882106.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.437263164606655 (T) = (0 -1.1806984451317704) / Math.Sqrt((19.101980341704333 / (299)) + (0.012052966132562697 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9123699925119424 = (13.473677327856885 - 1.1806984451317704) / 13.473677327856885 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

performanceautofiler · 2023-04-11T18:53:34Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
BitwiseAndBenchmark - Duration of single invocation	13.31 ns	1.00 ns	0.08	0.12	False
AddBenchmark - Duration of single invocation	15.58 ns	1.01 ns	0.06	0.12	False
OnesComplementOperatorBenchmark - Duration of single invocation	11.11 ns	1.09 ns	0.10	0.17	False
NegateBenchmark - Duration of single invocation	14.68 ns	1.15 ns	0.08	0.11	False
BitwiseOrOperatorBenchmark - Duration of single invocation	13.28 ns	1.02 ns	0.08	0.13	False
OnesComplementBenchmark - Duration of single invocation	11.14 ns	1.10 ns	0.10	0.17	False
XorBenchmark - Duration of single invocation	13.20 ns	1.05 ns	0.08	0.13	False
AddOperatorBenchmark - Duration of single invocation	15.57 ns	0.99 ns	0.06	0.15	False
BitwiseAndOperatorBenchmark - Duration of single invocation	13.15 ns	1.25 ns	0.09	0.14	False
AllBitsSetBenchmark - Duration of single invocation	6.49 ns	1.26 ns	0.19	0.16	True
EqualityOperatorBenchmark - Duration of single invocation	3.03 ns	0.03 ns	0.01	0.43	False
UnaryNegateOperatorBenchmark - Duration of single invocation	14.39 ns	1.08 ns	0.07	0.13	False
MultiplyBenchmark - Duration of single invocation	15.75 ns	3.19 ns	0.20	0.09	True
SubtractBenchmark - Duration of single invocation	15.67 ns	1.03 ns	0.07	0.13	False
MultiplyOperatorBenchmark - Duration of single invocation	15.69 ns	0.92 ns	0.06	0.05	True
BitwiseOrBenchmark - Duration of single invocation	13.10 ns	1.05 ns	0.08	0.12	False
SubtractionOperatorBenchmark - Duration of single invocation	15.67 ns	0.96 ns	0.06	0.12	False
ExclusiveOrOperatorBenchmark - Duration of single invocation	13.20 ns	1.00 ns	0.08	0.14	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.BitwiseAndBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9993321697891718 < 12.734299246676402.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.12708949234093 (T) = (0 -1.1970812818880285) / Math.Sqrt((19.209245465522393 / (299)) + (0.07822716787679257 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.9111186367848185 = (13.468304699489122 - 1.1970812818880285) / 13.468304699489122 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.AddBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.01175853000589 < 15.025499154795007.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.88190697092742 (T) = (0 -1.1472845962621234) / Math.Sqrt((33.70605312224219 / (299)) + (0.012664607484078267 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.933540589837232 = (17.262936782801265 - 1.1472845962621234) / 17.262936782801265 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0920380843451938 < 10.69507969409962.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 46.49983180243531 (T) = (0 -1.1742140317996748) / Math.Sqrt((7.296417509608775 / (299)) + (0.016008906401059193 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.8625948684420083 = (8.545634493309292 - 1.1742140317996748) / 8.545634493309292 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.NegateBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1521036986029114 < 13.970106540281662.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.9513358320361 (T) = (0 -1.149943561436605) / Math.Sqrt((18.957788859754267 / (299)) + (0.008638701776184447 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9148972667464712 = (13.512416317003806 - 1.149943561436605) / 13.512416317003806 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0190826085566331 < 12.697326235979592.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.52871935030455 (T) = (0 -1.1427982471704343) / Math.Sqrt((16.09088898896759 / (299)) + (0.011873241130417916 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9099275842157871 = (12.687549647921562 - 1.1427982471704343) / 12.687549647921562 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.OnesComplementBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1002342344225773 < 10.795970726213964.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 46.77164568436881 (T) = (0 -1.1450862532849224) / Math.Sqrt((7.350821760517005 / (299)) + (0.009770812712123406 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.8659870426959615 = (8.544593570060824 - 1.1450862532849224) / 8.544593570060824 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.XorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0521621671103996 < 12.735492522555498.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.854981926548426 (T) = (0 -1.1035880011236547) / Math.Sqrt((19.109411142810185 / (299)) + (0.006004512195044406 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9181292660842041 = (13.479639772848433 - 1.1035880011236547) / 13.479639772848433 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.AddOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9921779594714917 < 15.028637986239962.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.30976858983691 (T) = (0 -1.2071276836912723) / Math.Sqrt((27.018823355705546 / (299)) + (0.08245828239872866 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9246233556998409 = (16.014611620070813 - 1.2071276836912723) / 16.014611620070813 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2478190031411343 < 12.638921008847655.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.464758390150585 (T) = (0 -1.1847009817161578) / Math.Sqrt((16.067181163074853 / (299)) + (0.00868530019159381 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9066532492775226 = (12.691400317064145 - 1.1847009817161578) / 12.691400317064145 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.AllBitsSetBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2646000897946947 < 6.418441200189941.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 170.01358995680894 (T) = (0 -1.143389582906034) / Math.Sqrt((0.06233955733322748 / (299)) + (0.014526533979398874 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8116665654124023 = (6.071091866453595 - 1.143389582906034) / 6.071091866453595 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.EqualityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.026841566919768817 < 2.890304417941816.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 60.57155476788081 (T) = (0 -0.06435889683434044) / Math.Sqrt((2.07296212628873 / (299)) + (0.00868717862522339 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9877256890744365 = (5.243381663104314 - 0.06435889683434044) / 5.243381663104314 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.UnaryNegateOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.07562360757354 < 13.927157839570755.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.92105875114655 (T) = (0 -1.1666783915988974) / Math.Sqrt((18.985215833910825 / (299)) + (0.011764462252234577 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9138571575721627 = (13.543532564254955 - 1.1666783915988974) / 13.543532564254955 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.MultiplyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.18768927655051 < 15.134202387519693.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 124.3203716814545 (T) = (0 -1.2097421622656788) / Math.Sqrt((3.734803328889339 / (299)) + (0.19257496330487048 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9368839851715809 = (19.16696048624688 - 1.2097421622656788) / 19.16696048624688 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.SubtractBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0316877762762517 < 15.18654232131746.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.756085796875645 (T) = (0 -1.1633273625654084) / Math.Sqrt((33.93420695512169 / (299)) + (0.014052112153477188 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9327359738906451 = (17.29494099378055 - 1.1633273625654084) / 17.29494099378055 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.MultiplyOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9246677665453518 < 15.121410756835875.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 154.04833358359627 (T) = (0 -1.2131517913063727) / Math.Sqrt((1.1907876488321278 / (299)) + (0.17956097733957666 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9323760794611503 = (17.939684384454182 - 1.2131517913063727) / 17.939684384454182 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.BitwiseOrBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.050989957937357 < 12.752066244252271.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.91652660519649 (T) = (0 -1.1099080706036584) / Math.Sqrt((19.00829075991871 / (299)) + (0.008890592962720777 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9176686888917466 = (13.480995937794493 - 1.1099080706036584) / 13.480995937794493 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.SubtractionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9610116897009814 < 14.973596028001474.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.08814657995848 (T) = (0 -1.1238443974899937) / Math.Sqrt((27.303945932925853 / (299)) + (0.011971036121475993 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9297593025431743 = (15.999903733597996 - 1.1238443974899937) / 15.999903733597996 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.996437953806821 < 12.775630505911163.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.846570383474536 (T) = (0 -1.130713906140512) / Math.Sqrt((15.900713034391009 / (299)) + (0.009331325524349423 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9107527397476556 = (12.669452294036216 - 1.130713906140512) / 12.669452294036216 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-11T18:53:37Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
SubtractBenchmark - Duration of single invocation	14.10 ns	1.04 ns	0.07	0.11	False
AllBitsSetBenchmark - Duration of single invocation	6.58 ns	1.18 ns	0.18	0.17	True
SubtractionOperatorBenchmark - Duration of single invocation	13.70 ns	1.16 ns	0.08	0.11	False
BitwiseAndBenchmark - Duration of single invocation	13.24 ns	1.33 ns	0.10	0.12	False
OnesComplementBenchmark - Duration of single invocation	11.13 ns	1.06 ns	0.10	0.16	False
BitwiseOrOperatorBenchmark - Duration of single invocation	13.22 ns	1.10 ns	0.08	0.12	False
BitwiseAndOperatorBenchmark - Duration of single invocation	15.47 ns	1.31 ns	0.08	0.14	False
AddBenchmark - Duration of single invocation	13.78 ns	1.21 ns	0.09	0.11	False
OnesComplementOperatorBenchmark - Duration of single invocation	11.16 ns	1.02 ns	0.09	0.16	False
AddOperatorBenchmark - Duration of single invocation	13.83 ns	1.26 ns	0.09	0.13	False
BitwiseOrBenchmark - Duration of single invocation	13.09 ns	1.26 ns	0.10	0.13	False
InequalityOperatorBenchmark - Duration of single invocation	3.24 ns	0.00 ns	0.00	0.38	False
NegateBenchmark - Duration of single invocation	12.57 ns	1.01 ns	0.08	0.13	False
EqualityOperatorBenchmark - Duration of single invocation	3.01 ns	0.24 ns	0.08	0.41	False
XorBenchmark - Duration of single invocation	13.12 ns	1.28 ns	0.10	0.11	False
UnaryNegateOperatorBenchmark - Duration of single invocation	12.51 ns	1.14 ns	0.09	0.15	False
ExclusiveOrOperatorBenchmark - Duration of single invocation	13.18 ns	1.34 ns	0.10	0.13	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.SubtractBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0426979544151795 < 13.194701985836678.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 46.65686518948601 (T) = (0 -1.2465395615004753) / Math.Sqrt((25.478043264926555 / (299)) + (0.03882633490960453 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9168992633779195 = (15.000343103688687 - 1.2465395615004753) / 15.000343103688687 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.AllBitsSetBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1794092599863826 < 6.350584648534857.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 139.63197529971896 (T) = (0 -1.183275438520373) / Math.Sqrt((0.07151031241512737 / (299)) + (0.02243161125552628 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8043949510324907 = (6.049309282997696 - 1.183275438520373) / 6.049309282997696 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.SubtractionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1623854856396547 < 13.156538619892638.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.073318433563784 (T) = (0 -1.1707162461832612) / Math.Sqrt((19.49997708810553 / (299)) + (0.011726633702597317 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9148681151164921 = (13.751795203233625 - 1.1707162461832612) / 13.751795203233625 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.BitwiseAndBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.327937869362846 < 12.717232535014835.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 46.503281328560035 (T) = (0 -1.2894830987958514) / Math.Sqrt((19.069470434992805 / (299)) + (0.1154008993393429 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9043887469829852 = (13.486729418412477 - 1.2894830987958514) / 13.486729418412477 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.OnesComplementBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.058389107314445 < 10.689630361950329.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 46.18591936008059 (T) = (0 -1.1583731659773069) / Math.Sqrt((7.475637567716191 / (299)) + (0.011472594787516372 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8642607124973124 = (8.53380909307021 - 1.1583731659773069) / 8.53380909307021 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.100342455822463 < 12.604369797400786.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.32626304055191 (T) = (0 -1.2945192252951403) / Math.Sqrt((16.030985802206434 / (299)) + (0.09696643738082732 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8978731547509332 = (12.67560181789684 - 1.2945192252951403) / 12.67560181789684 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3073820150401652 < 12.727215962330684.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.88916323369804 (T) = (0 -1.172226405742053) / Math.Sqrt((16.203896345282704 / (299)) + (0.028517047571524377 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9075738813273214 = (12.682847906806744 - 1.172226405742053) / 12.682847906806744 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.AddBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2139295499745428 < 13.151701109349242.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.66480956610026 (T) = (0 -1.1378356286363605) / Math.Sqrt((25.371670562040634 / (299)) + (0.01143576995100295 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9244623774832746 = (15.063164430207252 - 1.1378356286363605) / 15.063164430207252 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0192760036717974 < 10.756394019719682.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 46.480644513106085 (T) = (0 -1.156714677997176) / Math.Sqrt((7.338631756763272 / (299)) + (0.015302274468929828 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8645000074523818 = (8.536640159523818 - 1.156714677997176) / 8.536640159523818 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.AddOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2607425593013195 < 13.177893199564375.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.69542752697276 (T) = (0 -1.1839553919964079) / Math.Sqrt((19.645338747754504 / (299)) + (0.08313152161272068 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9138399263877192 = (13.741346105670598 - 1.1839553919964079) / 13.741346105670598 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.BitwiseOrBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2552825849121727 < 12.688620857145562.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.59433734149957 (T) = (0 -1.1649267248409458) / Math.Sqrt((19.061030406932367 / (299)) + (0.011300108860125375 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9135908056414438 = (13.48151355291041 - 1.1649267248409458) / 13.48151355291041 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.InequalityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 3.0142557596120434.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 60.49853788389088 (T) = (0 -0.06150630071294972) / Math.Sqrt((2.0867998380830466 / (299)) + (0.010668037315473045 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9883531043730688 = (5.280917995927484 - 0.06150630071294972) / 5.280917995927484 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.NegateBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.008499427953375 < 12.200240462572333.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.18225333753618 (T) = (0 -1.1943895189038778) / Math.Sqrt((12.149120538531719 / (299)) + (0.02261998615788988 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8916512743659005 = (11.023567761538855 - 1.1943895189038778) / 11.023567761538855 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.EqualityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.24145343580470163 < 2.8804413224669596.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 60.96027127226264 (T) = (0 -0.04574682835839846) / Math.Sqrt((2.116215156546452 / (299)) + (0.006547107208638495 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9913298711618083 = (5.276372382943635 - 0.04574682835839846) / 5.276372382943635 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.XorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2785037194286992 < 12.699459842611233.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.30450912903664 (T) = (0 -1.1570780116254527) / Math.Sqrt((19.30871780346404 / (299)) + (0.01539690385180379 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9142634906946965 = (13.49574435676119 - 1.1570780116254527) / 13.49574435676119 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.UnaryNegateOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1415479096877115 < 12.131025700419793.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 46.48088044790336 (T) = (0 -1.2527202043869916) / Math.Sqrt((12.36758837100962 / (299)) + (0.06951049634257774 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8865827520373163 = (11.045235419564744 - 1.2527202043869916) / 11.045235419564744 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3419837629583091 < 12.74601968894984.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.26070366494885 (T) = (0 -1.16372076271833) / Math.Sqrt((16.06645746277327 / (299)) + (0.01971944074294859 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.908175978112545 = (12.673380437906072 - 1.16372076271833) / 12.673380437906072 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-11T18:53:38Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
BitwiseAndBenchmark - Duration of single invocation	13.12 ns	0.97 ns	0.07	0.14	False
OnesComplementBenchmark - Duration of single invocation	11.07 ns	1.12 ns	0.10	0.16	False
BitwiseOrBenchmark - Duration of single invocation	13.40 ns	1.22 ns	0.09	0.16	False
BitwiseOrOperatorBenchmark - Duration of single invocation	13.22 ns	1.17 ns	0.09	0.15	False
UnaryNegateOperatorBenchmark - Duration of single invocation	15.53 ns	1.07 ns	0.07	0.10	False
BitwiseAndOperatorBenchmark - Duration of single invocation	13.21 ns	1.26 ns	0.10	0.10	False
SubtractionOperatorBenchmark - Duration of single invocation	17.43 ns	1.09 ns	0.06	0.09	False
ExclusiveOrOperatorBenchmark - Duration of single invocation	13.07 ns	1.01 ns	0.08	0.12	False
DivisionOperatorBenchmark - Duration of single invocation	19.24 ns	1.01 ns	0.05	0.08	True
OnesComplementOperatorBenchmark - Duration of single invocation	11.14 ns	0.92 ns	0.08	0.15	False
AddOperatorBenchmark - Duration of single invocation	17.12 ns	1.24 ns	0.07	0.09	False
XorBenchmark - Duration of single invocation	13.65 ns	1.22 ns	0.09	0.11	False
MultiplyBenchmark - Duration of single invocation	17.60 ns	1.29 ns	0.07	0.06	True
DivideBenchmark - Duration of single invocation	19.00 ns	1.12 ns	0.06	0.06	True
AddBenchmark - Duration of single invocation	17.49 ns	1.10 ns	0.06	0.12	False
NegateBenchmark - Duration of single invocation	15.58 ns	1.15 ns	0.07	0.11	False
InequalityOperatorBenchmark - Duration of single invocation	3.79 ns	0.05 ns	0.01	0.23	False
AllBitsSetBenchmark - Duration of single invocation	6.53 ns	1.07 ns	0.16	0.18	True
MultiplyOperatorBenchmark - Duration of single invocation	17.70 ns	1.28 ns	0.07	0.08	True
EqualityOperatorBenchmark - Duration of single invocation	3.86 ns	0.00 ns	0.00	0.28	False
SubtractBenchmark - Duration of single invocation	17.39 ns	2.32 ns	0.13	0.13	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.BitwiseAndBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9729885026372219 < 12.65802811259805.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.32651493400949 (T) = (0 -1.2407474861057903) / Math.Sqrt((19.194905551391237 / (299)) + (0.05785734671754705 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.9079227848171996 = (13.475076148236466 - 1.2407474861057903) / 13.475076148236466 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.OnesComplementBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1231144242211513 < 10.78516749473044.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.97486130683023 (T) = (0 -1.201698790285761) / Math.Sqrt((7.177766854409304 / (299)) + (0.032554185876554766 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8591532355327159 = (8.531958791036988 - 1.201698790285761) / 8.531958791036988 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.BitwiseOrBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2218597044005948 < 12.722411126862218.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.221719465902424 (T) = (0 -1.2301708641318907) / Math.Sqrt((18.937564733099762 / (299)) + (0.023180416384141433 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9086201445465162 = (13.462166885983965 - 1.2301708641318907) / 13.462166885983965 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1738851884037944 < 12.71271369496759.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.600940036636736 (T) = (0 -1.1489410781810991) / Math.Sqrt((16.020988834838764 / (299)) + (0.012254079018403486 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9094422933851838 = (12.687391511228045 - 1.1489410781810991) / 12.687391511228045 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.UnaryNegateOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.070264951246614 < 15.0211082477955.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.200830896971425 (T) = (0 -1.1636387455971422) / Math.Sqrt((19.292089664204486 / (299)) + (0.00936276395485833 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9150665666539912 = (13.700597041174758 - 1.1636387455971422) / 13.700597041174758 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2595305310802154 < 12.67931242160061.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.11499677311157 (T) = (0 -1.1827360580663875) / Math.Sqrt((16.175617745858798 / (299)) + (0.013368995044751131 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9066337162909975 = (12.66769984925883 - 1.1827360580663875) / 12.66769984925883 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.SubtractionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.086209294709002 < 16.64024808868696.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.57503529276981 (T) = (0 -1.1616274679891545) / Math.Sqrt((29.072396678302972 / (299)) + (0.007913206882862834 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9302219859125939 = (16.64747102911331 - 1.1616274679891545) / 16.64747102911331 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0129075841759334 < 12.796097024914062.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.08975736267786 (T) = (0 -1.1508248934755219) / Math.Sqrt((16.17759666843246 / (299)) + (0.02038825366798439 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9091160004114752 = (12.662568754520644 - 1.1508248934755219) / 12.662568754520644 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.DivisionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.014678580486834 < 18.283521239181468.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 399.4092589806953 (T) = (0 -1.1958355048114861) / Math.Sqrt((0.47802010587075044 / (299)) + (0.015798103210136735 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9410651018621702 = (20.290787675831936 - 1.1958355048114861) / 20.290787675831936 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9217106988553415 < 10.581337006921796.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.02080821207384 (T) = (0 -1.1765006072857809) / Math.Sqrt((7.189187250302381 / (299)) + (0.011498096521419077 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8622853315946623 = (8.543030462252341 - 1.1765006072857809) / 8.543030462252341 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.AddOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2395890196008665 < 16.47531445373708.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.329119937096976 (T) = (0 -1.1654460169642396) / Math.Sqrt((29.300837830846422 / (299)) + (0.011011802720265154 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9299833658928822 = (16.64527339576528 - 1.1654460169642396) / 16.64527339576528 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.XorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2226156316748156 < 12.779609331022474.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.45434190698786 (T) = (0 -1.1766416497654584) / Math.Sqrt((19.136457903610836 / (299)) + (0.00856591465703581 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9126504139262704 = (13.470489130564244 - 1.1766416497654584) / 13.470489130564244 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.MultiplyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2907083810653313 < 16.89360781459856.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 230.9855947162148 (T) = (0 -1.1488809543479659) / Math.Sqrt((1.8152702270293215 / (299)) + (0.013040494097432818 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9424648960879093 = (19.96834760398399 - 1.1488809543479659) / 19.96834760398399 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.DivideBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1184140596893544 < 18.423515283498542.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 252.66064378810245 (T) = (0 -1.134031097612213) / Math.Sqrt((1.7705445413499243 / (299)) + (0.009559974072034478 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.946627491994559 = (21.24747627555003 - 1.134031097612213) / 21.24747627555003 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.AddBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1041976559290456 < 16.58150742517983.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.55024115497712 (T) = (0 -1.250996795763297) / Math.Sqrt((34.37897976880058 / (299)) + (0.09231364674686193 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9291366861037055 = (17.653659234651055 - 1.250996795763297) / 17.653659234651055 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.NegateBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1451940489441366 < 14.994229231064319.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.99060925465648 (T) = (0 -1.2371199470684635) / Math.Sqrt((19.41798200841343 / (299)) + (0.05314396295748141 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9095849454800067 = (13.682676559077892 - 1.2371199470684635) / 13.682676559077892 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.InequalityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.04722085432218112 < 3.6042257941348903.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 63.22436846468929 (T) = (0 -0.08015589498595728) / Math.Sqrt((2.7000614440707826 / (299)) + (0.00993227713100117 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.987134328831977 = (6.23021480489727 - 0.08015589498595728) / 6.23021480489727 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.AllBitsSetBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0706634722024289 < 6.428726604459288.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 218.37006818593918 (T) = (0 -1.1434632268582368) / Math.Sqrt((0.06621176064790855 / (299)) + (0.006539830780031905 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8111392329341836 = (6.054530247988185 - 1.1434632268582368) / 6.054530247988185 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.MultiplyOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2837900225168803 < 16.997584305798.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 316.373806187503 (T) = (0 -1.2167452023979803) / Math.Sqrt((0.5096698202583623 / (299)) + (0.03300058187055803 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9357689293724727 = (18.94324959105575 - 1.2167452023979803) / 18.94324959105575 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.EqualityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 3.580475817545534.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 64.365303621621 (T) = (0 -0.043951627008119144) / Math.Sqrt((2.735810072169402 / (299)) + (0.004184194398754577 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9929808972970804 = (6.261715901355479 - 0.043951627008119144) / 6.261715901355479 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.SubtractBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.317291811188187 < 16.558290445215427.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.254922038393005 (T) = (0 -1.1976711876203) / Math.Sqrt((33.91199706028163 / (299)) + (0.07189716593729742 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9322246775260764 = (17.671198659085718 - 1.1976711876203) / 17.671198659085718 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-11T18:53:39Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
SubtractionOperatorBenchmark - Duration of single invocation	35.81 ns	1.05 ns	0.03	0.12	False
XorBenchmark - Duration of single invocation	13.24 ns	0.99 ns	0.07	0.11	False
BitwiseAndOperatorBenchmark - Duration of single invocation	13.17 ns	1.00 ns	0.08	0.13	False
BitwiseAndBenchmark - Duration of single invocation	13.09 ns	1.05 ns	0.08	0.13	False
AddOperatorBenchmark - Duration of single invocation	35.24 ns	1.02 ns	0.03	0.13	False
NegateBenchmark - Duration of single invocation	32.94 ns	1.09 ns	0.03	0.10	False
OnesComplementOperatorBenchmark - Duration of single invocation	11.10 ns	0.86 ns	0.08	0.12	False
AddBenchmark - Duration of single invocation	35.87 ns	0.87 ns	0.02	0.12	False
OnesComplementBenchmark - Duration of single invocation	11.15 ns	1.05 ns	0.09	0.13	False
EqualityOperatorBenchmark - Duration of single invocation	3.10 ns	0.07 ns	0.02	0.28	False
MultiplyOperatorBenchmark - Duration of single invocation	35.39 ns	1.05 ns	0.03	0.05	True
InequalityOperatorBenchmark - Duration of single invocation	3.25 ns	0.00 ns	0.00	0.26	False
MultiplyBenchmark - Duration of single invocation	36.13 ns	1.04 ns	0.03	0.06	True
BitwiseOrBenchmark - Duration of single invocation	13.49 ns	0.98 ns	0.07	0.12	False
SubtractBenchmark - Duration of single invocation	36.42 ns	1.09 ns	0.03	0.11	False
BitwiseOrOperatorBenchmark - Duration of single invocation	13.11 ns	1.06 ns	0.08	0.17	False
AllBitsSetBenchmark - Duration of single invocation	6.60 ns	1.01 ns	0.15	0.14	True
UnaryNegateOperatorBenchmark - Duration of single invocation	32.52 ns	1.03 ns	0.03	0.08	False
ExclusiveOrOperatorBenchmark - Duration of single invocation	15.20 ns	1.11 ns	0.07	0.13	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.SubtractionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0492346790656888 < 34.184408764661754.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.701267726729725 (T) = (0 -1.1822902657516956) / Math.Sqrt((116.6798519039879 / (299)) + (0.01797296794367488 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9633524361878512 = (32.26108758039086 - 1.1822902657516956) / 32.26108758039086 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.XorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9877964276330589 < 12.765195167770203.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.59063881710688 (T) = (0 -1.1367561316419708) / Math.Sqrt((19.181233025943822 / (299)) + (0.009526541195579108 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9156928633825375 = (13.483510142206802 - 1.1367561316419708) / 13.483510142206802 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0021513187935298 < 12.721034381813922.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.25665536163871 (T) = (0 -1.2042625299152714) / Math.Sqrt((16.111980354617437 / (299)) + (0.05987427989156003 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9049791620051761 = (12.673667748340337 - 1.2042625299152714) / 12.673667748340337 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.BitwiseAndBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0456739761030514 < 12.709015840558724.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.819846377226504 (T) = (0 -1.1282364429434464) / Math.Sqrt((18.968921380012045 / (299)) + (0.013711949805995487 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9163178052492524 = (13.482395464219922 - 1.1282364429434464) / 13.482395464219922 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.AddOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0180944361613646 < 33.88609211970769.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.65111702915077 (T) = (0 -1.1703563124905418) / Math.Sqrt((116.56912533736943 / (299)) + (0.07422354803022112 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9637661465680815 = (32.300078562981945 - 1.1703563124905418) / 32.300078562981945 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.NegateBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0853432672487031 < 31.10792510652953.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.97402399523145 (T) = (0 -1.248537901944077) / Math.Sqrt((97.65828936730834 / (299)) + (0.10996330517452352 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.957592548585964 = (29.441474559606185 - 1.248537901944077) / 29.441474559606185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.8594402457700292 < 10.660423988515415.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.88804140514971 (T) = (0 -1.1653020292872818) / Math.Sqrt((7.381154184013548 / (299)) + (0.02709469413296633 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8636303706357967 = (8.545172665792775 - 1.1653020292872818) / 8.545172665792775 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.AddBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.866333634594738 < 34.06474441339136.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.53688388018699 (T) = (0 -1.141140046939638) / Math.Sqrt((128.1689718598458 / (299)) + (0.018247562936123063 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9660414877778289 = (33.603947059689034 - 1.141140046939638) / 33.603947059689034 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.OnesComplementBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.053606301121875 < 10.764752096755572.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 46.59524987036329 (T) = (0 -1.1723587010889336) / Math.Sqrt((7.251536117034407 / (299)) + (0.016376244819932648 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8626330359281835 = (8.53450252038777 - 1.1723587010889336) / 8.53450252038777 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.EqualityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.06762467836063357 < 3.0087103950301204.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 61.44521122885057 (T) = (0 -0.08259834737797851) / Math.Sqrt((2.187723249525882 / (299)) + (0.007882768780222179 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9848726671788923 = (5.460205599676283 - 0.08259834737797851) / 5.460205599676283 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.MultiplyOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0543938246374893 < 33.74370013264642.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 750.4944385130398 (T) = (0 -1.1576582937120428) / Math.Sqrt((0.4751967348521321 / (299)) + (0.014002168970654049 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9681468259647273 = (36.343577328592445 - 1.1576582937120428) / 36.343577328592445 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.InequalityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.0008151865349065017 < 3.038454635560563.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 61.81752792891952 (T) = (0 -0.06832174805878707) / Math.Sqrt((2.1930542116560647 / (299)) + (0.009542077472038108 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9876008508952349 = (5.510196504736814 - 0.06832174805878707) / 5.510196504736814 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.MultiplyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.041544218632171 < 34.22497328621521.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 292.35706445135577 (T) = (0 -1.2371634852342834) / Math.Sqrt((2.2408925331199465 / (299)) + (0.1860569243756327 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9672134597094758 = (37.73388330307732 - 1.2371634852342834) / 37.73388330307732 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.BitwiseOrBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.975116571953757 < 12.683203602760662.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.113685964708 (T) = (0 -1.1780957180301712) / Math.Sqrt((19.093510997330615 / (299)) + (0.10081597520501058 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9126405429316566 = (13.485611719272931 - 1.1780957180301712) / 13.485611719272931 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.SubtractBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.087787022544401 < 34.258402920207374.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.50818141869075 (T) = (0 -1.1650105119395868) / Math.Sqrt((128.1971345511906 / (299)) + (0.012552346375574248 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.965330382321514 = (33.60321197491963 - 1.1650105119395868) / 33.60321197491963 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0640803996105122 < 12.655178238555314.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.20350874807529 (T) = (0 -1.15800570385832) / Math.Sqrt((16.129763697475244 / (299)) + (0.01441704981289387 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.9084965372343543 = (12.655321108712016 - 1.15800570385832) / 12.655321108712016 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.AllBitsSetBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.014480870203742 < 6.396484465865878.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 152.68958889602524 (T) = (0 -1.16638739223017) / Math.Sqrt((0.05579402073396114 / (299)) + (0.01848153354692794 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.8074894858104393 = (6.058824356375957 - 1.16638739223017) / 6.058824356375957 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.UnaryNegateOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0345533047718205 < 31.112084722105113.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.38325950872839 (T) = (0 -1.1840969489385562) / Math.Sqrt((97.65466212983556 / (299)) + (0.014481002220241174 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.959770395560037 = (29.43347232522888 - 1.1840969489385562) / 29.43347232522888 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1058196125158373 < 12.749837902575834.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.3093758975007 (T) = (0 -1.1382575004425852) / Math.Sqrt((16.229035732959392 / (299)) + (0.011312840466557318 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9102184747346377 = (12.678081566093914 - 1.1382575004425852) / 12.678081566093914 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-11T18:53:40Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt16>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
SubtractionOperatorBenchmark - Duration of single invocation	22.69 ns	1.22 ns	0.05	0.12	False
BitwiseOrOperatorBenchmark - Duration of single invocation	9.54 ns	0.98 ns	0.10	0.15	False
SubtractBenchmark - Duration of single invocation	22.67 ns	1.30 ns	0.06	0.12	False
AndNotBenchmark - Duration of single invocation	20.25 ns	1.07 ns	0.05	0.15	False
ExclusiveOrOperatorBenchmark - Duration of single invocation	9.49 ns	0.95 ns	0.10	0.15	False
MultiplyBenchmark - Duration of single invocation	23.60 ns	1.14 ns	0.05	0.05	True
UnaryNegateOperatorBenchmark - Duration of single invocation	22.65 ns	1.01 ns	0.04	0.11	False
OnesComplementOperatorBenchmark - Duration of single invocation	8.91 ns	1.10 ns	0.12	0.15	False
MultiplyOperatorBenchmark - Duration of single invocation	23.41 ns	1.31 ns	0.06	0.05	True
AddOperatorBenchmark - Duration of single invocation	22.62 ns	1.25 ns	0.06	0.12	False
BitwiseAndOperatorBenchmark - Duration of single invocation	9.79 ns	1.57 ns	0.16	0.16	False
OnesComplementBenchmark - Duration of single invocation	8.78 ns	1.13 ns	0.13	0.14	False
NegateBenchmark - Duration of single invocation	22.68 ns	1.03 ns	0.05	0.12	False
AddBenchmark - Duration of single invocation	22.82 ns	2.75 ns	0.12	0.11	False
ConditionalSelectBenchmark - Duration of single invocation	32.79 ns	1.20 ns	0.04	0.41	False
OneBenchmark - Duration of single invocation	12.34 ns	1.10 ns	0.09	0.21	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt16>.SubtractionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2232188138058504 < 21.630247294905256.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.608682411695675 (T) = (0 -1.2650850392433763) / Math.Sqrt((49.89463636493455 / (299)) + (0.023720936837456977 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9414115165796944 = (21.592725487837466 - 1.2650850392433763) / 21.592725487837466 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9771047876483431 < 9.07163352694982.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 42.87882395747947 (T) = (0 -1.2486329367020763) / Math.Sqrt((16.144012102806208 / (299)) + (0.02503227589475132 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8896194670654144 = (11.312075630600999 - 1.2486329367020763) / 11.312075630600999 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.SubtractBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.295631756113688 < 21.676428205182145.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.87723423986376 (T) = (0 -1.3122839600333756) / Math.Sqrt((57.10518168525548 / (299)) + (0.049175189815186114 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9424231903278643 = (22.791883876616655 - 1.3122839600333756) / 22.791883876616655 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.AndNotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.071778581491835 < 16.911995028923112.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.07807904482796 (T) = (0 -1.249717058727427) / Math.Sqrt((51.59951899339555 / (299)) + (0.022047754230452243 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9376014522616798 = (20.02798308653506 - 1.249717058727427) / 20.02798308653506 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9515607757019794 < 9.060747556578495.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 42.97949877041836 (T) = (0 -1.2498039219659833) / Math.Sqrt((16.103356963874106 / (299)) + (0.026804296146790033 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8897051226079318 = (11.331477503921331 - 1.2498039219659833) / 11.331477503921331 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.MultiplyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1426691966238487 < 22.4424615613354.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 202.81844953575757 (T) = (0 -1.3631958840075806) / Math.Sqrt((2.0786369596880525 / (299)) + (0.17471594072400032 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.9478003174181294 = (26.11502247871964 - 1.3631958840075806) / 26.11502247871964 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.UnaryNegateOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0097573430349243 < 21.52275095135952.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.15022884166462 (T) = (0 -1.1835061319414173) / Math.Sqrt((50.353506711077124 / (299)) + (0.024195297825597473 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9447382077544336 = (21.416354480185547 - 1.1835061319414173) / 21.416354480185547 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.097268822204067 < 8.616585719155353.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.59832621117554 (T) = (0 -1.1623955716869236) / Math.Sqrt((5.565581032064656 / (299)) + (0.018826147196340282 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8454020065319416 = (7.518827027513053 - 1.1623955716869236) / 7.518827027513053 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.MultiplyOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3127992213985924 < 22.37245622614425.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 496.1114278211036 (T) = (0 -1.2244265120231965) / Math.Sqrt((0.37431055839061333 / (299)) + (0.023225823694332834 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9506640258752695 = (24.81812782144771 - 1.2244265120231965) / 24.81812782144771 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.AddOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2528257762092696 < 21.61621284842872.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.11558347209156 (T) = (0 -1.3079736569101563) / Math.Sqrt((50.246030739792886 / (299)) + (0.05384976569327884 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.9394128114322743 = (21.58828768639883 - 1.3079736569101563) / 21.58828768639883 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.573203498477966 < 9.140504230725323.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 42.90722193104904 (T) = (0 -1.2514484147152836) / Math.Sqrt((16.135485486207635 / (299)) + (0.029311376164735746 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8896035274779602 = (11.335945670414814 - 1.2514484147152836) / 11.335945670414814 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.OnesComplementBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1327388320055163 < 8.614408802126556.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.32582428404166 (T) = (0 -1.2199369810360958) / Math.Sqrt((5.526775723287009 / (299)) + (0.01798249246630846 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8375855857255261 = (7.5112605398092995 - 1.2199369810360958) / 7.5112605398092995 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.NegateBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0320142078267176 < 21.53497702098007.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.04953603218536 (T) = (0 -1.193429071337644) / Math.Sqrt((53.4434874020244 / (299)) + (0.019371106492886463 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9457020480755848 = (21.97926494536924 - 1.193429071337644) / 21.97926494536924 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.AddBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.7533582026405536 < 21.679356757873173.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.23601042682327 (T) = (0 -1.3377860904399388) / Math.Sqrt((57.376819883211866 / (299)) + (0.12894407917210424 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9412593346139922 = (22.77444563572484 - 1.3377860904399388) / 22.77444563572484 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.ConditionalSelectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2030029417113695 < 31.128962573209023.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 41.36719238862814 (T) = (0 -1.1743441556286516) / Math.Sqrt((363.7275933897926 / (299)) + (0.02367275070858895 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9749174880982817 = (46.81924044250933 - 1.1743441556286516) / 46.81924044250933 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt16>.OneBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1034465523908152 < 11.68713949843272.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 31.991877401723208 (T) = (0 -1.2472653248018295) / Math.Sqrt((32.84591273216309 / (299)) + (0.010244302239615154 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8949417505501267 = (11.872131235129153 - 1.2472653248018295) / 11.872131235129153 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-11T18:53:41Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
AllBitsSetBenchmark - Duration of single invocation	6.56 ns	1.01 ns	0.15	0.17	True
MultiplyOperatorBenchmark - Duration of single invocation	54.58 ns	1.05 ns	0.02	0.06	True
BitwiseOrOperatorBenchmark - Duration of single invocation	13.08 ns	1.24 ns	0.09	0.13	False
SubtractBenchmark - Duration of single invocation	51.50 ns	1.08 ns	0.02	0.11	False
XorBenchmark - Duration of single invocation	13.22 ns	1.02 ns	0.08	0.14	False
UnaryNegateOperatorBenchmark - Duration of single invocation	48.78 ns	1.05 ns	0.02	0.11	False
MultiplyBenchmark - Duration of single invocation	53.87 ns	1.02 ns	0.02	0.04	True
BitwiseOrBenchmark - Duration of single invocation	13.25 ns	1.08 ns	0.08	0.13	False
OnesComplementOperatorBenchmark - Duration of single invocation	11.75 ns	1.10 ns	0.09	0.12	False
InequalityOperatorBenchmark - Duration of single invocation	3.24 ns	0.12 ns	0.04	0.32	False
BitwiseAndOperatorBenchmark - Duration of single invocation	13.11 ns	1.07 ns	0.08	0.15	False
ExclusiveOrOperatorBenchmark - Duration of single invocation	14.43 ns	1.07 ns	0.07	0.11	False
AddOperatorBenchmark - Duration of single invocation	52.26 ns	1.05 ns	0.02	0.13	False
OnesComplementBenchmark - Duration of single invocation	10.85 ns	1.03 ns	0.09	0.17	False
AddBenchmark - Duration of single invocation	52.38 ns	0.97 ns	0.02	0.11	False
NegateBenchmark - Duration of single invocation	48.61 ns	0.99 ns	0.02	0.10	False
SubtractionOperatorBenchmark - Duration of single invocation	52.98 ns	0.95 ns	0.02	0.13	False
BitwiseAndBenchmark - Duration of single invocation	13.08 ns	0.94 ns	0.07	0.12	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.AllBitsSetBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0095445239590723 < 6.33176333358847.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 192.346807625689 (T) = (0 -1.149058151440351) / Math.Sqrt((0.08223974767396725 / (299)) + (0.0085905574635462 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8099942941188158 = (6.047492869287248 - 1.149058151440351) / 6.047492869287248 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.MultiplyOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.047319803057948 < 51.494664382638085.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 375.06553614829227 (T) = (0 -1.2253649765211398) / Math.Sqrt((4.2743609290196245 / (299)) + (0.14397530889545496 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.977720224664152 = (54.998982622124444 - 1.2253649765211398) / 54.998982622124444 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2423680623421485 < 12.685863682322816.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.69611307467224 (T) = (0 -1.1799325629835922) / Math.Sqrt((15.976205338460115 / (299)) + (0.05216848592805189 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.906969536893482 = (12.683292370936524 - 1.1799325629835922) / 12.683292370936524 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.SubtractBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0791376490611073 < 49.56631956834375.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.53485164675762 (T) = (0 -1.1466691650254461) / Math.Sqrt((273.65169500665627 / (299)) + (0.04023909524934085 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9763965584226235 = (48.58059199826645 - 1.1466691650254461) / 48.58059199826645 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.XorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.020709124147049 < 12.733187620555876.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.63155365649772 (T) = (0 -1.1440738962076247) / Math.Sqrt((19.088582854246713 / (299)) + (0.007229143285317427 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9150142552827744 = (13.461950589646758 - 1.1440738962076247) / 13.461950589646758 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.UnaryNegateOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0528303012693607 < 46.36839402564059.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.50662978441437 (T) = (0 -1.141403489812712) / Math.Sqrt((228.04989700611847 / (299)) + (0.02893261166457189 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9743001026375844 = (44.41276452262941 - 1.141403489812712) / 44.41276452262941 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.MultiplyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0221937243027404 < 51.25034506592946.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 430.7241743159972 (T) = (0 -1.140155557620321) / Math.Sqrt((4.592502151329993 / (299)) + (0.010098895469319687 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.97937451988618 = (55.27898266263181 - 1.140155557620321) / 55.27898266263181 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.BitwiseOrBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0786443914991988 < 12.656309474380818.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.031615155521386 (T) = (0 -1.1863484447112127) / Math.Sqrt((19.16293265926422 / (299)) + (0.091117849972964 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9118307682540697 = (13.455356491364363 - 1.1863484447112127) / 13.455356491364363 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1009064519008673 < 10.821494720533375.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 46.64104108486708 (T) = (0 -1.1355910526566044) / Math.Sqrt((7.43927133593022 / (299)) + (0.008163296878311096 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.8671387935687153 = (8.547198111165184 - 1.1355910526566044) / 8.547198111165184 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.InequalityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.11890893940948669 < 3.034979854526752.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 60.28948916866736 (T) = (0 -0.08356185701256248) / Math.Sqrt((2.286347958677136 / (299)) + (0.011992595692232115 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9848958198615386 = (5.532366288440942 - 0.08356185701256248) / 5.532366288440942 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0657999644386877 < 12.580367419693737.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.05651861685467 (T) = (0 -1.1481842631701462) / Math.Sqrt((16.31665811720869 / (299)) + (0.010561829245866433 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9092781360264456 = (12.656092069546133 - 1.1481842631701462) / 12.656092069546133 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0741227396065207 < 12.770647358575532.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.31621217483518 (T) = (0 -1.1353827479169307) / Math.Sqrt((16.199890857175873 / (299)) + (0.014247808361772862 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9104586407840125 = (12.679981160194501 - 1.1353827479169307) / 12.679981160194501 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.AddOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0548946470769105 < 49.98814364685258.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.54622441126835 (T) = (0 -1.1386227849164527) / Math.Sqrt((271.35300279307984 / (299)) + (0.0073702906866982745 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9764489398894396 = (48.3469864868583 - 1.1386227849164527) / 48.3469864868583 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.OnesComplementBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.027914428676164 < 10.756980855849184.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 43.181298616862954 (T) = (0 -1.2165534779798917) / Math.Sqrt((7.37591647816435 / (299)) + (0.08901297865184533 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.8574428966479859 = (8.533797680890546 - 1.2165534779798917) / 8.533797680890546 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.AddBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9693183539020012 < 49.33769293040486.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.4705505720009 (T) = (0 -1.1591658387544306) / Math.Sqrt((274.6854191839709 / (299)) + (0.018421068445008296 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.976147021045805 = (48.596271391526464 - 1.1591658387544306) / 48.596271391526464 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.NegateBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9919693964639675 < 46.8899769263761.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.50313155405929 (T) = (0 -1.1822464903552825) / Math.Sqrt((227.95385178191748 / (299)) + (0.015595983456108619 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9733878264400796 = (44.42502555055564 - 1.1822464903552825) / 44.42502555055564 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.SubtractionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9526892083150006 < 49.87601142871293.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.532442249867024 (T) = (0 -1.1507825147527446) / Math.Sqrt((271.26888558981557 / (299)) + (0.06894629589908018 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9762275533739609 = (48.40824896383356 - 1.1507825147527446) / 48.40824896383356 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.BitwiseAndBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9376496436286214 < 12.757188432667721.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.447997821789905 (T) = (0 -1.1455994817091624) / Math.Sqrt((19.164298621820176 / (299)) + (0.013079737146739268 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9149229417873651 = (13.465433640711236 - 1.1455994817091624) / 13.465433640711236 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-11T18:53:42Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
EqualityOperatorBenchmark - Duration of single invocation	3.37 ns	0.00 ns	0.00	0.39	False
SubtractionOperatorBenchmark - Duration of single invocation	52.22 ns	1.11 ns	0.02	0.13	False
OnesComplementOperatorBenchmark - Duration of single invocation	11.16 ns	1.01 ns	0.09	0.17	False
AddBenchmark - Duration of single invocation	52.02 ns	1.28 ns	0.02	0.10	False
BitwiseOrBenchmark - Duration of single invocation	13.20 ns	1.03 ns	0.08	0.15	False
InequalityOperatorBenchmark - Duration of single invocation	3.28 ns	0.04 ns	0.01	0.38	False
OnesComplementBenchmark - Duration of single invocation	11.19 ns	1.28 ns	0.11	0.16	False
NegateBenchmark - Duration of single invocation	48.92 ns	1.23 ns	0.03	0.11	False
BitwiseOrOperatorBenchmark - Duration of single invocation	13.27 ns	1.13 ns	0.09	0.13	False
XorBenchmark - Duration of single invocation	13.17 ns	1.18 ns	0.09	0.14	False
UnaryNegateOperatorBenchmark - Duration of single invocation	48.75 ns	1.09 ns	0.02	0.11	False
AllBitsSetBenchmark - Duration of single invocation	6.54 ns	0.92 ns	0.14	0.17	True
MultiplyBenchmark - Duration of single invocation	54.02 ns	1.00 ns	0.02	0.04	True
BitwiseAndOperatorBenchmark - Duration of single invocation	13.16 ns	1.01 ns	0.08	0.13	False
SubtractBenchmark - Duration of single invocation	51.46 ns	1.27 ns	0.02	0.11	False
BitwiseAndBenchmark - Duration of single invocation	13.17 ns	1.06 ns	0.08	0.13	False
MultiplyOperatorBenchmark - Duration of single invocation	54.84 ns	0.99 ns	0.02	0.06	True
ExclusiveOrOperatorBenchmark - Duration of single invocation	13.14 ns	0.96 ns	0.07	0.12	False
AddOperatorBenchmark - Duration of single invocation	51.78 ns	1.19 ns	0.02	0.12	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.EqualityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 2.970590264188213.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 62.26096258697371 (T) = (0 -0.03001913598675415) / Math.Sqrt((2.252129481603066 / (299)) + (0.004362087300176503 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9945431129429303 = (5.501146656107347 - 0.03001913598675415) / 5.501146656107347 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.SubtractionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1109356053949484 < 49.584629086518355.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.415538745480575 (T) = (0 -1.1894138527734242) / Math.Sqrt((272.2498188882269 / (299)) + (0.034826642074078554 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9754161232292291 = (48.38186685785807 - 1.1894138527734242) / 48.38186685785807 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0121767085477964 < 10.774080580384299.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 46.45409913559951 (T) = (0 -1.176163988839476) / Math.Sqrt((7.2959683174300265 / (299)) + (0.014547453396560286 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.8621204756546963 = (8.530374574646096 - 1.176163988839476) / 8.530374574646096 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.AddBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2799236307873545 < 49.445510250254735.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.42772464992511 (T) = (0 -1.208586555034839) / Math.Sqrt((274.07654471451104 / (299)) + (0.02974167556423808 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9751139453236969 = (48.56481152818788 - 1.208586555034839) / 48.56481152818788 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.BitwiseOrBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0345637536819405 < 12.698823329944506.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.10305919231857 (T) = (0 -1.2560823567919015) / Math.Sqrt((19.212896956764546 / (299)) + (0.06909161290969493 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9067644964811755 = (13.472146439776516 - 1.2560823567919015) / 13.472146439776516 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.InequalityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.04085178329861341 < 3.104567366542147.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 61.10605939347357 (T) = (0 -0.06134496754334595) / Math.Sqrt((2.2711641420867954 / (299)) + (0.009458462294835737 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9889053994231076 = (5.529263277049743 - 0.06134496754334595) / 5.529263277049743 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.OnesComplementBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2775403682814694 < 10.729606784943838.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 46.28326714276443 (T) = (0 -1.2059177455596226) / Math.Sqrt((7.286629522207985 / (299)) + (0.015806674225848943 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.8587402898151928 = (8.536883899747105 - 1.2059177455596226) / 8.536883899747105 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.NegateBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2342689385066323 < 46.78929522642719.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.35713116637932 (T) = (0 -1.2526815516079102) / Math.Sqrt((228.04450812563186 / (299)) + (0.046275816901059814 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9717954398927158 = (44.414149585846 - 1.2526815516079102) / 44.414149585846 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1303152384100403 < 12.648402310273669.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.246757341507546 (T) = (0 -1.164855588498392) / Math.Sqrt((16.158066606018053 / (299)) + (0.0098975221533152 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.9079936343667634 = (12.660597780177916 - 1.164855588498392) / 12.660597780177916 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.XorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1810057597076746 < 12.683144509878447.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.05590748930666 (T) = (0 -1.2399115094120676) / Math.Sqrt((19.24830514105451 / (299)) + (0.07491490567271888 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9080007762541975 = (13.477412731633393 - 1.2399115094120676) / 13.477412731633393 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.UnaryNegateOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.085483441731322 < 46.745077637588444.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.51407317438172 (T) = (0 -1.1623021775360127) / Math.Sqrt((227.9045536045062 / (299)) + (0.013401115047869415 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9738263034517735 = (44.40726113693591 - 1.1623021775360127) / 44.40726113693591 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.AllBitsSetBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9156393509555869 < 6.386909657377067.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 153.2573797135237 (T) = (0 -1.1548563693642464) / Math.Sqrt((0.06618725228570985 / (299)) + (0.01841722647472356 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8092586359196072 = (6.054567004551269 - 1.1548563693642464) / 6.054567004551269 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.MultiplyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9997371884168837 < 51.33652767616804.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 358.03531856279255 (T) = (0 -1.2716494589506862) / Math.Sqrt((4.5490608640252015 / (299)) + (0.1725704735038767 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9769777524037422 = (55.23567816885903 - 1.2716494589506862) / 55.23567816885903 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0098639336451367 < 12.803306273528534.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.430034794062756 (T) = (0 -1.1672736900348315) / Math.Sqrt((16.0723521161219 / (299)) + (0.010861000392277541 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9079275736134719 = (12.677777004969034 - 1.1672736900348315) / 12.677777004969034 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.SubtractBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2695152599941177 < 49.43270174017141.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.41130899244305 (T) = (0 -1.2064010177152602) / Math.Sqrt((274.4877779420455 / (299)) + (0.012543330653028121 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9751580617186951 = (48.56307925952596 - 1.2064010177152602) / 48.56307925952596 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.BitwiseAndBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0585264023851575 < 12.710088429947763.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.36179527017481 (T) = (0 -1.158505480060848) / Math.Sqrt((19.044410072177556 / (299)) + (0.025154738236612052 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9139822122990696 = (13.468208274418535 - 1.158505480060848) / 13.468208274418535 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.MultiplyOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9883009157255475 < 51.19181424247223.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 422.5496969362631 (T) = (0 -1.1690759181245358) / Math.Sqrt((4.354876964850811 / (299)) + (0.03847755080251955 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.978749284969565 = (55.01348620270908 - 1.1690759181245358) / 55.01348620270908 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9591988604981613 < 12.710738539931896.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.840241950823206 (T) = (0 -1.1747099711549767) / Math.Sqrt((16.14184036708218 / (299)) + (0.03289399338028257 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9072989519063773 = (12.672024699964435 - 1.1747099711549767) / 12.672024699964435 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.AddOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1861151243935821 < 49.96488783029815.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.59343744658832 (T) = (0 -1.1499711144049756) / Math.Sqrt((271.2820334510703 / (299)) + (0.009775855863725973 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9762401998435165 = (48.39986476448449 - 1.1499711144049756) / 48.39986476448449 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-11T18:53:43Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
UnaryNegateOperatorBenchmark - Duration of single invocation	7.32 ns	0.97 ns	0.13	0.10	False
BitwiseOrOperatorBenchmark - Duration of single invocation	9.54 ns	1.02 ns	0.11	0.14	False
BitwiseAndOperatorBenchmark - Duration of single invocation	9.28 ns	1.06 ns	0.11	0.15	False
AddBenchmark - Duration of single invocation	7.20 ns	1.08 ns	0.15	0.13	False
OnesComplementOperatorBenchmark - Duration of single invocation	8.98 ns	1.03 ns	0.12	0.18	False
DivideBenchmark - Duration of single invocation	8.83 ns	1.08 ns	0.12	0.07	False
DivisionOperatorBenchmark - Duration of single invocation	9.28 ns	1.12 ns	0.12	0.09	True
MultiplyBenchmark - Duration of single invocation	7.79 ns	1.00 ns	0.13	0.11	False
ExclusiveOrOperatorBenchmark - Duration of single invocation	9.62 ns	0.99 ns	0.10	0.16	False
SubtractBenchmark - Duration of single invocation	7.18 ns	1.01 ns	0.14	0.15	False
AddOperatorBenchmark - Duration of single invocation	7.28 ns	0.99 ns	0.14	0.13	False
NegateBenchmark - Duration of single invocation	7.36 ns	1.00 ns	0.14	0.12	False
ConditionalSelectBenchmark - Duration of single invocation	32.72 ns	0.98 ns	0.03	0.39	False
MultiplyOperatorBenchmark - Duration of single invocation	7.58 ns	1.26 ns	0.17	0.09	True
OnesComplementBenchmark - Duration of single invocation	9.03 ns	1.10 ns	0.12	0.13	False
SubtractionOperatorBenchmark - Duration of single invocation	7.24 ns	1.02 ns	0.14	0.12	False
OneBenchmark - Duration of single invocation	6.34 ns	0.96 ns	0.15	0.46	False
AndNotBenchmark - Duration of single invocation	17.72 ns	0.83 ns	0.05	0.11	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Double>.UnaryNegateOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9680533149868615 < 6.975677025382067.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.635877310762176 (T) = (0 -1.1250049996726992) / Math.Sqrt((6.637030115800201 / (299)) + (0.006360869029128274 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.858781752820657 = (7.966428008725931 - 1.1250049996726992) / 7.966428008725931 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0187419712895922 < 9.007696677394357.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 43.48793209907502 (T) = (0 -1.120783836301258) / Math.Sqrt((16.2616414489828 / (299)) + (0.014433471202433646 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9009989953981135 = (11.320933972420528 - 1.120783836301258) / 11.320933972420528 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0553865593768879 < 9.059663715336125.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 42.62692140915176 (T) = (0 -1.2310053723316172) / Math.Sqrt((16.158729044794757 / (299)) + (0.04652909150149152 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8912950478409248 = (11.324280521555314 - 1.2310053723316172) / 11.324280521555314 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.AddBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0809222274441184 < 6.979850668280357.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 41.87244794856658 (T) = (0 -1.1923933690753663) / Math.Sqrt((11.337880115161328 / (299)) + (0.0586739013342307 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.8761540525689757 = (9.628037039641269 - 1.1923933690753663) / 9.628037039641269 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0348909827729003 < 8.6627013583695.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 44.015057526113765 (T) = (0 -1.1807849285313228) / Math.Sqrt((5.535907337543723 / (299)) + (0.05116656117016065 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8429696340028248 = (7.519468741176872 - 1.1807849285313228) / 7.519468741176872 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.DivideBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0767844534390072 < 8.456625311895236.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 83.2680247486111 (T) = (0 -1.0982124571715624) / Math.Sqrt((4.5107611952242195 / (299)) + (0.00871406134935306 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.904113858087514 = (11.453296954776704 - 1.0982124571715624) / 11.453296954776704 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.DivisionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1211442194138568 < 8.481340252856256.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 134.18487108298183 (T) = (0 -1.1606060059893886) / Math.Sqrt((1.1097153002740427 / (299)) + (0.012435378094365977 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8828917615307119 = (9.910541061496371 - 1.1606060059893886) / 9.910541061496371 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.MultiplyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9981480601502696 < 7.47429270951442.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 64.18377236416295 (T) = (0 -1.2593216807773786) / Math.Sqrt((4.073154650847233 / (299)) + (0.1635781889141983 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8800816422601808 = (10.501492052698602 - 1.2593216807773786) / 10.501492052698602 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9893319071921785 < 9.061704020387273.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 43.75894339893306 (T) = (0 -1.10765026414981) / Math.Sqrt((16.160923270168897 / (299)) + (0.009171282203960612 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.902137773982137 = (11.318465859826526 - 1.10765026414981) / 11.318465859826526 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.SubtractBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0128885822474871 < 7.017871365641554.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 42.69278245900281 (T) = (0 -1.1343362910794716) / Math.Sqrt((11.552678658124309 / (299)) + (0.01283835043278806 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8816752398490834 = (9.586635034228575 - 1.1343362910794716) / 9.586635034228575 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.AddOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9932320370305845 < 6.942434234093177.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 46.24390601200193 (T) = (0 -1.1504370243688762) / Math.Sqrt((6.4489930312484605 / (299)) + (0.0283861627495442 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8585562473765357 = (8.133530134988998 - 1.1504370243688762) / 8.133530134988998 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.NegateBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9985531637160239 < 7.0078828642609015.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 41.67168828826344 (T) = (0 -1.165493340385015) / Math.Sqrt((9.376011191091283 / (299)) + (0.062229925031785045 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8684032369226916 = (8.85655021545119 - 1.165493340385015) / 8.85655021545119 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.ConditionalSelectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9813774132392183 < 31.151574892048714.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 41.534918384254006 (T) = (0 -1.1099418406692927) / Math.Sqrt((362.0188761444369 / (299)) + (0.014145246500402435 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9762956564611488 = (46.82440746988451 - 1.1099418406692927) / 46.82440746988451 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.MultiplyOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.261074432162496 < 7.296299387436848.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 126.99159049143307 (T) = (0 -1.1495374353462047) / Math.Sqrt((1.0336772939899852 / (299)) + (0.008081461022164704 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8720825567582499 = (8.986557315515627 - 1.1495374353462047) / 8.986557315515627 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.OnesComplementBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1017034298563577 < 8.602404488304366.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 46.243585544897805 (T) = (0 -1.1533928353324394) / Math.Sqrt((5.5450201829062955 / (299)) + (0.008045565376051044 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8464190227323807 = (7.509998020930803 - 1.1533928353324394) / 7.509998020930803 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.SubtractionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.016190160533396 < 6.957173304825667.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 46.26386563062812 (T) = (0 -1.1446932437917006) / Math.Sqrt((6.537525875134344 / (299)) + (0.020922391724904784 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8591397497676615 = (8.126446189777562 - 1.1446932437917006) / 8.126446189777562 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.OneBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9562755959548337 < 5.915689552464527.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 29.921344644823773 (T) = (0 -1.2437013906199648) / Math.Sqrt((10.233652231557627 / (299)) + (0.02358608801279407 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8187438462387919 = (6.8615678133524325 - 1.2437013906199648) / 6.8615678133524325 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Double>.AndNotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.8347233391388879 < 16.77999793339872.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 44.32570700400329 (T) = (0 -1.1607182574955321) / Math.Sqrt((51.9639625233674 / (299)) + (0.17030162828206452 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.942047959338772 = (20.02894538745885 - 1.1607182574955321) / 20.02894538745885 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-11T18:53:45Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorConvert

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Convert_float_int - Duration of single invocation	25.41 μs	16.29 μs	0.64	0.01	False
Widen_short - Duration of single invocation	61.78 μs	34.25 μs	0.55	0.01	False
Narrow_short - Duration of single invocation	44.28 μs	35.36 μs	0.80	0.01	False
Narrow_long - Duration of single invocation	21.78 μs	13.39 μs	0.61	0.01	False
Widen_ushort - Duration of single invocation	61.12 μs	34.41 μs	0.56	0.11	False
Convert_long_double - Duration of single invocation	56.22 μs	49.71 μs	0.88	0.01	False
Narrow_ushort - Duration of single invocation	44.26 μs	35.39 μs	0.80	0.01	False
Widen_int - Duration of single invocation	53.27 μs	26.39 μs	0.50	0.01	False
Narrow_double - Duration of single invocation	23.07 μs	14.54 μs	0.63	0.01	False
Narrow_uint - Duration of single invocation	29.31 μs	20.56 μs	0.70	0.01	False
Widen_uint - Duration of single invocation	52.60 μs	25.67 μs	0.49	0.01	False
Convert_double_long - Duration of single invocation	18.24 μs	10.29 μs	0.56	0.01	False
Convert_ulong_double - Duration of single invocation	56.08 μs	49.48 μs	0.88	0.03	False
Widen_float - Duration of single invocation	64.09 μs	39.44 μs	0.62	0.01	False
Widen_byte - Duration of single invocation	72.90 μs	49.55 μs	0.68	0.01	False
Convert_int_float - Duration of single invocation	26.29 μs	17.44 μs	0.66	0.01	False
Widen_sbyte - Duration of single invocation	72.67 μs	49.68 μs	0.68	0.01	False
Narrow_ulong - Duration of single invocation	21.90 μs	13.30 μs	0.61	0.01	False
Narrow_int - Duration of single invocation	28.97 μs	20.31 μs	0.70	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorConvert* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorConvert* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorConvert.Convert_float_int

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.28550833118113 < 24.162744106800766.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.83489631327007 (T) = (0 -16198.545755714216) / Math.Sqrt((9027205.263891865 / (299)) + (6140.202000127072 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.34934840769375664 = (24895.882753930182 - 16198.545755714216) / 24895.882753930182 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorConvert.Widen_short

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 34.246972766229035 < 58.5664415097042.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 39.76789794304391 (T) = (0 -34242.325768524875) / Math.Sqrt((55617822.36904002 / (299)) + (8242.692193215653 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.3339418679980646 = (51410.41618335106 - 34242.325768524875) / 51410.41618335106 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorConvert.Narrow_short

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 35.35735500755858 < 42.22975779459919.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.246437744216664 (T) = (0 -35444.253810836104) / Math.Sqrt((7283392.9447731655 / (299)) + (11380.764891040833 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1796835977605321 = (43208.02767575184 - 35444.253810836104) / 43208.02767575184 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorConvert.Narrow_long

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.39209874434069 < 20.694708297481764.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.27689895290594 (T) = (0 -13334.65620106077) / Math.Sqrt((8258811.588088463 / (299)) + (4448.1979421918095 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.3813071520105457 = (21552.950295763014 - 13334.65620106077) / 21552.950295763014 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorConvert.Widen_ushort

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 34.40747043956044 < 59.11217809576746.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.903531910245405 (T) = (0 -34384.3047592745) / Math.Sqrt((60149323.03764479 / (299)) + (4405.373858262346 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.3846773527613336 = (55880.122263625686 - 34384.3047592745) / 55880.122263625686 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorConvert.Convert_long_double

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 49.70834182641108 < 53.06411965752398.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 43.49441535923764 (T) = (0 -50039.331140149225) / Math.Sqrt((4056645.3178655435 / (299)) + (60848.473728826146 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.0996473645074635 = (55577.48060877868 - 50039.331140149225) / 55577.48060877868 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorConvert.Narrow_ushort

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 35.39001535229477 < 42.090118624848294.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.43989539189363 (T) = (0 -35471.3900278144) / Math.Sqrt((7205855.024912623 / (299)) + (4705.603085854582 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.17850388344730983 = (43179.01121269548 - 35471.3900278144) / 43179.01121269548 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorConvert.Widen_int

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 26.391712008150648 < 50.59942385096701.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 42.54745220313224 (T) = (0 -26290.29157684702) / Math.Sqrt((52388494.7883333 / (299)) + (5453.794453395877 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.4040105851292456 = (44112.01091977162 - 26290.29157684702) / 44112.01091977162 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorConvert.Narrow_double

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.538138475836432 < 21.692261823778246.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.04805598746905 (T) = (0 -14374.28144662862) / Math.Sqrt((14964676.647131955 / (299)) + (23961.556803399526 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.414647545498373 = (24556.626244724608 - 14374.28144662862) / 24556.626244724608 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorConvert.Narrow_uint

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.559712028508773 < 27.773478410101568.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.01002406617386 (T) = (0 -20565.615090616408) / Math.Sqrt((8782057.354230002 / (299)) + (3378.7003389744377 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.2904977989895458 = (28985.977860713334 - 20565.615090616408) / 28985.977860713334 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorConvert.Widen_uint

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 25.670634435261704 < 50.150751353558796.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 29.2689701124354 (T) = (0 -25805.765345159805) / Math.Sqrt((63356923.97836274 / (299)) + (8493.099308322786 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.3432083548921479 = (39290.6419217958 - 25805.765345159805) / 39290.6419217958 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorConvert.Convert_double_long

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.294461904369786 < 17.385175204633494.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 46.350163669099516 (T) = (0 -10301.993799898228) / Math.Sqrt((11956905.89135593 / (299)) + (721.2437260839095 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.4737067339301817 = (19574.62590549574 - 10301.993799898228) / 19574.62590549574 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorConvert.Convert_ulong_double

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 49.47961801201672 < 53.28061846699522.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 43.94970514823177 (T) = (0 -49370.691448731865) / Math.Sqrt((5773136.850769505 / (299)) + (42689.371364532155 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.11465671179710121 = (55764.46120571631 - 49370.691448731865) / 55764.46120571631 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorConvert.Widen_float

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 39.43773002083333 < 60.8770006147541.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.6386197162067 (T) = (0 -39464.303674353636) / Math.Sqrt((91200705.64285205 / (299)) + (34263.91595757517 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.4005900510300692 = (65838.5863333966 - 39464.303674353636) / 65838.5863333966 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorConvert.Widen_byte

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 49.55470714155307 < 70.05187064103659.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.52936186178807 (T) = (0 -49742.983365598986) / Math.Sqrt((52768417.85699896 / (299)) + (39172.6928038918 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.29592747544239895 = (70650.36857794534 - 49742.983365598986) / 70650.36857794534 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorConvert.Convert_int_float

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 17.43937959259259 < 25.02077278646175.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.54994628518143 (T) = (0 -17406.05296025321) / Math.Sqrt((10686610.841475192 / (299)) + (2082.235317388982 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.34554363008059563 = (26596.200694626517 - 17406.05296025321) / 26596.200694626517 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorConvert.Widen_sbyte

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 49.68173034018987 < 69.71394450958326.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.8255604794073 (T) = (0 -49476.598347142346) / Math.Sqrt((53521619.98870557 / (299)) + (189031.51522278623 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.2994365605502063 = (70624.00856373567 - 49476.598347142346) / 70624.00856373567 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorConvert.Narrow_ulong

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.299431800158604 < 20.81139085972426.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.74747918727588 (T) = (0 -13294.25448378589) / Math.Sqrt((8049523.003241178 / (299)) + (15402.780960107191 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.3785262513955116 = (21391.498053840525 - 13294.25448378589) / 21391.498053840525 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorConvert.Narrow_int

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.311872639973956 < 27.38372430537659.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.87020352285725 (T) = (0 -20337.403076366187) / Math.Sqrt((9111215.2158979 / (299)) + (2774.758797520369 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.2959233110479144 = (28885.210085048286 - 20337.403076366187) / 28885.210085048286 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-11T18:53:46Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Float

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
AddBenchmark - Duration of single invocation	37.82 ns	1.26 ns	0.03	0.17	False
AllBitsSetBenchmark - Duration of single invocation	6.58 ns	1.08 ns	0.16	0.09	False
NegateBenchmark - Duration of single invocation	35.51 ns	0.96 ns	0.03	0.16	False
EqualityOperatorBenchmark - Duration of single invocation	7.63 ns	0.00 ns	0.00	0.20	False
DivideBenchmark - Duration of single invocation	41.35 ns	1.04 ns	0.03	0.12	False
OnesComplementOperatorBenchmark - Duration of single invocation	12.98 ns	1.18 ns	0.09	0.19	False
BitwiseOrBenchmark - Duration of single invocation	13.15 ns	1.02 ns	0.08	0.13	False
MultiplyOperatorBenchmark - Duration of single invocation	38.69 ns	1.22 ns	0.03	0.08	False
DivisionOperatorBenchmark - Duration of single invocation	41.17 ns	0.58 ns	0.01	0.13	False
SubtractionOperatorBenchmark - Duration of single invocation	37.86 ns	1.04 ns	0.03	0.18	False
AddOperatorBenchmark - Duration of single invocation	38.00 ns	1.03 ns	0.03	0.13	False
InequalityOperatorBenchmark - Duration of single invocation	8.36 ns	0.05 ns	0.01	0.26	False
BitwiseOrOperatorBenchmark - Duration of single invocation	13.20 ns	0.98 ns	0.07	0.15	False
BitwiseAndOperatorBenchmark - Duration of single invocation	13.58 ns	1.07 ns	0.08	0.15	False
OnesComplementBenchmark - Duration of single invocation	11.10 ns	1.34 ns	0.12	0.21	False
SubtractBenchmark - Duration of single invocation	37.88 ns	1.04 ns	0.03	0.16	False
UnaryNegateOperatorBenchmark - Duration of single invocation	35.81 ns	1.02 ns	0.03	0.16	False
XorBenchmark - Duration of single invocation	13.18 ns	1.00 ns	0.08	0.18	False
MultiplyBenchmark - Duration of single invocation	38.75 ns	1.16 ns	0.03	0.10	False
ExclusiveOrOperatorBenchmark - Duration of single invocation	13.20 ns	1.04 ns	0.08	0.18	False
BitwiseAndBenchmark - Duration of single invocation	13.19 ns	1.18 ns	0.09	0.16	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Float*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Float* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Float*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Float* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.AddBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2553987281938601 < 35.94699929401828.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.551453607285936 (T) = (0 -1.1019892043155233) / Math.Sqrt((179.0994403541137 / (299)) + (0.022321570004502057 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9715312234348412 = (38.7086955350298 - 1.1019892043155233) / 38.7086955350298 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.AllBitsSetBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0751173390174458 < 6.24330207769909.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 186.10142058832383 (T) = (0 -0.9938472097682919) / Math.Sqrt((0.04957901962772271 / (299)) + (0.01319339109751031 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8358482971505653 = (6.054443496573907 - 0.9938472097682919) / 6.054443496573907 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.NegateBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9554740139391183 < 33.88726872878973.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.28122351979976 (T) = (0 -1.0731967038257837) / Math.Sqrt((96.34657557409669 / (299)) + (0.005968790842394488 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9630684410864184 = (29.05906859596748 - 1.0731967038257837) / 29.05906859596748 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.EqualityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.004679477290501698 < 7.868932290231129.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/21/2023 3:33:39 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 76.34452736544517 (T) = (0 -0.03948237749275646) / Math.Sqrt((5.521172859857063 / (299)) + (0.005708091819374225 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9962337640985018 = (10.483246011502033 - 0.03948237749275646) / 10.483246011502033 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.DivideBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0434396570267166 < 39.16560788207255.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 191.17672493306483 (T) = (0 -1.0716137138464328) / Math.Sqrt((16.261350727341682 / (299)) + (0.013727189758420945 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9766530144698243 = (45.89944652432271 - 1.0716137138464328) / 45.89944652432271 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1815689867012296 < 10.661879781778948.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.8089895755453 (T) = (0 -1.0759505323855925) / Math.Sqrt((7.19355141332896 / (299)) + (0.006566015081085339 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.8739720179838223 = (8.537393959442095 - 1.0759505323855925) / 8.537393959442095 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.BitwiseOrBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0175549759275748 < 12.625545915245837.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.91535879202539 (T) = (0 -1.0120551547202694) / Math.Sqrt((18.960829647622965 / (299)) + (0.028814061800399134 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9247603106334935 = (13.451080981878595 - 1.0120551547202694) / 13.451080981878595 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.MultiplyOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.216515302828329 < 36.76279153769067.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 335.7552398579071 (T) = (0 -1.1369666663804996) / Math.Sqrt((2.8270413263962766 / (299)) + (0.08998882788695285 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9715447816866815 = (39.95635014503971 - 1.1369666663804996) / 39.95635014503971 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.DivisionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.5831366775377174 < 39.05327577794914.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 408.62181474860853 (T) = (0 -1.0529071494979683) / Math.Sqrt((2.7778463634655575 / (299)) + (0.020634657465191376 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9751065921274888 = (42.29662547170384 - 1.0529071494979683) / 42.29662547170384 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.SubtractionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0358057996655952 < 35.9558201319017.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.69405854045517 (T) = (0 -1.072170623140086) / Math.Sqrt((143.03815193020367 / (299)) + (0.008843014927508846 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9697615452668491 = (35.45718961507156 - 1.072170623140086) / 35.45718961507156 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.AddOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0339649612458923 < 35.95598202036198.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.78722511389363 (T) = (0 -1.0620505908761126) / Math.Sqrt((142.2513784994868 / (299)) + (0.010724736892521625 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9700152471780417 = (35.41968803886074 - 1.0620505908761126) / 35.41968803886074 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.InequalityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.04991218052789881 < 7.937533162211661.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/21/2023 3:33:39 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 76.00333104689254 (T) = (0 -0.07112671858025854) / Math.Sqrt((5.451452967385607 / (299)) + (0.01512174192149817 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9932369918135014 = (10.517023877370546 - 0.07112671858025854) / 10.517023877370546 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9775614700399752 < 12.571351691254307.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.289268694510326 (T) = (0 -1.0934739287803101) / Math.Sqrt((16.03324451637232 / (299)) + (0.03335289846120833 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9136310802754166 = (12.660502554243164 - 1.0934739287803101) / 12.660502554243164 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0677742076042505 < 12.541536711824543.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.574481222967535 (T) = (0 -1.0531034841165074) / Math.Sqrt((16.032498331862037 / (299)) + (0.02629859187710284 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9167793936157335 = (12.654359657677333 - 1.0531034841165074) / 12.654359657677333 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.OnesComplementBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3396173828195077 < 10.526483629386194.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.69181878384261 (T) = (0 -1.1240845620185729) / Math.Sqrt((7.131227986130819 / (299)) + (0.05184179828487715 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.8680790967367491 = (8.520898009434024 - 1.1240845620185729) / 8.520898009434024 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.SubtractBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.043198976399381 < 35.971685738369466.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.55954202573848 (T) = (0 -1.0819472410047763) / Math.Sqrt((179.37650315791538 / (299)) + (0.01034420678903683 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9720482244214196 = (38.70763908944222 - 1.0819472410047763) / 38.70763908944222 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.UnaryNegateOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0216149051370864 < 33.92605213106175.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.20821339400399 (T) = (0 -1.0925882002699665) / Math.Sqrt((96.46195940338056 / (299)) + (0.014510670280081002 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9624149951401174 = (29.0697900490674 - 1.0925882002699665) / 29.0697900490674 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.XorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9977840513917069 < 12.543089579390116.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.62226809703301 (T) = (0 -1.151721763561023) / Math.Sqrt((18.982375323840586 / (299)) + (0.07579532435691137 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9144231809378595 = (13.458338089485588 - 1.151721763561023) / 13.458338089485588 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.MultiplyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1598742711964174 < 36.75810407828732.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 187.05317207978814 (T) = (0 -1.040545948353989) / Math.Sqrt((15.236404341866551 / (299)) + (0.009164261167090817 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.976041074855261 = (43.430410257051015 - 1.040545948353989) / 43.430410257051015 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0435366986024894 < 12.59087742201826.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.68645813518616 (T) = (0 -1.0866618614507615) / Math.Sqrt((16.00030902809044 / (299)) + (0.016480835304497756 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9141470072461245 = (12.657239154912379 - 1.0866618614507615) / 12.657239154912379 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.BitwiseAndBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1841313951272603 < 12.548883423768311.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.99454125845711 (T) = (0 -1.0765471986737836) / Math.Sqrt((18.913615105479224 / (299)) + (0.011058898623143406 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.9199469803070086 = (13.447927421131805 - 1.0765471986737836) / 13.447927421131805 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-11T18:53:47Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
AndNotBenchmark - Duration of single invocation	17.71 ns	1.09 ns	0.06	0.14	False
OnesComplementOperatorBenchmark - Duration of single invocation	8.99 ns	1.10 ns	0.12	0.17	False
BitwiseAndOperatorBenchmark - Duration of single invocation	9.30 ns	1.19 ns	0.13	0.19	False
BitwiseOrBenchmark - Duration of single invocation	9.52 ns	1.03 ns	0.11	0.17	False
OnesComplementBenchmark - Duration of single invocation	9.00 ns	1.09 ns	0.12	0.15	False
ExclusiveOrOperatorBenchmark - Duration of single invocation	9.49 ns	1.05 ns	0.11	0.22	False
OneBenchmark - Duration of single invocation	5.82 ns	1.25 ns	0.21	0.35	False
ConditionalSelectBenchmark - Duration of single invocation	33.00 ns	3.66 ns	0.11	0.12	False
BitwiseAndBenchmark - Duration of single invocation	9.59 ns	1.02 ns	0.11	0.15	False
BitwiseOrOperatorBenchmark - Duration of single invocation	9.52 ns	1.05 ns	0.11	0.15	False
XorBenchmark - Duration of single invocation	9.53 ns	1.24 ns	0.13	0.14	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt64>.AndNotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0912051895742432 < 16.829513953285637.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.18743069860824 (T) = (0 -1.2120533654078651) / Math.Sqrt((26.90262519112322 / (299)) + (0.019601237506665977 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9244165863106375 = (16.035970145371298 - 1.2120533654078651) / 16.035970145371298 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt64>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0981805592643041 < 8.557420597247173.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 40.92290576834518 (T) = (0 -1.2003914576454786) / Math.Sqrt((3.962049666360687 / (299)) + (0.027276847082735815 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8037754244928661 = (6.117436893636379 - 1.2003914576454786) / 6.117436893636379 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt64>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1860871606608565 < 9.11753036150609.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.62691055411747 (T) = (0 -1.2079753042122938) / Math.Sqrt((8.25185632605875 / (299)) + (0.07442728624839667 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8689817870244059 = (9.219903681920265 - 1.2079753042122938) / 9.219903681920265 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt64>.BitwiseOrBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0255231418988466 < 9.089507086154692.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.649570754289215 (T) = (0 -1.2160121420858696) / Math.Sqrt((10.927960081484526 / (299)) + (0.03102326629590553 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8796350152513273 = (10.102706735060494 - 1.2160121420858696) / 10.102706735060494 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt64>.OnesComplementBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0852284967708656 < 8.591523891025899.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 40.89235988656378 (T) = (0 -1.217819327263767) / Math.Sqrt((3.8848956874293346 / (299)) + (0.03371165103745261 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8014929338217908 = (6.134891571922546 - 1.217819327263767) / 6.134891571922546 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt64>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.052303730598432 < 9.05427795665917.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.39754973037204 (T) = (0 -1.2679494291793632) / Math.Sqrt((8.317176562976902 / (299)) + (0.06550206440366851 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8624433120627711 = (9.217650178942705 - 1.2679494291793632) / 9.217650178942705 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt64>.OneBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.250179400189516 < 5.609057707248157.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 29.56949079261517 (T) = (0 -1.2366376611262284) / Math.Sqrt((9.886049792511445 / (299)) + (0.047123442757527015 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.8177344778023463 = (6.784813969288087 - 1.2366376611262284) / 6.784813969288087 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt64>.ConditionalSelectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.663017415216087 < 31.331483208540657.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.11440789010736 (T) = (0 -1.3738163858790045) / Math.Sqrt((110.1350802833062 / (299)) + (0.2690608132310944 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9565957906515192 = (31.65168555079534 - 1.3738163858790045) / 31.65168555079534 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt64>.BitwiseAndBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0175348078699966 < 9.105464282666762.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.4890013417799 (T) = (0 -1.2445190552913181) / Math.Sqrt((10.98138251656547 / (299)) + (0.03024074534589102 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8769853860170929 = (10.116839089250357 - 1.2445190552913181) / 10.116839089250357 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt64>.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0523162226416607 < 9.062077201378779.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 41.363662233547295 (T) = (0 -1.2978750201302403) / Math.Sqrt((8.241322850189723 / (299)) + (0.2054437415532428 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.8595937945996244 = (9.243715521185702 - 1.2978750201302403) / 9.243715521185702 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt64>.XorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2427096537362998 < 9.06736022816907.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 44.086741674559775 (T) = (0 -1.309889053699943) / Math.Sqrt((10.838423531510028 / (299)) + (0.0828132900785852 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8704444015081113 = (10.110632569706764 - 1.309889053699943) / 10.110632569706764 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-11T18:53:48Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
NegateBenchmark - Duration of single invocation	38.30 ns	1.44 ns	0.04	0.11	False
AddOperatorBenchmark - Duration of single invocation	38.79 ns	0.93 ns	0.02	0.11	False
OnesComplementBenchmark - Duration of single invocation	9.10 ns	1.24 ns	0.14	0.16	False
ExclusiveOrOperatorBenchmark - Duration of single invocation	9.54 ns	1.18 ns	0.12	0.15	False
BitwiseAndOperatorBenchmark - Duration of single invocation	9.59 ns	0.96 ns	0.10	0.17	False
SubtractionOperatorBenchmark - Duration of single invocation	37.58 ns	1.24 ns	0.03	0.13	False
MultiplyBenchmark - Duration of single invocation	39.79 ns	0.99 ns	0.02	0.04	True
ConditionalSelectBenchmark - Duration of single invocation	32.74 ns	1.21 ns	0.04	0.40	False
AndNotBenchmark - Duration of single invocation	17.73 ns	1.30 ns	0.07	0.12	False
AddBenchmark - Duration of single invocation	38.11 ns	1.07 ns	0.03	0.12	False
MultiplyOperatorBenchmark - Duration of single invocation	40.19 ns	1.29 ns	0.03	0.05	True
UnaryNegateOperatorBenchmark - Duration of single invocation	38.36 ns	1.09 ns	0.03	0.08	False
SubtractBenchmark - Duration of single invocation	38.23 ns	1.24 ns	0.03	0.10	False
BitwiseOrOperatorBenchmark - Duration of single invocation	9.09 ns	1.27 ns	0.14	0.18	False
OnesComplementOperatorBenchmark - Duration of single invocation	9.34 ns	0.98 ns	0.10	0.16	False
OneBenchmark - Duration of single invocation	22.40 ns	1.31 ns	0.06	0.25	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<SByte>.NegateBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.4429153349677033 < 36.23029518426999.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.443833576163975 (T) = (0 -1.2140430531486262) / Math.Sqrt((146.31682820329985 / (299)) + (0.03001687360037638 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9661335841423686 = (35.84799342960464 - 1.2140430531486262) / 35.84799342960464 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.AddOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9317329445190736 < 36.56172711245007.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.68406759788131 (T) = (0 -1.2300117284850287) / Math.Sqrt((141.48851621709315 / (299)) + (0.038197261653216935 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9653201108492533 = (35.467579585921136 - 1.2300117284850287) / 35.467579585921136 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.OnesComplementBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.242523889114185 < 8.65923064888196.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 44.90543551761194 (T) = (0 -1.2175031563432674) / Math.Sqrt((5.479448819091446 / (299)) + (0.03102211104236686 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8380168503531805 = (7.516233379816691 - 1.2175031563432674) / 7.516233379816691 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1804091246133157 < 9.100235915089637.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 42.96753493170831 (T) = (0 -1.223080224056925) / Math.Sqrt((16.168179170829507 / (299)) + (0.022766629293572607 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8918176153795911 = (11.30572438709387 - 1.223080224056925) / 11.30572438709387 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9582587470247541 < 9.077255029756085.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 43.0634685421938 (T) = (0 -1.2590151801936105) / Math.Sqrt((15.964277783380988 / (299)) + (0.027733714105401628 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8887925662966647 = (11.321322129887896 - 1.2590151801936105) / 11.321322129887896 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.SubtractionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2435775841255334 < 35.92690465285832.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.77944354804727 (T) = (0 -1.2355764125920392) / Math.Sqrt((141.03950700623776 / (299)) + (0.01916169818708234 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9651504938700316 = (35.454631924597685 - 1.2355764125920392) / 35.454631924597685 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.MultiplyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9939485472428701 < 37.93196582303564.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 373.35894807534186 (T) = (0 -1.2517837743751645) / Math.Sqrt((3.294225449710834 / (299)) + (0.030373734886136493 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9707002302052202 = (42.72333138256225 - 1.2517837743751645) / 42.72333138256225 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.ConditionalSelectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2109640570142877 < 31.046190222037907.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 41.0481104842266 (T) = (0 -1.3962911308654136) / Math.Sqrt((361.00020295452384 / (299)) + (0.19413633593909294 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9700734607785321 = (46.657287049876395 - 1.3962911308654136) / 46.657287049876395 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.AndNotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3015739559070874 < 16.828284653896354.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.091683517575454 (T) = (0 -1.224479396208972) / Math.Sqrt((51.88721510140133 / (299)) + (0.017687934224294002 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9389292662001902 = (20.050183124093785 - 1.224479396208972) / 20.050183124093785 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.AddBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0669104003068153 < 36.25243466517998.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.634856792130414 (T) = (0 -1.2191005808732962) / Math.Sqrt((153.69754127340389 / (299)) + (0.02781272581966363 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.966914865190441 = (36.84738139622362 - 1.2191005808732962) / 36.84738139622362 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.MultiplyOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2881083825411064 < 37.99860271899436.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 277.36626609506277 (T) = (0 -1.4075657595490267) / Math.Sqrt((0.5509458767210761 / (299)) + (0.4236223350124703 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.965575246576339 = (40.88818712007309 - 1.4075657595490267) / 40.88818712007309 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.UnaryNegateOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0948518996240846 < 36.393181983131065.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.49045364561081 (T) = (0 -1.169496343850459) / Math.Sqrt((141.93223929747577 / (299)) + (0.017314460032070114 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.9668656167269122 = (35.295551880705766 - 1.169496343850459) / 35.295551880705766 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.SubtractBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2367853725333424 < 36.43005217220565.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.37665715187596 (T) = (0 -1.2850003743080771) / Math.Sqrt((154.66137687117316 / (299)) + (0.028375630982888 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.9651207533462441 = (36.84140277065602 - 1.2850003743080771) / 36.84140277065602 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2672629343586368 < 9.075084041308.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 42.94538850735213 (T) = (0 -1.2411826747413433) / Math.Sqrt((16.06995218926205 / (299)) + (0.023270905699502683 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8900687509269463 = (11.290535541141066 - 1.2411826747413433) / 11.290535541141066 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9769822644863946 < 8.607734880430934.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.02308617945957 (T) = (0 -1.2098042143944319) / Math.Sqrt((5.473365125796831 / (299)) + (0.033910030694211904 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8395889439557269 = (7.541900441454168 - 1.2098042143944319) / 7.541900441454168 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<SByte>.OneBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.311041168821004 < 19.553396124968454.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 32.1233719302394 (T) = (0 -1.2698299657458516) / Math.Sqrt((81.86503404592013 / (299)) + (0.01648760369538968 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9298457576556604 = (18.10054421959426 - 1.2698299657458516) / 18.10054421959426 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-11T18:53:50Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
AddBenchmark - Duration of single invocation	15.59 ns	1.04 ns	0.07	0.15	False
OnesComplementBenchmark - Duration of single invocation	11.03 ns	1.00 ns	0.09	0.17	False
InequalityOperatorBenchmark - Duration of single invocation	3.06 ns	0.00 ns	0.00	0.40	False
ExclusiveOrOperatorBenchmark - Duration of single invocation	13.28 ns	1.24 ns	0.09	0.12	False
BitwiseOrBenchmark - Duration of single invocation	13.02 ns	1.16 ns	0.09	0.13	False
MultiplyBenchmark - Duration of single invocation	15.74 ns	1.17 ns	0.07	0.06	True
BitwiseAndOperatorBenchmark - Duration of single invocation	13.20 ns	1.41 ns	0.11	0.13	False
OnesComplementOperatorBenchmark - Duration of single invocation	10.96 ns	1.01 ns	0.09	0.15	False
MultiplyOperatorBenchmark - Duration of single invocation	15.75 ns	1.23 ns	0.08	0.08	True
EqualityOperatorBenchmark - Duration of single invocation	3.10 ns	0.00 ns	0.00	0.44	False
UnaryNegateOperatorBenchmark - Duration of single invocation	14.59 ns	1.01 ns	0.07	0.12	False
NegateBenchmark - Duration of single invocation	14.53 ns	1.11 ns	0.08	0.12	False
SubtractionOperatorBenchmark - Duration of single invocation	15.73 ns	1.30 ns	0.08	0.13	False
BitwiseAndBenchmark - Duration of single invocation	13.19 ns	0.99 ns	0.08	0.13	False
AddOperatorBenchmark - Duration of single invocation	15.52 ns	1.17 ns	0.08	0.12	False
XorBenchmark - Duration of single invocation	13.15 ns	0.98 ns	0.07	0.10	False
BitwiseOrOperatorBenchmark - Duration of single invocation	14.70 ns	1.36 ns	0.09	0.13	False
SubtractBenchmark - Duration of single invocation	15.45 ns	1.18 ns	0.08	0.12	False
AllBitsSetBenchmark - Duration of single invocation	6.36 ns	1.05 ns	0.17	0.13	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.AddBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.037485431582039 < 14.932557117758494.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.68125559744273 (T) = (0 -1.1720717349728702) / Math.Sqrt((33.933399822439654 / (299)) + (0.015447477833096109 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9321815828464346 = (17.282499122898678 - 1.1720717349728702) / 17.282499122898678 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.OnesComplementBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9989775239072181 < 10.711825509915625.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 44.27267552885427 (T) = (0 -1.210665552577408) / Math.Sqrt((7.327723522972432 / (299)) + (0.06716631259040257 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.8585757520583551 = (8.560523179002221 - 1.210665552577408) / 8.560523179002221 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.InequalityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 3.021844733421663.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 60.83968593536266 (T) = (0 -0.06666224080229284) / Math.Sqrt((2.093283594719773 / (299)) + (0.008350593266480884 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9873926038049665 = (5.287550242020096 - 0.06666224080229284) / 5.287550242020096 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2428730462243507 < 12.701342546771484.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.393660017095755 (T) = (0 -1.2310893869269255) / Math.Sqrt((16.232306958271884 / (299)) + (0.09378336598403597 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9029183504593614 = (12.68096898592137 - 1.2310893869269255) / 12.68096898592137 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.BitwiseOrBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1630673865293735 < 12.70148927065443.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.32302573535854 (T) = (0 -1.1988861989919157) / Math.Sqrt((19.068669487742206 / (299)) + (0.017386152508271146 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9110244705305113 = (13.474336215139184 - 1.1988861989919157) / 13.474336215139184 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.MultiplyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1651865104415775 < 15.124020386964068.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 130.8022305339436 (T) = (0 -1.2609835149694981) / Math.Sqrt((3.6558968547621626 / (299)) + (0.14957762862775337 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9341951781930997 = (19.16247898474324 - 1.2609835149694981) / 19.16247898474324 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.4136183097602557 < 12.750915878989998.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.17854068029556 (T) = (0 -1.2372711796595706) / Math.Sqrt((16.37781011064412 / (299)) + (0.09008051572536065 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9023227386852118 = (12.666931515126851 - 1.2372711796595706) / 12.666931515126851 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0055211170565128 < 10.596215768429373.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 46.43334871494899 (T) = (0 -1.1631253936852737) / Math.Sqrt((7.33803002711674 / (299)) + (0.011141269421047656 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.8633542321936373 = (8.511975250733778 - 1.1631253936852737) / 8.511975250733778 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.MultiplyOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2342773274783634 < 15.138714124000952.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 249.8086553090519 (T) = (0 -1.1687072368235714) / Math.Sqrt((1.172233191969301 / (299)) + (0.013428465866491187 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.934834672813637 = (17.934495034165096 - 1.1687072368235714) / 17.934495034165096 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.EqualityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 2.885623143336114.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 59.84050730512657 (T) = (0 -0.050797051541904226) / Math.Sqrt((2.1168354823436015 / (299)) + (0.013483050773450389 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9903978101436021 = (5.290152798640817 - 0.050797051541904226) / 5.290152798640817 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.UnaryNegateOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0123220585159827 < 13.97707113161734.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.05891673179907 (T) = (0 -1.1639481161524134) / Math.Sqrt((18.861949770823284 / (299)) + (0.008710717294025151 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9139264977319642 = (13.52272285293841 - 1.1639481161524134) / 13.52272285293841 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.NegateBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1078644459934524 < 14.031010800986857.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.06550974715126 (T) = (0 -1.1589866042304318) / Math.Sqrt((18.888296949745673 / (299)) + (0.007946792709973957 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9143063585503176 = (13.524767819686655 - 1.1589866042304318) / 13.524767819686655 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.SubtractionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2959668357466694 < 15.050131143774212.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.717819671331156 (T) = (0 -1.1927213977180653) / Math.Sqrt((27.46732872378662 / (299)) + (0.011167216443936088 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9254437741922648 = (15.997609653603469 - 1.1927213977180653) / 15.997609653603469 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.BitwiseAndBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9930072070123077 < 12.657442607333888.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.47220313443469 (T) = (0 -1.1828479564478744) / Math.Sqrt((19.06375719533953 / (299)) + (0.0731576063834377 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9121548093498963 = (13.465141889887612 - 1.1828479564478744) / 13.465141889887612 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.AddOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1733113508582098 < 15.04530001925383.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.36755605738163 (T) = (0 -1.1446487906464617) / Math.Sqrt((26.874834292883243 / (299)) + (0.017531229199782332 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9284945755125312 = (16.007859527455288 - 1.1446487906464617) / 16.007859527455288 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.XorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9752100936112758 < 12.705087523578332.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.64040827854123 (T) = (0 -1.1202992703764239) / Math.Sqrt((19.095738884363342 / (299)) + (0.016764200513952447 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.916906852236884 = (13.482450725903426 - 1.1202992703764239) / 13.482450725903426 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3562742603519884 < 12.693769017254528.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.41314315637758 (T) = (0 -1.1713938302356) / Math.Sqrt((15.95815982859023 / (299)) + (0.021964927023612538 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9076818359349595 = (12.68866037468339 - 1.1713938302356) / 12.68866037468339 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.SubtractBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1844027932970909 < 15.166670612079407.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 46.83823995583112 (T) = (0 -1.2899182024227316) / Math.Sqrt((33.78182145023547 / (299)) + (0.09293817232863918 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9254928128853775 = (17.312668111310014 - 1.2899182024227316) / 17.312668111310014 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.AllBitsSetBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0536561121203296 < 6.418370213261517.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 199.97897193601128 (T) = (0 -1.1710339590779433) / Math.Sqrt((0.07303476062217805 / (299)) + (0.008099157118086378 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8065935914470915 = (6.054783643622614 - 1.1710339590779433) / 6.054783643622614 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-11T18:53:51Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
OnesComplementOperatorBenchmark - Duration of single invocation	8.94 ns	1.16 ns	0.13	0.15	False
NegateBenchmark - Duration of single invocation	22.72 ns	1.12 ns	0.05	0.10	False
AddBenchmark - Duration of single invocation	22.88 ns	1.04 ns	0.05	0.12	False
ExclusiveOrOperatorBenchmark - Duration of single invocation	9.54 ns	1.06 ns	0.11	0.16	False
OneBenchmark - Duration of single invocation	12.26 ns	1.32 ns	0.11	0.21	False
ConditionalSelectBenchmark - Duration of single invocation	32.61 ns	1.03 ns	0.03	0.41	False
MultiplyOperatorBenchmark - Duration of single invocation	23.65 ns	1.00 ns	0.04	0.04	True
BitwiseOrOperatorBenchmark - Duration of single invocation	9.34 ns	1.02 ns	0.11	0.13	False
UnaryNegateOperatorBenchmark - Duration of single invocation	22.56 ns	1.01 ns	0.04	0.11	False
OnesComplementBenchmark - Duration of single invocation	9.13 ns	1.11 ns	0.12	0.14	False
AndNotBenchmark - Duration of single invocation	17.68 ns	1.02 ns	0.06	0.10	False
BitwiseAndOperatorBenchmark - Duration of single invocation	9.49 ns	1.09 ns	0.12	0.18	False
AddOperatorBenchmark - Duration of single invocation	22.65 ns	0.98 ns	0.04	0.11	False
SubtractBenchmark - Duration of single invocation	22.85 ns	1.05 ns	0.05	0.10	False
SubtractionOperatorBenchmark - Duration of single invocation	22.63 ns	1.06 ns	0.05	0.11	False
MultiplyBenchmark - Duration of single invocation	23.48 ns	1.04 ns	0.04	0.05	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int16>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1630577870463885 < 8.485673789503938.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.42256743285315 (T) = (0 -1.1732270505823414) / Math.Sqrt((5.636419408831696 / (299)) + (0.013718553337558446 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.843727588480322 = (7.507576284087787 - 1.1732270505823414) / 7.507576284087787 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.NegateBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1174388910912574 < 21.60939984426568.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.21667185965124 (T) = (0 -1.1197780350767155) / Math.Sqrt((53.41484631448334 / (299)) + (0.011159439557025627 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9489853570365201 = (21.950129806423192 - 1.1197780350767155) / 21.950129806423192 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.AddBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0368236602864769 < 21.70700928168459.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.75750346472648 (T) = (0 -1.2463844717660992) / Math.Sqrt((57.171185201345885 / (299)) + (0.08481346473913066 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9452652266609997 = (22.77134617963256 - 1.2463844717660992) / 22.77134617963256 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.055885456168843 < 9.08849538144432.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 43.48714843376722 (T) = (0 -1.1333434482722993) / Math.Sqrt((16.04027528342284 / (299)) + (0.027358800099412325 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8998526533638072 = (11.316759618099693 - 1.1333434482722993) / 11.316759618099693 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.OneBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3160600377528808 < 11.681190633071592.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 31.84701396157189 (T) = (0 -1.2510088071149368) / Math.Sqrt((33.135859637429846 / (299)) + (0.012503526312277973 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8946859875107689 = (11.878844776167451 - 1.2510088071149368) / 11.878844776167451 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.ConditionalSelectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0347371440794935 < 31.157730103831888.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 41.305942261540075 (T) = (0 -1.1700439618581928) / Math.Sqrt((361.4676101372342 / (299)) + (0.20246212699960767 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9749730604526473 = (46.75138003367871 - 1.1700439618581928) / 46.75138003367871 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.MultiplyOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0015803705497628 < 22.45565614005946.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 245.42560294466577 (T) = (0 -1.2030304543173789) / Math.Sqrt((0.37952987133574945 / (299)) + (0.18361294443843582 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9515112398014027 = (24.81050143146744 - 1.2030304543173789) / 24.81050143146744 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0213973521550852 < 9.074988584395877.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 42.18382978094054 (T) = (0 -1.1642505976169948) / Math.Sqrt((16.21731208078377 / (299)) + (0.08495522360870013 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8971284985801773 = (11.317523138557512 - 1.1642505976169948) / 11.317523138557512 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.UnaryNegateOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.008378297284898 < 21.562851612372388.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.735386844070376 (T) = (0 -1.2397627449511939) / Math.Sqrt((50.14911059733527 / (299)) + (0.06188202089668953 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9419540230229851 = (21.358288886103438 - 1.2397627449511939) / 21.358288886103438 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.OnesComplementBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1059755095117767 < 8.67260070368684.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.97964799649685 (T) = (0 -1.135406101318273) / Math.Sqrt((5.6364495990142265 / (299)) + (0.009418933411879158 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8489461759396301 = (7.516566418500594 - 1.135406101318273) / 7.516566418500594 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.AndNotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0241615056717686 < 16.76171213917877.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.60895960710188 (T) = (0 -1.0919340436795613) / Math.Sqrt((51.49831567296663 / (299)) + (0.0038445027585623627 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9454834200246499 = (20.029393703223562 - 1.0919340436795613) / 20.029393703223562 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0930926596067414 < 9.01884078745355.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 43.312614235468835 (T) = (0 -1.1825847896439405) / Math.Sqrt((16.06594836463777 / (299)) + (0.022077580043063573 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.8954932176228501 = (11.315866422680221 - 1.1825847896439405) / 11.315866422680221 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.AddOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9796811902246699 < 21.68924898402092.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.75302217529948 (T) = (0 -1.1270670898870194) / Math.Sqrt((50.348084303137114 / (299)) + (0.014920690005470845 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9477788457486196 = (21.58257713840608 - 1.1270670898870194) / 21.58257713840608 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.SubtractBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0511454783162795 < 21.881266410082983.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.45689464303878 (T) = (0 -1.141056185491) / Math.Sqrt((57.165468896162444 / (299)) + (0.014733281697582607 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.949958842190448 = (22.802353811110088 - 1.141056185491) / 22.802353811110088 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.SubtractionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0564329353217254 < 21.695844840644835.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.264798530697725 (T) = (0 -1.3095593536411085) / Math.Sqrt((50.21423421692578 / (299)) + (0.19238381663637502 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9393029029726188 = (21.575321024831716 - 1.3095593536411085) / 21.575321024831716 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int16>.MultiplyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0396300766283524 < 22.39870457644168.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 201.54867713448846 (T) = (0 -1.2065866683074085) / Math.Sqrt((2.0516349424615097 / (299)) + (0.19286216118191896 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9537591750265844 = (26.09353680435178 - 1.2065866683074085) / 26.09353680435178 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-11T18:53:52Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
SubtractionOperatorBenchmark - Duration of single invocation	13.78 ns	1.06 ns	0.08	0.13	False
BitwiseAndOperatorBenchmark - Duration of single invocation	13.17 ns	1.45 ns	0.11	0.10	False
InequalityOperatorBenchmark - Duration of single invocation	3.01 ns	0.00 ns	0.00	0.36	False
BitwiseOrOperatorBenchmark - Duration of single invocation	13.26 ns	0.97 ns	0.07	0.12	False
NegateBenchmark - Duration of single invocation	12.47 ns	2.20 ns	0.18	0.14	False
SubtractBenchmark - Duration of single invocation	13.82 ns	1.00 ns	0.07	0.14	False
EqualityOperatorBenchmark - Duration of single invocation	3.27 ns	0.02 ns	0.01	0.42	False
OnesComplementBenchmark - Duration of single invocation	11.11 ns	1.11 ns	0.10	0.16	False
AddOperatorBenchmark - Duration of single invocation	13.82 ns	1.00 ns	0.07	0.14	False
BitwiseOrBenchmark - Duration of single invocation	13.18 ns	1.11 ns	0.08	0.13	False
ExclusiveOrOperatorBenchmark - Duration of single invocation	13.39 ns	1.07 ns	0.08	0.12	False
XorBenchmark - Duration of single invocation	13.17 ns	1.08 ns	0.08	0.10	False
BitwiseAndBenchmark - Duration of single invocation	13.14 ns	0.97 ns	0.07	0.12	False
UnaryNegateOperatorBenchmark - Duration of single invocation	12.66 ns	1.18 ns	0.09	0.15	False
OnesComplementOperatorBenchmark - Duration of single invocation	11.04 ns	1.70 ns	0.15	0.16	False
AllBitsSetBenchmark - Duration of single invocation	6.50 ns	0.99 ns	0.15	0.16	True
AddBenchmark - Duration of single invocation	13.76 ns	0.98 ns	0.07	0.15	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.SubtractionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0570346959002355 < 13.177199489133184.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.89245628170256 (T) = (0 -1.1747091331656503) / Math.Sqrt((19.64862288855335 / (299)) + (0.010220200567818414 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9145697881651788 = (13.750511767861969 - 1.1747091331656503) / 13.750511767861969 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.4548531629976076 < 12.871941536537106.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.974121793515856 (T) = (0 -1.1880716957924482) / Math.Sqrt((16.31181110348059 / (299)) + (0.007166775210457057 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9061520437639596 = (12.659537228538964 - 1.1880716957924482) / 12.659537228538964 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.InequalityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 2.889915593302027.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 56.40719210326165 (T) = (0 -0.11049861889627606) / Math.Sqrt((2.07440086144547 / (299)) + (0.035181687638417024 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9791551879018486 = (5.3010129511350215 - 0.11049861889627606) / 5.3010129511350215 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9661697947686755 < 12.694167182026225.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.868682474050445 (T) = (0 -1.1816689718969458) / Math.Sqrt((16.274211288813365 / (299)) + (0.02005541953999601 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9067620087270093 = (12.673685434053878 - 1.1816689718969458) / 12.673685434053878 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.NegateBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.1986734967284662 < 12.184347890064899.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.26944865545661 (T) = (0 -1.2194751606805674) / Math.Sqrt((12.110893897542038 / (299)) + (0.05739626996291768 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8893548221120997 = (11.021493968007105 - 1.2194751606805674) / 11.021493968007105 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.SubtractBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9979743018825743 < 13.229944555840806.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.44556785901855 (T) = (0 -1.1263461069042087) / Math.Sqrt((25.44474750639283 / (299)) + (0.016842460960863673 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9250426964182282 = (15.026502463171783 - 1.1263461069042087) / 15.026502463171783 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.EqualityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.018071721948485615 < 2.8229175452695436.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 59.673105180452744 (T) = (0 -0.06071774489578087) / Math.Sqrt((2.1200679483573 / (299)) + (0.014272359318869766 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9885454915763061 = (5.300772643388631 - 0.06071774489578087) / 5.300772643388631 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.OnesComplementBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1126302315228693 < 10.775826227481627.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 44.84420614294881 (T) = (0 -1.205894147863364) / Math.Sqrt((7.306626693911008 / (299)) + (0.05333512628381695 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8588136142841444 = (8.541150350645594 - 1.205894147863364) / 8.541150350645594 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.AddOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.001111788681402 < 13.165043415969047.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.79851181766138 (T) = (0 -1.1475538343983545) / Math.Sqrt((19.718692286449993 / (299)) + (0.01312457710984265 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9164403857596279 = (13.733354860845093 - 1.1475538343983545) / 13.733354860845093 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.BitwiseOrBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1122187884542862 < 12.707640517841739.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.62931135603725 (T) = (0 -1.126899531521776) / Math.Sqrt((19.131960883671766 / (299)) + (0.009403098786161329 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9163225750953764 = (13.467187031702132 - 1.126899531521776) / 13.467187031702132 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.065925825013285 < 12.732532566935555.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.97315478213417 (T) = (0 -1.1845084078833157) / Math.Sqrt((16.1976210437971 / (299)) + (0.07018291465742688 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9064401326967529 = (12.660432747772896 - 1.1845084078833157) / 12.660432747772896 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.XorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0845733839507652 < 12.691703922986125.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 46.79571722812393 (T) = (0 -1.212961721834003) / Math.Sqrt((19.085420110351052 / (299)) + (0.1087783986883945 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9099238944370036 = (13.465965410612634 - 1.212961721834003) / 13.465965410612634 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.BitwiseAndBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9705404662384561 < 12.66887908673279.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.23551057541538 (T) = (0 -1.142183542904918) / Math.Sqrt((19.15903790845171 / (299)) + (0.029508450576638742 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9152298259104013 = (13.473884596458124 - 1.142183542904918) / 13.473884596458124 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.UnaryNegateOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.182690899529573 < 12.216468245549155.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.549396421597635 (T) = (0 -1.1382850844915293) / Math.Sqrt((12.30828007116407 / (299)) + (0.011296103391194811 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8969608707988729 = (11.047114754528407 - 1.1382850844915293) / 11.047114754528407 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.7035074340945147 < 10.795322894889635.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.99392716007922 (T) = (0 -1.1573396946286694) / Math.Sqrt((7.437447414082756 / (299)) + (0.020592263334761284 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8644915863807356 = (8.54072203870991 - 1.1573396946286694) / 8.54072203870991 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.AllBitsSetBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9918121723050183 < 6.256589893814055.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 171.22728353026022 (T) = (0 -1.1495067311663585) / Math.Sqrt((0.06941873700427038 / (299)) + (0.012950933297617807 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.8101634968212149 = (6.05524602443699 - 1.1495067311663585) / 6.05524602443699 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.AddBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9777300680387262 < 13.216615698500169.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.168712179466105 (T) = (0 -1.1477834510105027) / Math.Sqrt((25.21550957183444 / (299)) + (0.05627576975308559 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9236994203266647 = (15.04291914851097 - 1.1477834510105027) / 15.04291914851097 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-11T18:53:53Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Int

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
UnaryNegateOperatorBenchmark - Duration of single invocation	14.58 ns	1.10 ns	0.08	0.17	False
OnesComplementOperatorBenchmark - Duration of single invocation	11.09 ns	1.08 ns	0.10	0.17	False
NegateBenchmark - Duration of single invocation	14.50 ns	1.34 ns	0.09	0.16	False
XorBenchmark - Duration of single invocation	13.15 ns	1.29 ns	0.10	0.14	False
MultiplyOperatorBenchmark - Duration of single invocation	15.70 ns	1.12 ns	0.07	0.15	False
OnesComplementBenchmark - Duration of single invocation	11.08 ns	1.12 ns	0.10	0.20	False
MultiplyBenchmark - Duration of single invocation	15.79 ns	1.06 ns	0.07	0.12	False
AddBenchmark - Duration of single invocation	15.64 ns	1.49 ns	0.10	0.19	False
BitwiseOrOperatorBenchmark - Duration of single invocation	13.21 ns	1.21 ns	0.09	0.19	False
SubtractionOperatorBenchmark - Duration of single invocation	15.64 ns	1.06 ns	0.07	0.15	False
BitwiseOrBenchmark - Duration of single invocation	13.17 ns	1.23 ns	0.09	0.11	False
BitwiseAndOperatorBenchmark - Duration of single invocation	13.17 ns	1.09 ns	0.08	0.15	False
BitwiseAndBenchmark - Duration of single invocation	13.18 ns	1.44 ns	0.11	0.16	False
ExclusiveOrOperatorBenchmark - Duration of single invocation	13.60 ns	0.97 ns	0.07	0.16	False
SubtractBenchmark - Duration of single invocation	15.62 ns	1.27 ns	0.08	0.12	False
AllBitsSetBenchmark - Duration of single invocation	6.51 ns	1.03 ns	0.16	0.10	False
AddOperatorBenchmark - Duration of single invocation	15.60 ns	1.07 ns	0.07	0.18	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Int*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Int* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Int*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Int* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.UnaryNegateOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.098239945902561 < 13.790694516163809.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.4134137822502 (T) = (0 -1.1094934255171505) / Math.Sqrt((18.67717781369693 / (299)) + (0.016998770002741248 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9180113496451009 = (13.532280635363025 - 1.1094934255171505) / 13.532280635363025 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0820237100346686 < 10.541583817521875.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.872983514981755 (T) = (0 -1.0656677931577372) / Math.Sqrt((7.222572762052699 / (299)) + (0.004536071909317293 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8751627748235664 = (8.536458509484001 - 1.0656677931577372) / 8.536458509484001 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.NegateBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3394940819361205 < 13.78219832647317.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.60773712095399 (T) = (0 -1.0897368592791794) / Math.Sqrt((18.644192295045336 / (299)) + (0.007515115686337092 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9193367423927812 = (13.509705553742176 - 1.0897368592791794) / 13.509705553742176 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.XorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2885275180134055 < 12.529088039641618.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.71417662265204 (T) = (0 -1.1050044949486124) / Math.Sqrt((18.888423473681197 / (299)) + (0.026680387514367167 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9179091339765498 = (13.46074841789287 - 1.1050044949486124) / 13.46074841789287 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.MultiplyOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1224207088086053 < 14.965680150404651.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 228.5950609999487 (T) = (0 -1.0723260257927654) / Math.Sqrt((1.2230387763543098 / (299)) + (0.031016388481556675 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9401973120698944 = (17.931067363494545 - 1.0723260257927654) / 17.931067363494545 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.OnesComplementBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.123417695989281 < 10.564398799667762.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.785881543866594 (T) = (0 -1.0653119896288978) / Math.Sqrt((7.128899328555934 / (299)) + (0.01346437817662431 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8751680450474195 = (8.533968646358009 - 1.0653119896288978) / 8.533968646358009 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.MultiplyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.059598909188585 < 14.984526931688531.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 155.81740743785565 (T) = (0 -1.076178857758521) / Math.Sqrt((3.827306802676462 / (299)) + (0.015149165461679064 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9438115434436577 = (19.15302401445028 - 1.076178857758521) / 19.15302401445028 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.AddBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.4864645442823003 < 14.839260701183406.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.94372366705103 (T) = (0 -1.0828475730086817) / Math.Sqrt((33.735282947498035 / (299)) + (0.026479513874154956 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.937295192835753 = (17.26897222046 - 1.0828475730086817) / 17.26897222046 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2110560583393333 < 12.547643076362585.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.62179507193344 (T) = (0 -1.1050623020946573) / Math.Sqrt((15.976744435636366 / (299)) + (0.022572671561091967 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9128526351596278 = (12.680386883972899 - 1.1050623020946573) / 12.680386883972899 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.SubtractionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0623850989384982 < 14.80093492488216.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.859909115826824 (T) = (0 -1.1091802366480719) / Math.Sqrt((27.015754064866364 / (299)) + (0.06244284651078411 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9307409244906246 = (16.014944301385096 - 1.1091802366480719) / 16.014944301385096 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.BitwiseOrBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2349853354044986 < 12.58147359646215.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.203809971717554 (T) = (0 -1.0655329887248635) / Math.Sqrt((18.864265116007143 / (299)) + (0.013199350625969263 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9209580917610899 = (13.48060810354186 - 1.0655329887248635) / 13.48060810354186 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.088578509039951 < 12.533771067269697.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 50.27235620675063 (T) = (0 -1.0737154002499203) / Math.Sqrt((15.78280273695835 / (299)) + (0.009783016949519567 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9152570148455391 = (12.67025699286922 - 1.0737154002499203) / 12.67025699286922 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.BitwiseAndBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.4376010851386947 < 12.511225999492584.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.378572191643016 (T) = (0 -1.1232782825305194) / Math.Sqrt((19.19017452238214 / (299)) + (0.08636998775908816 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9166232452201984 = (13.472319539146161 - 1.1232782825305194) / 13.472319539146161 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9749256800135011 < 12.566794718603818.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 50.043761274835795 (T) = (0 -1.0816784300395637) / Math.Sqrt((15.816066150361157 / (299)) + (0.015441029305786697 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9145875399956547 = (12.664176046264604 - 1.0816784300395637) / 12.664176046264604 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.SubtractBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2659908642747946 < 14.791715219302583.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.205193441196926 (T) = (0 -1.079728464339306) / Math.Sqrt((33.56852688391347 / (299)) + (0.011111140229022485 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9374663896809032 = (17.26637017804763 - 1.079728464339306) / 17.26637017804763 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.AllBitsSetBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.026370005806942 < 6.172567578765395.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 146.4047342040854 (T) = (0 -1.079380885982193) / Math.Sqrt((0.04419297164974591 / (299)) + (0.022151991836221668 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.8217186079667459 = (6.054366491489254 - 1.079380885982193) / 6.054366491489254 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.AddOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0655656741491393 < 14.820070766713506.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.53270989672432 (T) = (0 -1.0715357440179396) / Math.Sqrt((27.033073681003025 / (299)) + (0.009701692736212058 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9330289892247231 = (15.99999360340414 - 1.0715357440179396) / 15.99999360340414 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-11T18:53:54Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
UnaryNegateOperatorBenchmark - Duration of single invocation	38.31 ns	1.07 ns	0.03	0.10	False
NegateBenchmark - Duration of single invocation	38.38 ns	1.09 ns	0.03	0.11	False
AndNotBenchmark - Duration of single invocation	17.69 ns	1.10 ns	0.06	0.12	False
MultiplyOperatorBenchmark - Duration of single invocation	39.46 ns	1.27 ns	0.03	0.06	True
OneBenchmark - Duration of single invocation	20.75 ns	1.30 ns	0.06	0.22	False
OnesComplementOperatorBenchmark - Duration of single invocation	9.13 ns	1.10 ns	0.12	0.15	False
ExclusiveOrOperatorBenchmark - Duration of single invocation	9.46 ns	1.02 ns	0.11	0.16	False
ConditionalSelectBenchmark - Duration of single invocation	32.91 ns	1.14 ns	0.03	0.39	False
BitwiseAndOperatorBenchmark - Duration of single invocation	9.49 ns	1.25 ns	0.13	0.13	False
MultiplyBenchmark - Duration of single invocation	39.84 ns	1.29 ns	0.03	0.07	True
AddBenchmark - Duration of single invocation	38.26 ns	1.00 ns	0.03	0.12	False
SubtractBenchmark - Duration of single invocation	37.52 ns	1.26 ns	0.03	0.12	False
AddOperatorBenchmark - Duration of single invocation	38.72 ns	1.30 ns	0.03	0.11	False
OnesComplementBenchmark - Duration of single invocation	8.60 ns	1.06 ns	0.12	0.19	False
BitwiseOrOperatorBenchmark - Duration of single invocation	9.49 ns	1.31 ns	0.14	0.15	False
SubtractionOperatorBenchmark - Duration of single invocation	38.77 ns	1.17 ns	0.03	0.11	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Byte>.UnaryNegateOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.067288866423577 < 36.27047886972653.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.386783890662734 (T) = (0 -1.207735939443431) / Math.Sqrt((141.26690524692688 / (299)) + (0.07840213566427236 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9657637300799987 = (35.27650477886487 - 1.207735939443431) / 35.27650477886487 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.NegateBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.091042324076078 < 36.3506773123944.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.28111435403784 (T) = (0 -1.2819849512876538) / Math.Sqrt((146.68917782053526 / (299)) + (0.055063559159523705 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9642741625180923 = (35.8839719834945 - 1.2819849512876538) / 35.8839719834945 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.AndNotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1010612779372972 < 16.812694373551818.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 43.89078959795058 (T) = (0 -1.3380165619270235) / Math.Sqrt((51.86886482655031 / (299)) + (0.18323194054756237 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9332116883223772 = (20.033693446024365 - 1.3380165619270235) / 20.033693446024365 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.MultiplyOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2676651507860668 < 37.63266990201574.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 720.7145353794452 (T) = (0 -1.3082871394748725) / Math.Sqrt((0.5201900736382169 / (299)) + (0.02970034673588136 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9680807784678114 = (40.987438811925394 - 1.3082871394748725) / 40.987438811925394 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.OneBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2985982330218249 < 19.62210183171891.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 32.154437965041566 (T) = (0 -1.2540587360700577) / Math.Sqrt((81.72939990221401 / (299)) + (0.017317470598501087 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9306699123942589 = (18.088232387668445 - 1.2540587360700577) / 18.088232387668445 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.104227573292958 < 8.66522850051008.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.63033884437839 (T) = (0 -1.183443983006654) / Math.Sqrt((5.538645025273503 / (299)) + (0.017759816670422698 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.842668273560026 = (7.521966546640343 - 1.183443983006654) / 7.521966546640343 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.018252124931857 < 9.140378522204458.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 43.38245715109771 (T) = (0 -1.1900386051157192) / Math.Sqrt((15.932458526207292 / (299)) + (0.029583159609920057 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.8949647156740118 = (11.329893690031891 - 1.1900386051157192) / 11.329893690031891 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.ConditionalSelectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1413553438817459 < 31.297686144661824.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 41.36039340532526 (T) = (0 -1.262745014388979) / Math.Sqrt((359.30258451208476 / (299)) + (0.02620122262870185 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9729164120166125 = (46.623992920122596 - 1.262745014388979) / 46.623992920122596 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2487838462501253 < 9.0734637137535.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 40.36970564963259 (T) = (0 -1.3070613217523317) / Math.Sqrt((15.971118628230816 / (299)) + (0.18686952116747227 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8845522273038386 = (11.32166772235869 - 1.3070613217523317) / 11.32166772235869 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.MultiplyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2945866788930007 < 37.952103831916986.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 372.0613028591057 (T) = (0 -1.2345089261798623) / Math.Sqrt((3.2822965732421614 / (299)) + (0.033778557295750945 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9711176221829672 = (42.742634765058476 - 1.2345089261798623) / 42.742634765058476 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.AddBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0046203343403097 < 36.31923852310385.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.586813712513674 (T) = (0 -1.2325537724910405) / Math.Sqrt((153.97994560980473 / (299)) + (0.021685071486785893 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9665519206442582 = (36.84976226533181 - 1.2325537724910405) / 36.84976226533181 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.SubtractBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2552997868790465 < 36.61138267899033.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.281585499570426 (T) = (0 -1.3059692591619148) / Math.Sqrt((154.07047826654892 / (299)) + (0.11571614544699091 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9645639509188837 = (36.85425697916915 - 1.3059692591619148) / 36.85425697916915 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.AddOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3035926866773684 < 36.483418784913404.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.76736298017029 (T) = (0 -1.2527004633455856) / Math.Sqrt((140.65761492883055 / (299)) + (0.020005077782898974 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.96463147979072 = (35.41851499393232 - 1.2527004633455856) / 35.41851499393232 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.OnesComplementBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0606536614027071 < 8.671855437943805.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 42.27393150265897 (T) = (0 -1.1908856052206338) / Math.Sqrt((5.555661224429916 / (299)) + (0.08686201492483897 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8414657675957433 = (7.511851460471439 - 1.1908856052206338) / 7.511851460471439 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3110858276217396 < 9.084085667404109.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 42.6950465694097 (T) = (0 -1.2937858039281462) / Math.Sqrt((16.077073111069488 / (299)) + (0.026621864706836064 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8855057812442119 = (11.30000988685502 - 1.2937858039281462) / 11.30000988685502 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Byte>.SubtractionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1715036105472267 < 36.755979401159365.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.78393782287246 (T) = (0 -1.2640224148685533) / Math.Sqrt((140.56924334376873 / (299)) + (0.014157801552459041 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9643145990399838 = (35.421275391716435 - 1.2640224148685533) / 35.421275391716435 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-11T18:53:56Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
OnesComplementOperatorBenchmark - Duration of single invocation	9.11 ns	1.30 ns	0.14	0.15	False
SubtractBenchmark - Duration of single invocation	10.70 ns	0.97 ns	0.09	0.14	False
BitwiseAndOperatorBenchmark - Duration of single invocation	9.53 ns	1.26 ns	0.13	0.15	False
UnaryNegateOperatorBenchmark - Duration of single invocation	11.09 ns	1.10 ns	0.10	0.14	False
AddOperatorBenchmark - Duration of single invocation	10.63 ns	1.35 ns	0.13	0.10	False
SubtractionOperatorBenchmark - Duration of single invocation	10.62 ns	1.25 ns	0.12	0.16	False
NegateBenchmark - Duration of single invocation	10.73 ns	1.29 ns	0.12	0.14	False
ConditionalSelectBenchmark - Duration of single invocation	33.16 ns	0.92 ns	0.03	0.34	False
BitwiseOrOperatorBenchmark - Duration of single invocation	9.67 ns	1.52 ns	0.16	0.18	False
AndNotBenchmark - Duration of single invocation	17.75 ns	1.26 ns	0.07	0.13	False
OnesComplementBenchmark - Duration of single invocation	9.14 ns	1.18 ns	0.13	0.15	False
ExclusiveOrOperatorBenchmark - Duration of single invocation	9.51 ns	1.00 ns	0.10	0.16	False
MultiplyBenchmark - Duration of single invocation	10.98 ns	1.26 ns	0.11	0.09	True
MultiplyOperatorBenchmark - Duration of single invocation	11.03 ns	1.01 ns	0.09	0.08	True
OneBenchmark - Duration of single invocation	8.81 ns	1.22 ns	0.14	0.37	False
AddBenchmark - Duration of single invocation	10.62 ns	1.41 ns	0.13	0.12	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int32>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3035484348598816 < 8.64530261626004.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.1313992368993 (T) = (0 -1.2599324025218182) / Math.Sqrt((5.530911102122433 / (299)) + (0.017810209270442957 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8325737973300608 = (7.525299997430055 - 1.2599324025218182) / 7.525299997430055 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.SubtractBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9734948317147375 < 10.14533678458027.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.48364958260072 (T) = (0 -1.2679446478614762) / Math.Sqrt((13.94280554904296 / (299)) + (0.03385352832563786 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8914643244837004 = (11.68228457444907 - 1.2679446478614762) / 11.68228457444907 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2593534631155838 < 9.065473948456152.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 43.23779221782179 (T) = (0 -1.2649036805897163) / Math.Sqrt((15.872678983761224 / (299)) + (0.026583697516477812 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8884064668778153 = (11.334919194688126 - 1.2649036805897163) / 11.334919194688126 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.UnaryNegateOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0979390256004329 < 10.400363533670077.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.78341473938551 (T) = (0 -1.2533515156141262) / Math.Sqrt((10.65545040062328 / (299)) + (0.01872848890353697 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8792097083794255 = (10.376260366612435 - 1.2533515156141262) / 10.376260366612435 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.AddOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.350774126842148 < 10.251084529127573.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.21386453946231 (T) = (0 -1.1852361935536997) / Math.Sqrt((10.948551964052001 / (299)) + (0.017606224959160834 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8892359093500706 = (10.70054551614246 - 1.1852361935536997) / 10.70054551614246 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.SubtractionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.254356289446951 < 10.212715817474301.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.95054009586892 (T) = (0 -1.2624139736757303) / Math.Sqrt((10.907660660062765 / (299)) + (0.01490816742951176 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8819577456316706 = (10.694593901405822 - 1.2624139736757303) / 10.694593901405822 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.NegateBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2906192588634646 < 10.199875105858093.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.066868589234964 (T) = (0 -1.1770648516331286) / Math.Sqrt((12.145457425897542 / (299)) + (0.011489991947361578 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8922503948067901 = (10.92407577292268 - 1.1770648516331286) / 10.92407577292268 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.ConditionalSelectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9231329527982977 < 31.08837401644105.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 40.90589850288271 (T) = (0 -1.3848730909151903) / Math.Sqrt((366.5470219657404 / (299)) + (0.3418851921464495 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9705032193993477 = (46.94997429260359 - 1.3848730909151903) / 46.94997429260359 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.5238787663355382 < 9.128564719234921.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 42.71474467678351 (T) = (0 -1.2666676503661132) / Math.Sqrt((15.94654464533339 / (299)) + (0.047114818841200605 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8880920594421983 = (11.318836215307387 - 1.2666676503661132) / 11.318836215307387 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.AndNotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2641439213188592 < 16.876489203384804.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.326770912492314 (T) = (0 -1.2669554188837415) / Math.Sqrt((51.171374208213244 / (299)) + (0.012420889467200913 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9368035062513008 = (20.047875186268897 - 1.2669554188837415) / 20.047875186268897 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.OnesComplementBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1780015389835043 < 8.672134293945012.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.69108645232053 (T) = (0 -1.1478807352650562) / Math.Sqrt((5.575825563574982 / (299)) + (0.017341724356969953 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8471994021798499 = (7.5122790855578865 - 1.1478807352650562) / 7.5122790855578865 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9954871464885958 < 9.092387838953137.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 41.07730340325246 (T) = (0 -1.3402647645618506) / Math.Sqrt((16.042542005776944 / (299)) + (0.12048328879503015 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8814848344061832 = (11.308803880469586 - 1.3402647645618506) / 11.308803880469586 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.MultiplyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2626606140770624 < 10.449160192951704.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 108.13662878333892 (T) = (0 -1.3666015708418233) / Math.Sqrt((0.9969342327840981 / (299)) + (0.19241598290676185 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.895387284916382 = (13.063436598022374 - 1.3666015708418233) / 13.063436598022374 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.MultiplyOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.00792851287508 < 10.488522384498525.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 97.18960434482285 (T) = (0 -1.4200724752743359) / Math.Sqrt((0.19088299357228364 / (299)) + (0.2601135046123151 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8820873589243421 = (12.043428612231283 - 1.4200724752743359) / 12.043428612231283 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.OneBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2190789239879096 < 8.346101846083275.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 31.25930159966048 (T) = (0 -1.1495735367021183) / Math.Sqrt((16.834955076156685 / (299)) + (0.013684020145887621 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8664222362997654 = (8.606024721913354 - 1.1495735367021183) / 8.606024721913354 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int32>.AddBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.4081336426422844 < 10.184874980169138.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 47.34127437728857 (T) = (0 -1.2980589588067477) / Math.Sqrt((14.052318545137224 / (299)) + (0.02317195837567698 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8887750545554565 = (11.670574021130504 - 1.2980589588067477) / 11.670574021130504 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-11T18:53:57Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
BitwiseOrOperatorBenchmark - Duration of single invocation	16.07 ns	0.99 ns	0.06	0.14	False
BitwiseOrBenchmark - Duration of single invocation	13.26 ns	1.00 ns	0.08	0.12	False
AllBitsSetBenchmark - Duration of single invocation	6.55 ns	1.11 ns	0.17	0.17	True
AddOperatorBenchmark - Duration of single invocation	37.95 ns	1.09 ns	0.03	0.11	False
SubtractionOperatorBenchmark - Duration of single invocation	38.15 ns	1.16 ns	0.03	0.12	False
EqualityOperatorBenchmark - Duration of single invocation	8.27 ns	0.11 ns	0.01	0.16	False
OnesComplementBenchmark - Duration of single invocation	11.14 ns	1.23 ns	0.11	0.16	False
EqualsAllBenchmark - Duration of single invocation	8.21 ns	0.15 ns	0.02	0.14	False
SubtractBenchmark - Duration of single invocation	38.00 ns	1.13 ns	0.03	0.12	False
UnaryNegateOperatorBenchmark - Duration of single invocation	35.69 ns	1.03 ns	0.03	0.11	False
OnesComplementOperatorBenchmark - Duration of single invocation	11.18 ns	1.01 ns	0.09	0.17	False
DivisionOperatorBenchmark - Duration of single invocation	41.24 ns	1.02 ns	0.02	0.09	True
MultiplyBenchmark - Duration of single invocation	38.70 ns	1.02 ns	0.03	0.06	True
InequalityOperatorBenchmark - Duration of single invocation	8.47 ns	0.10 ns	0.01	0.15	False
NegateBenchmark - Duration of single invocation	35.75 ns	1.08 ns	0.03	0.09	False
DivideBenchmark - Duration of single invocation	41.27 ns	1.06 ns	0.03	0.06	True
BitwiseAndBenchmark - Duration of single invocation	13.48 ns	1.04 ns	0.08	0.12	False
XorBenchmark - Duration of single invocation	13.11 ns	1.19 ns	0.09	0.15	False
MultiplyOperatorBenchmark - Duration of single invocation	38.85 ns	1.01 ns	0.03	0.06	True
BitwiseAndOperatorBenchmark - Duration of single invocation	13.18 ns	1.26 ns	0.10	0.12	False
ExclusiveOrOperatorBenchmark - Duration of single invocation	13.22 ns	1.30 ns	0.10	0.14	False
AddBenchmark - Duration of single invocation	37.91 ns	1.15 ns	0.03	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9890761223798242 < 12.666572654092619.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.34342790581603 (T) = (0 -1.188465742903994) / Math.Sqrt((15.990189849323057 / (299)) + (0.06513154214999282 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9061265829558738 = (12.66030129003765 - 1.188465742903994) / 12.66030129003765 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.BitwiseOrBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0028286319210087 < 12.529268541163265.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.477014260180184 (T) = (0 -1.142870089685792) / Math.Sqrt((19.107202414009823 / (299)) + (0.016283791428929762 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9151239043575183 = (13.465158606020616 - 1.142870089685792) / 13.465158606020616 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.AllBitsSetBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1062294821994132 < 6.205069129533133.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 146.251301947073 (T) = (0 -1.176220062691854) / Math.Sqrt((0.05020148917732999 / (299)) + (0.021816970703117385 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8059999279422705 = (6.062987761890321 - 1.176220062691854) / 6.062987761890321 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.AddOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0853209231761747 < 36.01556834142325.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.330129454445185 (T) = (0 -1.2322635698813138) / Math.Sqrt((142.03487265619867 / (299)) + (0.06771167757581381 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9651283168122081 = (35.33708319283859 - 1.2322635698813138) / 35.33708319283859 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.SubtractionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1587546294668953 < 36.06531326980353.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.416501630417976 (T) = (0 -1.1413045776004718) / Math.Sqrt((143.97553126813537 / (299)) + (0.012690014009478175 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.967807039349767 = (35.45199181896959 - 1.1413045776004718) / 35.45199181896959 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.EqualityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.1054924235175729 < 7.8912593632360535.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 75.78485063762689 (T) = (0 -0.06234628183141613) / Math.Sqrt((5.507314461823998 / (299)) + (0.008379985396430619 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.9940358160109108 = (10.453447101141153 - 0.06234628183141613) / 10.453447101141153 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.OnesComplementBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2343458257604742 < 10.586151926233985.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 46.896238180965256 (T) = (0 -1.1295162672570132) / Math.Sqrt((7.284160645387523 / (299)) + (0.01650138385011523 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8679885232606234 = (8.556197500062504 - 1.1295162672570132) / 8.556197500062504 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.EqualsAllBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.14743217672559647 < 7.829383514760114.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 67.76812639037344 (T) = (0 -0.11352003135352291) / Math.Sqrt((8.486655855319505 / (299)) + (0.011089329053029938 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.9902403893102324 = (11.631614719277925 - 0.11352003135352291) / 11.631614719277925 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.SubtractBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1347368994046856 < 36.0422202013849.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.293393456128975 (T) = (0 -1.1637332823570201) / Math.Sqrt((180.58403195253757 / (299)) + (0.010493724154979527 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9699364033937548 = (38.70905060358854 - 1.1637332823570201) / 38.70905060358854 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.UnaryNegateOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0323082525772656 < 34.014965061319096.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.940602010312936 (T) = (0 -1.1869608475802438) / Math.Sqrt((96.95281901615247 / (299)) + (0.03708657365215341 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9592455440251322 = (29.124688802427162 - 1.1869608475802438) / 29.124688802427162 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0097774085693436 < 10.559453543843766.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 42.45510006354069 (T) = (0 -1.2197410952211838) / Math.Sqrt((7.289067080526882 / (299)) + (0.11968561553385393 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8568673966730617 = (8.521755818519523 - 1.2197410952211838) / 8.521755818519523 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.DivisionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0247489412870103 < 39.1343767069596.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 399.10783942602114 (T) = (0 -1.1484893609187876) / Math.Sqrt((2.9574433129130093 / (299)) + (0.017552938351105433 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9728773186709717 = (42.34424122697655 - 1.1484893609187876) / 42.34424122697655 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.MultiplyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0245599780545975 < 36.87390975348868.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 176.04344548919656 (T) = (0 -1.2174815343637098) / Math.Sqrt((15.315230159158316 / (299)) + (0.14558679536296107 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9719797553072849 = (43.45006789609638 - 1.2174815343637098) / 43.45006789609638 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.InequalityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.09697877397158391 < 8.03293551525679.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 73.21016766833809 (T) = (0 -0.11665526643450362) / Math.Sqrt((5.4305669518699835 / (299)) + (0.04314392550424597 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9888688076797424 = (10.480033322414474 - 0.11665526643450362) / 10.480033322414474 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.NegateBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.079884526052723 < 33.9848262008158.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.91195945486629 (T) = (0 -1.1834065557575175) / Math.Sqrt((96.64003587935184 / (299)) + (0.04949315405171231 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9593094328105977 = (29.08306857088319 - 1.1834065557575175) / 29.08306857088319 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.DivideBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0617009064316538 < 39.151210719091374.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 169.46332506939376 (T) = (0 -1.311983334556331) / Math.Sqrt((16.493919290175356 / (299)) + (0.3259133359936361 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9714375714257699 = (45.93388587901944 - 1.311983334556331) / 45.93388587901944 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.BitwiseAndBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0401480478685807 < 12.515449959247544.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.0336713802878 (T) = (0 -1.1702792151797081) / Math.Sqrt((18.67608555922613 / (299)) + (0.022139405812413953 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.9134618629422011 = (13.523277192783537 - 1.1702792151797081) / 13.523277192783537 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.XorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1854413566202044 < 12.533091898807921.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.18099934647665 (T) = (0 -1.219153807995259) / Math.Sqrt((18.992240357915996 / (299)) + (0.025138071680441812 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9094659372544724 = (13.466244317589577 - 1.219153807995259) / 13.466244317589577 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.MultiplyOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0067371288458768 < 36.84343248878254.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 369.64476243508506 (T) = (0 -1.2036411192144667) / Math.Sqrt((2.8697588601868422 / (299)) + (0.03348482680255792 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9699591834941917 = (40.06685766952262 - 1.2036411192144667) / 40.06685766952262 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2585726646020843 < 12.526482030187418.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.189713913633724 (T) = (0 -1.178625542708572) / Math.Sqrt((16.12256409459062 / (299)) + (0.013614856150188353 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9069274156579799 = (12.663509357142123 - 1.178625542708572) / 12.663509357142123 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3019168485414492 < 12.528406544097669.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.31979799326314 (T) = (0 -1.1627695128945432) / Math.Sqrt((16.0606504167414 / (299)) + (0.017889926921979354 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.908268453001225 = (12.675786585285344 - 1.1627695128945432) / 12.675786585285344 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.AddBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1455804381177663 < 36.00894923207618.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.44153575706583 (T) = (0 -1.1626979381671407) / Math.Sqrt((179.65299763369148 / (299)) + (0.012465532633163627 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.969978413159485 = (38.72873024146891 - 1.1626979381671407) / 38.72873024146891 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-11T18:53:59Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<Single>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
OneBenchmark - Duration of single invocation	11.05 ns	1.01 ns	0.09	0.35	False
AddOperatorBenchmark - Duration of single invocation	29.09 ns	1.07 ns	0.04	0.12	False
EqualityOperatorBenchmark - Duration of single invocation	5.05 ns	0.00 ns	0.00	0.23	False
MultiplyBenchmark - Duration of single invocation	30.13 ns	1.02 ns	0.03	0.06	True
AndNotBenchmark - Duration of single invocation	17.64 ns	1.06 ns	0.06	0.11	False
BitwiseOrOperatorBenchmark - Duration of single invocation	9.37 ns	1.04 ns	0.11	0.16	False
ExclusiveOrOperatorBenchmark - Duration of single invocation	9.46 ns	1.03 ns	0.11	0.14	False
SubtractionOperatorBenchmark - Duration of single invocation	29.21 ns	1.09 ns	0.04	0.09	False
DivisionOperatorBenchmark - Duration of single invocation	32.78 ns	1.05 ns	0.03	0.05	True
MultiplyOperatorBenchmark - Duration of single invocation	29.99 ns	1.06 ns	0.04	0.06	True
AddBenchmark - Duration of single invocation	29.12 ns	0.97 ns	0.03	0.11	False
SubtractBenchmark - Duration of single invocation	29.29 ns	1.01 ns	0.03	0.11	False
ConditionalSelectBenchmark - Duration of single invocation	32.97 ns	0.96 ns	0.03	0.36	False
UnaryNegateOperatorBenchmark - Duration of single invocation	29.25 ns	1.04 ns	0.04	0.09	False
OnesComplementOperatorBenchmark - Duration of single invocation	8.97 ns	1.10 ns	0.12	0.13	False
NegateBenchmark - Duration of single invocation	29.38 ns	1.12 ns	0.04	0.11	False
DivideBenchmark - Duration of single invocation	32.50 ns	1.02 ns	0.03	0.06	True
BitwiseAndOperatorBenchmark - Duration of single invocation	9.58 ns	0.99 ns	0.10	0.12	False
InequalityOperatorBenchmark - Duration of single invocation	4.86 ns	0.22 ns	0.05	0.28	False
OnesComplementBenchmark - Duration of single invocation	8.79 ns	1.12 ns	0.13	0.15	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Single>.OneBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.010252239737983 < 10.459703124878159.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 31.585218688095846 (T) = (0 -1.2403903658588542) / Math.Sqrt((20.84300154445499 / (299)) + (0.029157303919878707 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8715307537072817 = (9.655154067243565 - 1.2403903658588542) / 9.655154067243565 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.AddOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0731603957964606 < 27.644206856758277.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.0946100217073 (T) = (0 -1.1039575812023106) / Math.Sqrt((60.396960732627576 / (299)) + (0.0064895287063534285 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9523837180778285 = (23.184455749962257 - 1.1039575812023106) / 23.184455749962257 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.EqualityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 4.686477406138721.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 62.5265231257179 (T) = (0 -0.07534204430172356) / Math.Sqrt((3.5670842379969927 / (299)) + (0.02959892349082666 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9896276019239324 = (7.263705437179538 - 0.07534204430172356) / 7.263705437179538 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.MultiplyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0247269198176574 < 28.56319031006255.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 289.96949320343265 (T) = (0 -1.1852853953439553) / Math.Sqrt((0.8803812410742657 / (299)) + (0.13153336221759052 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9579308171116038 = (28.174671195500835 - 1.1852853953439553) / 28.174671195500835 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.AndNotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0577056201004447 < 16.70983967550056.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.44149789702103 (T) = (0 -1.104955179789576) / Math.Sqrt((51.87341965492458 / (299)) + (0.005928529577154546 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9448799713552023 = (20.046346254826616 - 1.104955179789576) / 20.046346254826616 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0448080037845986 < 8.91355803273863.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 42.74756923559313 (T) = (0 -1.1377706802702716) / Math.Sqrt((16.1902721650217 / (299)) + (0.06146283038472635 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8995569338180496 = (11.32751839942068 - 1.1377706802702716) / 11.32751839942068 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0254015758292023 < 9.123020374121309.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 41.29667912053534 (T) = (0 -1.2397315948947023) / Math.Sqrt((16.130944469436926 / (299)) + (0.13201504657688273 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8905710142004126 = (11.329097001458058 - 1.2397315948947023) / 11.329097001458058 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.SubtractionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0948025987568406 < 27.632034561310128.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.59668812071798 (T) = (0 -1.1989800294703616) / Math.Sqrt((60.16212692390185 / (299)) + (0.06859004622907679 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9482277329549654 = (23.158731458049136 - 1.1989800294703616) / 23.158731458049136 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.DivisionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0504691487353677 < 30.841898872018476.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 269.91769462867575 (T) = (0 -1.1751108209009808) / Math.Sqrt((3.0785322156326216 / (299)) + (0.012270445569337863 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9598447427528585 = (29.264183607854466 - 1.1751108209009808) / 29.264183607854466 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.MultiplyOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0557494015100939 < 28.586507364659123.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 294.3269977914741 (T) = (0 -1.1376458438594756) / Math.Sqrt((2.062128595711911 / (299)) + (0.024773990844567908 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9585104288171846 = (27.420043433244278 - 1.1376458438594756) / 27.420043433244278 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.AddBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9670815582553184 < 27.630993334695948.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.60158056935345 (T) = (0 -1.1383647014660085) / Math.Sqrt((65.0165599748777 / (299)) + (0.012626307036334885 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9531485975385344 = (24.297345258816858 - 1.1383647014660085) / 24.297345258816858 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.SubtractBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0076610369956103 < 27.68135035152784.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 49.62442074946206 (T) = (0 -1.146965799186609) / Math.Sqrt((64.97127380039348 / (299)) + (0.009862310138637302 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9528039773732112 = (24.30217072011453 - 1.146965799186609) / 24.30217072011453 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.ConditionalSelectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.959985900938347 < 31.30385003474121.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 41.27102907912047 (T) = (0 -1.1498849480583442) / Math.Sqrt((365.8759143665352 / (299)) + (0.1370567684183585 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9754898683756825 = (46.91467861875935 - 1.1498849480583442) / 46.91467861875935 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.UnaryNegateOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.043177493199453 < 27.875397810394258.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.552556838497985 (T) = (0 -1.1173817966266681) / Math.Sqrt((61.944304622923646 / (299)) + (0.011992174950944573 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9519290029382247 = (23.244406501299224 - 1.1173817966266681) / 23.244406501299224 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0959487795270622 < 8.627066160154413.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.94966678622078 (T) = (0 -1.1591230582495935) / Math.Sqrt((5.5809698821365155 / (299)) + (0.011734280818350045 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8459034028777532 = (7.522054866208674 - 1.1591230582495935) / 7.522054866208674 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.NegateBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1195607386802995 < 27.818435235397754.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.53818788681897 (T) = (0 -1.2276795501774518) / Math.Sqrt((65.23351746244576 / (299)) + (0.08084723021420694 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9490192763371171 = (24.081249970001508 - 1.2276795501774518) / 24.081249970001508 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.DivideBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0191263339190342 < 30.782829357533654.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 401.2407511170766 (T) = (0 -1.170140155891863) / Math.Sqrt((1.3402632952411833 / (299)) + (0.016819505544040568 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.9612978648475131 = (30.234511643388615 - 1.170140155891863) / 30.234511643388615 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9850850197155193 < 9.087644459019222.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 43.73391162535451 (T) = (0 -1.1206149598909931) / Math.Sqrt((16.066730597488206 / (299)) + (0.019359254171870434 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9011591678492722 = (11.33757107773137 - 1.1206149598909931) / 11.33757107773137 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.InequalityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.22244670062417002 < 4.624246116396517.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 65.63143082013616 (T) = (0 -0.04975733417374043) / Math.Sqrt((3.505196033801136 / (299)) + (0.005741841756926974 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9931190196336628 = (7.231140262681375 - 0.04975733417374043) / 7.231140262681375 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Single>.OnesComplementBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1196270451561654 < 8.616214800010406.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.9174153216137 (T) = (0 -1.1507238645294533) / Math.Sqrt((5.633219120238884 / (299)) + (0.009475571659482237 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8470161964354856 = (7.521867267760704 - 1.1507238645294533) / 7.521867267760704 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-11T18:54:01Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
BitwiseAndOperatorBenchmark - Duration of single invocation	9.50 ns	1.12 ns	0.12	0.14	False
OneBenchmark - Duration of single invocation	5.74 ns	1.21 ns	0.21	0.30	False
ConditionalSelectBenchmark - Duration of single invocation	32.82 ns	1.01 ns	0.03	0.35	False
OnesComplementOperatorBenchmark - Duration of single invocation	8.77 ns	1.06 ns	0.12	0.18	False
ExclusiveOrOperatorBenchmark - Duration of single invocation	9.54 ns	1.00 ns	0.10	0.12	False
OnesComplementBenchmark - Duration of single invocation	9.17 ns	0.98 ns	0.11	0.17	False
AndNotBenchmark - Duration of single invocation	17.66 ns	1.02 ns	0.06	0.09	False
BitwiseOrOperatorBenchmark - Duration of single invocation	9.59 ns	1.08 ns	0.11	0.13	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int64>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1152195887562462 < 9.102688779752945.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 41.88316237682448 (T) = (0 -1.2256458386576279) / Math.Sqrt((16.172842232887465 / (299)) + (0.09633737489300231 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8918861556786153 = (11.336622486702185 - 1.2256458386576279) / 11.336622486702185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int64>.OneBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.207363262377634 < 5.54078713967165.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 29.588957218517493 (T) = (0 -1.289226916384479) / Math.Sqrt((9.755755988175489 / (299)) + (0.025042101315622785 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8082188448166236 = (6.722385810804779 - 1.289226916384479) / 6.722385810804779 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int64>.ConditionalSelectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.005026018167203 < 31.146610066789968.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 41.53738260231109 (T) = (0 -1.0992953805005816) / Math.Sqrt((358.81502336889184 / (299)) + (0.024415977788517854 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.976421249878185 = (46.622292310715494 - 1.0992953805005816) / 46.622292310715494 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int64>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.062626449903031 < 8.608483426243263.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.582457762215846 (T) = (0 -1.1726857082646225) / Math.Sqrt((5.650069939291198 / (299)) + (0.010605282478631099 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8439468235521265 = (7.514654523269733 - 1.1726857082646225) / 7.514654523269733 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int64>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.99682157486541 < 9.013208898625326.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 43.85647619496421 (T) = (0 -1.1134456725954107) / Math.Sqrt((16.06361956341404 / (299)) + (0.010174177940904493 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9016434274888516 = (11.320500950449501 - 1.1134456725954107) / 11.320500950449501 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int64>.OnesComplementBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9817457618989923 < 8.644797905872787.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.4164385822807 (T) = (0 -1.1861585725799106) / Math.Sqrt((5.541920159186196 / (299)) + (0.019364645671558704 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8420172294491832 = (7.508151480343677 - 1.1861585725799106) / 7.508151480343677 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int64>.AndNotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0209594389874934 < 16.762803839115566.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.377191061630604 (T) = (0 -1.1269290556947584) / Math.Sqrt((51.865593419242344 / (299)) + (0.007975391089328801 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9437798573459937 = (20.04493411961225 - 1.1269290556947584) / 20.04493411961225 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<Int64>.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0793403037583973 < 9.088040052525947.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 43.72548658916773 (T) = (0 -1.1242027458092374) / Math.Sqrt((16.13944838962675 / (299)) + (0.008819065345962109 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.900680416163016 = (11.319044063398636 - 1.1242027458092374) / 11.319044063398636 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-11T18:54:02Z

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	8b1d1eabe32ba781ffcce2867333dfdc53bdd635
Compare	e0c94f84b47fdd48435a625446123f89268b9c20
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
BitwiseAndOperatorBenchmark - Duration of single invocation	9.58 ns	1.01 ns	0.11	0.15	False
UnaryNegateOperatorBenchmark - Duration of single invocation	10.67 ns	1.07 ns	0.10	0.12	False
ConditionalSelectBenchmark - Duration of single invocation	32.92 ns	1.11 ns	0.03	0.35	False
OnesComplementOperatorBenchmark - Duration of single invocation	9.18 ns	1.28 ns	0.14	0.14	False
SubtractionOperatorBenchmark - Duration of single invocation	10.69 ns	1.07 ns	0.10	0.14	False
MultiplyOperatorBenchmark - Duration of single invocation	11.07 ns	1.16 ns	0.11	0.09	True
BitwiseOrOperatorBenchmark - Duration of single invocation	9.53 ns	1.07 ns	0.11	0.15	False
OnesComplementBenchmark - Duration of single invocation	9.01 ns	1.08 ns	0.12	0.15	False
AndNotBenchmark - Duration of single invocation	17.68 ns	1.04 ns	0.06	0.13	False
OneBenchmark - Duration of single invocation	9.00 ns	1.22 ns	0.14	0.33	False
AddBenchmark - Duration of single invocation	10.86 ns	0.99 ns	0.09	0.12	False
NegateBenchmark - Duration of single invocation	11.09 ns	1.15 ns	0.10	0.11	False
MultiplyBenchmark - Duration of single invocation	11.03 ns	1.04 ns	0.09	0.08	True
SubtractBenchmark - Duration of single invocation	10.80 ns	1.12 ns	0.10	0.13	False
AddOperatorBenchmark - Duration of single invocation	11.08 ns	0.99 ns	0.09	0.12	False
ExclusiveOrOperatorBenchmark - Duration of single invocation	9.52 ns	1.03 ns	0.11	0.13	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt32>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0060859092702796 < 9.102710834638344.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 43.69172304016606 (T) = (0 -1.1215842748190703) / Math.Sqrt((16.08856739188478 / (299)) + (0.014363902429821452 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9008780447118715 = (11.315195221470768 - 1.1215842748190703) / 11.315195221470768 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.UnaryNegateOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.068871947799924 < 10.197940155901296.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.200824847505366 (T) = (0 -1.1430746381801786) / Math.Sqrt((10.757615734196433 / (299)) + (0.01862902285395077 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8899637517969168 = (10.388164417152044 - 1.1430746381801786) / 10.388164417152044 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.ConditionalSelectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1136199482225264 < 31.190737492989246.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 41.303817105829225 (T) = (0 -1.1686019266010865) / Math.Sqrt((364.38731413948545 / (299)) + (0.1272537877022563 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9750666252644956 = (46.86898340067186 - 1.1686019266010865) / 46.86898340067186 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2780429881177606 < 8.646500611523468.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 46.170102572270395 (T) = (0 -1.1357605362982457) / Math.Sqrt((5.594380259645179 / (299)) + (0.007567574785888761 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8486954423905961 = (7.506452906925888 - 1.1357605362982457) / 7.506452906925888 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.SubtractionOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.068808248405259 < 10.180242846072483.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 44.950227861688504 (T) = (0 -1.2939919436149) / Math.Sqrt((10.879456612227251 / (299)) + (0.16915680264744684 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8790098232018717 = (10.695016553070435 - 1.2939919436149) / 10.695016553070435 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.MultiplyOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1643150078783584 < 10.527685627205.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 112.56288978539975 (T) = (0 -1.2160756627761233) / Math.Sqrt((0.18658207233256327 / (299)) + (0.1983908316448334 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.899012615889441 = (12.041857242730323 - 1.2160756627761233) / 12.041857242730323 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.073551418180387 < 9.02410440779617.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 43.96483824667346 (T) = (0 -1.1046898522839468) / Math.Sqrt((16.05526737274017 / (299)) + (0.0053176524195028925 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.902363894136634 = (11.314357967429313 - 1.1046898522839468) / 11.314357967429313 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.OnesComplementBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0829400164653953 < 8.60081149630128.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.34539663555495 (T) = (0 -1.1571374806815056) / Math.Sqrt((5.617304521704531 / (299)) + (0.01992968386766101 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8460054982559883 = (7.514148022018602 - 1.1571374806815056) / 7.514148022018602 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.AndNotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.042697804176136 < 16.755661753216543.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 45.51455953651413 (T) = (0 -1.1102457996372241) / Math.Sqrt((51.66561684684393 / (299)) + (0.008197053690973274 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.9446272222130524 = (20.05039017383246 - 1.1102457996372241) / 20.05039017383246 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.OneBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2182211230405287 < 8.26473694690517.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 30.934875753858098 (T) = (0 -1.2226125122659286) / Math.Sqrt((16.6844366961198 / (299)) + (0.0102031002087453 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8571563765768888 = (8.559097584947695 - 1.2226125122659286) / 8.559097584947695 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.AddBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9860357713181541 < 10.266838124526286.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.803593685127595 (T) = (0 -1.1254106971047626) / Math.Sqrt((13.893175562530711 / (299)) + (0.008976339582556394 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9037248841907404 = (11.689528365091011 - 1.1254106971047626) / 11.689528365091011 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.NegateBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.146352770536958 < 10.33673275202075.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.02464071524457 (T) = (0 -1.1746472373228978) / Math.Sqrt((12.245780468478074 / (299)) + (0.00893801985567747 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8926248069279771 = (10.939651922534766 - 1.1746472373228978) / 10.939651922534766 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.MultiplyBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0406595315816725 < 10.497477292157845.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 107.17271416097635 (T) = (0 -1.2348460954630263) / Math.Sqrt((1.0304238968549029 / (299)) + (0.20094586655307609 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9054787304616931 = (13.064214028172547 - 1.2348460954630263) / 13.064214028172547 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.SubtractBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1186994270278185 < 10.22948869295336.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.09689830878424 (T) = (0 -1.1283785472301433) / Math.Sqrt((14.11291167259068 / (299)) + (0.01850121624630808 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9032796698128869 = (11.666405036533753 - 1.1283785472301433) / 11.666405036533753 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.AddOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9852174806015588 < 10.305598995308962.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 48.90876860727581 (T) = (0 -1.1987765010289504) / Math.Sqrt((11.000399065375424 / (299)) + (0.021884523453932363 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8879694897367575 = (10.700446674857924 - 1.1987765010289504) / 10.700446674857924 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Numerics.Tests.Perf_VectorOf<UInt32>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.034942601427347 < 9.066251950557513.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/15/2023 10:57:55 AM, 3/24/2023 12:04:20 PM, 4/3/2023 11:26:06 PM, 4/11/2023 5:52:15 AM falls between 4/2/2023 3:55:31 PM and 4/11/2023 5:52:15 AM.
IsImprovementStdDev: Marked as improvement because 42.42390362088458 (T) = (0 -1.174529658490979) / Math.Sqrt((16.23356259094096 / (299)) + (0.06149746621860522 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8960605615128656 = (11.300134728324142 - 1.174529658490979) / 11.300134728324142 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

lewing · 2023-04-11T19:25:11Z

looks like this is dotnet/runtime#84180

cc @fanyang-mono @SamMonoRT

kotlarmilos · 2023-04-11T20:57:17Z

Similar to #15718.

performanceautofiler bot added ampere arch-arm64 branch-refs/heads/main kind-micro_mono mono-aot os-linux perf-improvement runtime-mono untriaged labels Apr 11, 2023

kotlarmilos mentioned this issue Apr 12, 2023

.NET 8 Per-Preview Performance report on WASM, Mono AOT, and Interpreter dotnet/runtime#84302

Closed

kotlarmilos removed the untriaged label Apr 19, 2023

kotlarmilos closed this as completed Apr 19, 2023

[Perf] Linux/arm64: 398 Improvements on 4/4/2023 3:37:20 PM #15797

[Perf] Linux/arm64: 398 Improvements on 4/4/2023 3:37:20 PM #15797

Comments

performanceautofiler bot commented Apr 11, 2023 • edited Loading

Run Information

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>

Repro

Payloads

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Payloads

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.ExclusiveOrOperatorBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.AddOperatorBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.UnaryNegateOperatorBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.AddBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.SubtractionOperatorBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.AllBitsSetBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.MultiplyBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.XorBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.SubtractBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.BitwiseAndOperatorBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.BitwiseOrBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.OnesComplementBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.MultiplyOperatorBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.EqualityOperatorBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.NegateBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.OnesComplementOperatorBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.BitwiseOrOperatorBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.BitwiseAndBenchmark

Description of detection logic

Docs

performanceautofiler bot commented Apr 11, 2023 • edited Loading

Run Information

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>

Repro

Payloads

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

Payloads

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.BitwiseAndBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.AddBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.OnesComplementOperatorBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.NegateBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.BitwiseOrOperatorBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.OnesComplementBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.XorBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.AddOperatorBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.BitwiseAndOperatorBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.AllBitsSetBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.EqualityOperatorBenchmark

Description of detection logic

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.UnaryNegateOperatorBenchmark

Description of detection logic

performanceautofiler bot commented Apr 11, 2023 •

edited

Loading

performanceautofiler bot commented Apr 11, 2023 •

edited

Loading

performanceautofiler bot commented Apr 11, 2023 •

edited

Loading

performanceautofiler bot commented Apr 11, 2023 •

edited

Loading