Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Windows/x64: 41 Improvements on 2/9/2023 4:10:34 PM #12986

Closed
performanceautofiler bot opened this issue Feb 14, 2023 · 3 comments
Closed

[Perf] Windows/x64: 41 Improvements on 2/9/2023 4:10:34 PM #12986

performanceautofiler bot opened this issue Feb 14, 2023 · 3 comments
Assignees
@EgorBo
Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-improvement runtime-coreclr untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Feb 14, 2023 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Scoped - Duration of single invocation 57.15 ns 52.61 ns 0.92 0.19 False 526.139899141428 512.7997869872003 0.9746453135829528) Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable.Scoped

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 52.606187482373144 < 54.35935317409639.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 16.60421872080435 (T) = (0 -53.1979112016808) / Math.Sqrt((2.907363344328133 / (299)) + (1.7996988957777353 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.08793790297452281 = (58.32707156143864 - 53.1979112016808) / 58.32707156143864 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable.Scoped()
       sub       rsp,28
       mov       rcx,[rcx+8]
       test      rcx,rcx
       je        short M00_L00
       mov       r11,7FFCDC580678
       mov       rdx,18B7997A1D8
       call      qword ptr [r11]
       mov       rdx,rax
       mov       rcx,offset MT_System.Collections.Generic.IEnumerable`1[[Microsoft.Extensions.DependencyInjection.ServiceProviderEngineBenchmark+A, MicroBenchmarks]]
       call      qword ptr [7FFCDC72D858]; System.Runtime.CompilerServices.CastHelpers.ChkCastAny(Void*, System.Object)
       nop
       add       rsp,28
       ret
M00_L00:
       mov       ecx,2F1
       mov       rdx,7FFCDCC2AC10
       call      CORINFO_HELP_STRCNS
       mov       rcx,rax
       call      qword ptr [7FFCDCC3BED0]
       int       3
; Total bytes of code 91

; System.Runtime.CompilerServices.CastHelpers.ChkCastAny(Void*, System.Object)
       push      rdi
       push      rsi
       sub       rsp,28
       test      rdx,rdx
       je        near ptr M01_L03
       mov       rax,[rdx]
       cmp       rax,rcx
       je        near ptr M01_L03
       mov       r8,14AE5000BA0
       mov       r8,[r8]
       cmp       [r8],r8b
       add       r8,10
       rorx      r9,rax,20
       xor       r9,rcx
       mov       r10,9E3779B97F4A7C15
       imul      r9,r10
       mov       r10d,[r8]
       shrx      r9,r9,r10
       xor       r10d,r10d
M01_L00:
       lea       r11d,[r9+1]
       movsxd    r11,r11d
       lea       r11,[r11+r11*2]
       lea       r11,[r8+r11*8]
       mov       esi,[r11]
       mov       rdi,[r11+8]
       and       esi,0FFFFFFFE
       cmp       rdi,rax
       jne       short M01_L01
       mov       rdi,rcx
       xor       rdi,[r11+10]
       cmp       rdi,1
       jbe       short M01_L02
M01_L01:
       test      esi,esi
       je        short M01_L04
       inc       r10d
       add       r9d,r10d
       and       r9d,[r8+4]
       cmp       r10d,8
       jl        short M01_L00
       jmp       short M01_L04
M01_L02:
       cmp       esi,[r11]
       jne       short M01_L04
       cmp       edi,1
       jne       short M01_L04
M01_L03:
       mov       rax,rdx
       add       rsp,28
       pop       rsi
       pop       rdi
       ret
M01_L04:
       call      System.Runtime.CompilerServices.CastHelpers.ChkCastAny_NoCacheLookup(Void*, System.Object)
       nop
       add       rsp,28
       pop       rsi
       pop       rdi
       ret
; Total bytes of code 179

Compare Jit Disasm

; Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable.Scoped()
       sub       rsp,28
       mov       rcx,[rcx+8]
       test      rcx,rcx
       je        short M00_L00
       mov       r11,7FF8669A0680
       mov       rdx,211DD61B7F8
       call      qword ptr [r11]
       mov       rdx,rax
       mov       rcx,offset MT_System.Collections.Generic.IEnumerable`1[[Microsoft.Extensions.DependencyInjection.ServiceProviderEngineBenchmark+A, MicroBenchmarks]]
       call      qword ptr [7FF866B4D858]; System.Runtime.CompilerServices.CastHelpers.ChkCastAny(Void*, System.Object)
       nop
       add       rsp,28
       ret
M00_L00:
       mov       ecx,2F1
       mov       rdx,7FF86704BE10
       call      CORINFO_HELP_STRCNS
       mov       rcx,rax
       call      qword ptr [7FF86705BED0]
       int       3
; Total bytes of code 91
; System.Runtime.CompilerServices.CastHelpers.ChkCastAny(Void*, System.Object)
       push      rdi
       push      rsi
       sub       rsp,28
       test      rdx,rdx
       je        near ptr M01_L03
       mov       rax,[rdx]
       cmp       rax,rcx
       je        near ptr M01_L03
       mov       r8,1D148C00BA8
       mov       r8,[r8]
       cmp       [r8],r8b
       add       r8,10
       rorx      r9,rax,20
       xor       r9,rcx
       mov       r10,9E3779B97F4A7C15
       imul      r9,r10
       mov       r10d,[r8]
       shrx      r9,r9,r10
       xor       r10d,r10d
M01_L00:
       lea       r11d,[r9+1]
       movsxd    r11,r11d
       lea       r11,[r11+r11*2]
       lea       r11,[r8+r11*8]
       mov       esi,[r11]
       mov       rdi,[r11+8]
       and       esi,0FFFFFFFE
       cmp       rdi,rax
       jne       short M01_L01
       mov       rdi,rcx
       xor       rdi,[r11+10]
       cmp       rdi,1
       jbe       short M01_L02
M01_L01:
       test      esi,esi
       je        short M01_L04
       inc       r10d
       add       r9d,r10d
       and       r9d,[r8+4]
       cmp       r10d,8
       jl        short M01_L00
       jmp       short M01_L04
M01_L02:
       cmp       esi,[r11]
       jne       short M01_L04
       cmp       edi,1
       jne       short M01_L04
M01_L03:
       mov       rax,rdx
       add       rsp,28
       pop       rsi
       pop       rdi
       ret
M01_L04:
       call      System.Runtime.CompilerServices.CastHelpers.ChkCastAny_NoCacheLookup(Void*, System.Object)
       nop
       add       rsp,28
       pop       rsi
       pop       rdi
       ret
; Total bytes of code 179

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in Microsoft.Extensions.DependencyInjection.GetService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ServiceScope - Duration of single invocation 54.64 ns 47.13 ns 0.86 0.25 False Trace Trace
EmptyEnumerable - Duration of single invocation 27.01 ns 24.11 ns 0.89 0.23 False Trace Trace
ServiceScopeProvider - Duration of single invocation 25.99 ns 23.46 ns 0.90 0.06 False
Transient - Duration of single invocation 31.33 ns 26.45 ns 0.84 0.22 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetService*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.GetService.ServiceScope

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 47.1332375677146 < 48.6127887132489.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 13.637535656216233 (T) = (0 -47.397147380668656) / Math.Sqrt((3.6725320873556293 / (299)) + (0.8472054239333758 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.061918944627840644 = (50.525641797408504 - 47.397147380668656) / 50.525641797408504 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; Microsoft.Extensions.DependencyInjection.GetService.ServiceScope()
       sub       rsp,28
       mov       rcx,[rcx+28]
       test      rcx,rcx
       je        short M00_L00
       mov       rdx,27C201798E0
       call      qword ptr [7FFA14753528]; Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(System.IServiceProvider, System.Type)
       mov       rdx,rax
       mov       rcx,offset MT_Microsoft.Extensions.DependencyInjection.IServiceScopeFactory
       call      qword ptr [7FFA141BD888]; System.Runtime.CompilerServices.CastHelpers.ChkCastInterface(Void*, System.Object)
       mov       rcx,rax
       mov       r11,7FFA140105D0
       cmp       [rcx],ecx
       add       rsp,28
       jmp       qword ptr [r11]
M00_L00:
       mov       ecx,2F1
       mov       rdx,7FFA14588E18
       call      CORINFO_HELP_STRCNS
       mov       rcx,rax
       call      qword ptr [7FFA14753618]
       int       3
; Total bytes of code 100

; Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(System.IServiceProvider, System.Type)
       push      rdi
       push      rsi
       sub       rsp,28
       mov       rsi,rcx
       mov       rdi,rdx
       test      rsi,rsi
       je        short M01_L01
       test      rdi,rdi
       je        short M01_L02
       mov       rdx,rsi
       mov       rcx,offset MT_Microsoft.Extensions.DependencyInjection.ISupportRequiredService
       call      qword ptr [7FFA141BD810]; System.Runtime.CompilerServices.CastHelpers.IsInstanceOfInterface(Void*, System.Object)
       test      rax,rax
       je        short M01_L00
       mov       rcx,rax
       mov       rdx,rdi
       mov       r11,7FFA140105E8
       add       rsp,28
       pop       rsi
       pop       rdi
       jmp       qword ptr [r11]
M01_L00:
       mov       rcx,rsi
       mov       rdx,rdi
       mov       r11,7FFA140105E0
       call      qword ptr [r11]
       test      rax,rax
       je        short M01_L03
       add       rsp,28
       pop       rsi
       pop       rdi
       ret
M01_L01:
       mov       ecx,2F1
       mov       rdx,7FFA14588E18
       call      CORINFO_HELP_STRCNS
       mov       rcx,rax
       call      qword ptr [7FFA14753618]
       int       3
M01_L02:
       mov       ecx,3AB
       mov       rdx,7FFA14588E18
       call      CORINFO_HELP_STRCNS
       mov       rcx,rax
       call      qword ptr [7FFA14753618]
       int       3
M01_L03:
       mov       rcx,offset MT_System.InvalidOperationException
       call      CORINFO_HELP_NEWSFAST
       mov       rsi,rax
       call      qword ptr [7FFA14753798]
       mov       rcx,rax
       mov       rdx,rdi
       call      qword ptr [7FFA14753678]
       mov       rdx,rax
       mov       rcx,rsi
       call      qword ptr [7FFA143111C8]
       mov       rcx,rsi
       call      CORINFO_HELP_THROW
       int       3
; Total bytes of code 219
; System.Runtime.CompilerServices.CastHelpers.ChkCastInterface(Void*, System.Object)
       test      rdx,rdx
       je        short M02_L01
       mov       rax,[rdx]
       movzx     r8d,word ptr [rax+0E]
       test      r8,r8
       je        short M02_L04
       mov       rax,[rax+38]
       cmp       r8,4
       jl        short M02_L03
M02_L00:
       cmp       [rax],rcx
       je        short M02_L01
       cmp       [rax+8],rcx
       je        short M02_L01
       cmp       [rax+10],rcx
       je        short M02_L01
       cmp       [rax+18],rcx
       jne       short M02_L02
M02_L01:
       mov       rax,rdx
       ret
M02_L02:
       add       rax,20
       add       r8,0FFFFFFFFFFFFFFFC
       cmp       r8,4
       jge       short M02_L00
       test      r8,r8
       je        short M02_L04
M02_L03:
       cmp       [rax],rcx
       je        short M02_L01
       add       rax,8
       dec       r8
       test      r8,r8
       jg        short M02_L03
M02_L04:
       jmp       qword ptr [7FFA141BD870]
; Total bytes of code 97

Compare Jit Disasm

; Microsoft.Extensions.DependencyInjection.GetService.ServiceScope()
       sub       rsp,28
       mov       rcx,[rcx+28]
       test      rcx,rcx
       je        short M00_L00
       mov       rdx,2541687AE08
       call      qword ptr [7FF919173EE8]; Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(System.IServiceProvider, System.Type)
       mov       rdx,rax
       mov       rcx,offset MT_Microsoft.Extensions.DependencyInjection.IServiceScopeFactory
       call      qword ptr [7FF918BDD888]; System.Runtime.CompilerServices.CastHelpers.ChkCastInterface(Void*, System.Object)
       mov       rcx,rax
       mov       r11,7FF918A305D8
       cmp       [rcx],ecx
       add       rsp,28
       jmp       qword ptr [r11]
M00_L00:
       mov       ecx,2F1
       mov       rdx,7FF918FA9328
       call      CORINFO_HELP_STRCNS
       mov       rcx,rax
       call      qword ptr [7FF919173FD8]
       int       3
; Total bytes of code 100
; Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(System.IServiceProvider, System.Type)
       push      rdi
       push      rsi
       sub       rsp,28
       mov       rsi,rcx
       mov       rdi,rdx
       test      rsi,rsi
       je        short M01_L01
       test      rdi,rdi
       je        short M01_L02
       mov       rdx,rsi
       mov       rcx,offset MT_Microsoft.Extensions.DependencyInjection.ISupportRequiredService
       call      qword ptr [7FF918BDD810]; System.Runtime.CompilerServices.CastHelpers.IsInstanceOfInterface(Void*, System.Object)
       test      rax,rax
       je        short M01_L00
       mov       rcx,rax
       mov       rdx,rdi
       mov       r11,7FF918A305F0
       add       rsp,28
       pop       rsi
       pop       rdi
       jmp       qword ptr [r11]
M01_L00:
       mov       rcx,rsi
       mov       rdx,rdi
       mov       r11,7FF918A305E8
       call      qword ptr [r11]
       test      rax,rax
       je        short M01_L03
       add       rsp,28
       pop       rsi
       pop       rdi
       ret
M01_L01:
       mov       ecx,2F1
       mov       rdx,7FF918FA9328
       call      CORINFO_HELP_STRCNS
       mov       rcx,rax
       call      qword ptr [7FF919173FD8]
       int       3
M01_L02:
       mov       ecx,3AB
       mov       rdx,7FF918FA9328
       call      CORINFO_HELP_STRCNS
       mov       rcx,rax
       call      qword ptr [7FF919173FD8]
       int       3
M01_L03:
       mov       rcx,offset MT_System.InvalidOperationException
       call      CORINFO_HELP_NEWSFAST
       mov       rsi,rax
       call      qword ptr [7FF919175168]
       mov       rcx,rax
       mov       rdx,rdi
       call      qword ptr [7FF919175048]
       mov       rdx,rax
       mov       rcx,rsi
       call      qword ptr [7FF918D311C8]
       mov       rcx,rsi
       call      CORINFO_HELP_THROW
       int       3
; Total bytes of code 219
; System.Runtime.CompilerServices.CastHelpers.ChkCastInterface(Void*, System.Object)
       test      rdx,rdx
       je        short M02_L01
       mov       rax,[rdx]
       movzx     r8d,word ptr [rax+0E]
       test      r8,r8
       je        short M02_L04
       mov       rax,[rax+38]
       cmp       r8,4
       jl        short M02_L03
M02_L00:
       cmp       [rax],rcx
       je        short M02_L01
       cmp       [rax+8],rcx
       je        short M02_L01
       cmp       [rax+10],rcx
       je        short M02_L01
       cmp       [rax+18],rcx
       jne       short M02_L02
M02_L01:
       mov       rax,rdx
       ret
M02_L02:
       add       rax,20
       add       r8,0FFFFFFFFFFFFFFFC
       cmp       r8,4
       jge       short M02_L00
       test      r8,r8
       je        short M02_L04
M02_L03:
       cmp       [rax],rcx
       je        short M02_L01
       add       rax,8
       dec       r8
       test      r8,r8
       jg        short M02_L03
M02_L04:
       jmp       qword ptr [7FF918BDD870]
; Total bytes of code 97

Microsoft.Extensions.DependencyInjection.GetService.EmptyEnumerable

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.10781360641537 < 25.063845523067883.
IsChangePoint: Marked as a change because one of 1/12/2023 4:43:01 PM, 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 29.88902044864413 (T) = (0 -22.80649585541591) / Math.Sqrt((1.4028093314339316 / (299)) + (0.3394533826328905 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.15914437058045205 = (27.122962679288346 - 22.80649585541591) / 27.122962679288346 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; Microsoft.Extensions.DependencyInjection.GetService.EmptyEnumerable()
       sub       rsp,28
       mov       rcx,[rcx+30]
       test      rcx,rcx
       je        short M00_L00
       mov       r11,7FFC0BE005D8
       mov       rdx,28F6233A0F0
       call      qword ptr [r11]
       mov       rdx,rax
       mov       rcx,offset MT_System.Collections.Generic.IEnumerable`1[[Microsoft.Extensions.DependencyInjection.ServiceProviderEngineBenchmark+A, MicroBenchmarks]]
       call      qword ptr [7FFC0BFAD858]; System.Runtime.CompilerServices.CastHelpers.ChkCastAny(Void*, System.Object)
       nop
       add       rsp,28
       ret
M00_L00:
       mov       ecx,2F1
       mov       rdx,7FFC0C4AAD68
       call      CORINFO_HELP_STRCNS
       mov       rcx,rax
       call      qword ptr [7FFC0C543630]
       int       3
; Total bytes of code 91

; System.Runtime.CompilerServices.CastHelpers.ChkCastAny(Void*, System.Object)
       push      rdi
       push      rsi
       sub       rsp,28
       test      rdx,rdx
       je        near ptr M01_L03
       mov       rax,[rdx]
       cmp       rax,rcx
       je        near ptr M01_L03
       mov       r8,24ECD800BA0
       mov       r8,[r8]
       cmp       [r8],r8b
       add       r8,10
       rorx      r9,rax,20
       xor       r9,rcx
       mov       r10,9E3779B97F4A7C15
       imul      r9,r10
       mov       r10d,[r8]
       shrx      r9,r9,r10
       xor       r10d,r10d
M01_L00:
       lea       r11d,[r9+1]
       movsxd    r11,r11d
       lea       r11,[r11+r11*2]
       lea       r11,[r8+r11*8]
       mov       esi,[r11]
       mov       rdi,[r11+8]
       and       esi,0FFFFFFFE
       cmp       rdi,rax
       jne       short M01_L01
       mov       rdi,rcx
       xor       rdi,[r11+10]
       cmp       rdi,1
       jbe       short M01_L02
M01_L01:
       test      esi,esi
       je        short M01_L04
       inc       r10d
       add       r9d,r10d
       and       r9d,[r8+4]
       cmp       r10d,8
       jl        short M01_L00
       jmp       short M01_L04
M01_L02:
       cmp       esi,[r11]
       jne       short M01_L04
       cmp       edi,1
       jne       short M01_L04
M01_L03:
       mov       rax,rdx
       add       rsp,28
       pop       rsi
       pop       rdi
       ret
M01_L04:
       call      System.Runtime.CompilerServices.CastHelpers.ChkCastAny_NoCacheLookup(Void*, System.Object)
       nop
       add       rsp,28
       pop       rsi
       pop       rdi
       ret
; Total bytes of code 179

Compare Jit Disasm

; Microsoft.Extensions.DependencyInjection.GetService.EmptyEnumerable()
       sub       rsp,28
       mov       rcx,[rcx+30]
       test      rcx,rcx
       je        short M00_L00
       mov       r11,7FFA626705E0
       mov       rdx,2CC78D4B6E8
       call      qword ptr [r11]
       mov       rdx,rax
       mov       rcx,offset MT_System.Collections.Generic.IEnumerable`1[[Microsoft.Extensions.DependencyInjection.ServiceProviderEngineBenchmark+A, MicroBenchmarks]]
       call      qword ptr [7FFA6281D858]; System.Runtime.CompilerServices.CastHelpers.ChkCastAny(Void*, System.Object)
       nop
       add       rsp,28
       ret
M00_L00:
       mov       ecx,2F1
       mov       rdx,7FFA62D1BF68
       call      CORINFO_HELP_STRCNS
       mov       rcx,rax
       call      qword ptr [7FFA62DB5018]
       int       3
; Total bytes of code 91
; System.Runtime.CompilerServices.CastHelpers.ChkCastAny(Void*, System.Object)
       push      rdi
       push      rsi
       sub       rsp,28
       test      rdx,rdx
       je        near ptr M01_L03
       mov       rax,[rdx]
       cmp       rax,rcx
       je        near ptr M01_L03
       mov       r8,28BE4400BA8
       mov       r8,[r8]
       cmp       [r8],r8b
       add       r8,10
       rorx      r9,rax,20
       xor       r9,rcx
       mov       r10,9E3779B97F4A7C15
       imul      r9,r10
       mov       r10d,[r8]
       shrx      r9,r9,r10
       xor       r10d,r10d
M01_L00:
       lea       r11d,[r9+1]
       movsxd    r11,r11d
       lea       r11,[r11+r11*2]
       lea       r11,[r8+r11*8]
       mov       esi,[r11]
       mov       rdi,[r11+8]
       and       esi,0FFFFFFFE
       cmp       rdi,rax
       jne       short M01_L01
       mov       rdi,rcx
       xor       rdi,[r11+10]
       cmp       rdi,1
       jbe       short M01_L02
M01_L01:
       test      esi,esi
       je        short M01_L04
       inc       r10d
       add       r9d,r10d
       and       r9d,[r8+4]
       cmp       r10d,8
       jl        short M01_L00
       jmp       short M01_L04
M01_L02:
       cmp       esi,[r11]
       jne       short M01_L04
       cmp       edi,1
       jne       short M01_L04
M01_L03:
       mov       rax,rdx
       add       rsp,28
       pop       rsi
       pop       rdi
       ret
M01_L04:
       call      System.Runtime.CompilerServices.CastHelpers.ChkCastAny_NoCacheLookup(Void*, System.Object)
       nop
       add       rsp,28
       pop       rsi
       pop       rdi
       ret
; Total bytes of code 179

Microsoft.Extensions.DependencyInjection.GetService.ServiceScopeProvider

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.45958755154614 < 24.965723471467506.
IsChangePoint: Marked as a change because one of 1/18/2023 3:20:52 PM, 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 49.19486884424391 (T) = (0 -23.64775213861588) / Math.Sqrt((0.6837507449936806 / (299)) + (0.045488412302767484 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.12189803066478062 = (26.930530808988816 - 23.64775213861588) / 26.930530808988816 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.GetService.Transient

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.449912101919754 < 29.015318056812948.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 20.78921749173307 (T) = (0 -26.21381753102009) / Math.Sqrt((1.2266295136410263 / (299)) + (0.5403342785094624 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.12047621846584748 = (29.804557968057843 - 26.21381753102009) / 29.804557968057843 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.IndexerSet<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentDictionary - Duration of single invocation 35.80 μs 27.20 μs 0.76 0.03 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSet<String>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.202815918249854 < 34.0686926641861.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 130.77212831153855 (T) = (0 -27489.83911120044) / Math.Sqrt((746570.0344707388 / (299)) + (43170.86181952096 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.24298496239226547 = (36313.46505093464 - 27489.83911120044) / 36313.46505093464 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in PerfLabTests.CastingPerf2.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IFooObjIsIFoo - Duration of single invocation 768.93 μs 663.57 μs 0.86 0.03 False Trace Trace
FooObjIsFoo - Duration of single invocation 810.74 μs 753.71 μs 0.93 0.02 False
IFooObjIsDescendantOfIFoo - Duration of single invocation 790.81 μs 662.55 μs 0.84 0.03 False
IFooObjIsIFooInterAlia - Duration of single invocation 553.15 μs 506.90 μs 0.92 0.01 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf2.CastingPerf.IFooObjIsIFoo

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 663.5667410714287 < 742.8926504485851.
IsChangePoint: Marked as a change because one of 12/3/2022 7:24:45 AM, 12/14/2022 5:20:21 PM, 12/28/2022 5:13:50 PM, 1/18/2023 3:20:52 PM, 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 31.348620294205418 (T) = (0 -664659.1897150252) / Math.Sqrt((1579603001.0093668 / (299)) + (2388457.561717275 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.09874821656302663 = (737484.4654179887 - 664659.1897150252) / 737484.4654179887 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; PerfLabTests.CastingPerf2.CastingPerf.IFooObjIsIFoo()
       push      rdi
       push      rsi
       push      rbp
       push      rbx
       sub       rsp,28
       xor       esi,esi
       mov       edi,[7FF8D1E029BC]
       test      edi,edi
       jle       short M00_L01
       mov       rdx,26160C064E0
       mov       rbx,[rdx]
       mov       rdx,rbx
       mov       rcx,offset MT_PerfLabTests.CastingPerf2.IFoo
       call      qword ptr [7FF8D1A3D888]; System.Runtime.CompilerServices.CastHelpers.ChkCastInterface(Void*, System.Object)
       mov       rbp,26160C06580
M00_L00:
       mov       rdx,rbx
       mov       rcx,offset MT_PerfLabTests.CastingPerf2.IFoo
       call      qword ptr [7FF8D1A3D888]; System.Runtime.CompilerServices.CastHelpers.ChkCastInterface(Void*, System.Object)
       mov       rdx,rax
       mov       rcx,rbp
       call      CORINFO_HELP_ASSIGN_REF
       inc       esi
       cmp       esi,edi
       jl        short M00_L00
M00_L01:
       add       rsp,28
       pop       rbx
       pop       rbp
       pop       rsi
       pop       rdi
       ret
; Total bytes of code 107

; System.Runtime.CompilerServices.CastHelpers.ChkCastInterface(Void*, System.Object)
       test      rdx,rdx
       je        short M01_L01
       mov       rax,[rdx]
       movzx     r8d,word ptr [rax+0E]
       test      r8,r8
       je        short M01_L04
       mov       rax,[rax+38]
       cmp       r8,4
       jl        short M01_L03
M01_L00:
       cmp       [rax],rcx
       je        short M01_L01
       cmp       [rax+8],rcx
       je        short M01_L01
       cmp       [rax+10],rcx
       je        short M01_L01
       cmp       [rax+18],rcx
       jne       short M01_L02
M01_L01:
       mov       rax,rdx
       ret
M01_L02:
       add       rax,20
       add       r8,0FFFFFFFFFFFFFFFC
       cmp       r8,4
       jge       short M01_L00
       test      r8,r8
       je        short M01_L04
M01_L03:
       cmp       [rax],rcx
       je        short M01_L01
       add       rax,8
       dec       r8
       test      r8,r8
       jg        short M01_L03
M01_L04:
       jmp       qword ptr [7FF8D1A3D870]
; Total bytes of code 97

Compare Jit Disasm

; PerfLabTests.CastingPerf2.CastingPerf.IFooObjIsIFoo()
       push      rdi
       push      rsi
       push      rbp
       push      rbx
       sub       rsp,28
       xor       esi,esi
       mov       edi,[7FF8D25A29BC]
       test      edi,edi
       jle       short M00_L01
       mov       rdx,1E827C064E0
       mov       rbx,[rdx]
       mov       rdx,rbx
       mov       rcx,offset MT_PerfLabTests.CastingPerf2.IFoo
       call      qword ptr [7FF8D21DD888]; System.Runtime.CompilerServices.CastHelpers.ChkCastInterface(Void*, System.Object)
       mov       rbp,1E827C06580
M00_L00:
       mov       rdx,rbx
       mov       rcx,offset MT_PerfLabTests.CastingPerf2.IFoo
       call      qword ptr [7FF8D21DD888]; System.Runtime.CompilerServices.CastHelpers.ChkCastInterface(Void*, System.Object)
       mov       rdx,rax
       mov       rcx,rbp
       call      CORINFO_HELP_ASSIGN_REF
       inc       esi
       cmp       esi,edi
       jl        short M00_L00
M00_L01:
       add       rsp,28
       pop       rbx
       pop       rbp
       pop       rsi
       pop       rdi
       ret
; Total bytes of code 107
; System.Runtime.CompilerServices.CastHelpers.ChkCastInterface(Void*, System.Object)
       test      rdx,rdx
       je        short M01_L01
       mov       rax,[rdx]
       movzx     r8d,word ptr [rax+0E]
       test      r8,r8
       je        short M01_L04
       mov       rax,[rax+38]
       cmp       r8,4
       jl        short M01_L03
M01_L00:
       cmp       [rax],rcx
       je        short M01_L01
       cmp       [rax+8],rcx
       je        short M01_L01
       cmp       [rax+10],rcx
       je        short M01_L01
       cmp       [rax+18],rcx
       jne       short M01_L02
M01_L01:
       mov       rax,rdx
       ret
M01_L02:
       add       rax,20
       add       r8,0FFFFFFFFFFFFFFFC
       cmp       r8,4
       jge       short M01_L00
       test      r8,r8
       je        short M01_L04
M01_L03:
       cmp       [rax],rcx
       je        short M01_L01
       add       rax,8
       dec       r8
       test      r8,r8
       jg        short M01_L03
M01_L04:
       jmp       qword ptr [7FF8D21DD870]
; Total bytes of code 97

PerfLabTests.CastingPerf2.CastingPerf.FooObjIsFoo

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 753.7072355769232 < 770.2188338770604.
IsChangePoint: Marked as a change because one of 12/3/2022 7:24:45 AM, 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/21/2022 9:23:17 AM, 12/28/2022 5:13:50 PM, 1/18/2023 3:20:52 PM, 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 27.44551611949537 (T) = (0 -757035.9641964234) / Math.Sqrt((2086532406.6752172 / (299)) + (14612679.40512005 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.09126590882607735 = (833066.5389899346 - 757035.9641964234) / 833066.5389899346 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.IFooObjIsDescendantOfIFoo

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 662.5502435064934 < 732.923607207557.
IsChangePoint: Marked as a change because one of 12/3/2022 7:24:45 AM, 12/14/2022 5:20:21 PM, 12/28/2022 5:13:50 PM, 1/18/2023 3:20:52 PM, 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 31.208504656630296 (T) = (0 -665172.9350628201) / Math.Sqrt((1567591458.7511985 / (299)) + (3436951.1998691494 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.09836182573428018 = (737738.2125646207 - 665172.9350628201) / 737738.2125646207 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.IFooObjIsIFooInterAlia

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 506.9049596774194 < 525.9660872844827.
IsChangePoint: Marked as a change because one of 12/3/2022 7:24:45 AM, 1/13/2023 4:37:08 PM, 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 24.857694084600638 (T) = (0 -507308.7091397266) / Math.Sqrt((366806617.6643079 / (299)) + (9059869.577346586 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.0593518457453794 = (539318.2422621382 - 507308.7091397266) / 539318.2422621382 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.Concurrent.IsEmpty<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 235.67 ns 214.47 ns 0.91 0.05 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.IsEmpty&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.IsEmpty<Int32>.Dictionary(Size: 0)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 214.46681016525483 < 224.61531635417944.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 31.619451227342765 (T) = (0 -216.00749949290062) / Math.Sqrt((12.78911682471256 / (299)) + (9.633059541341062 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.0939249253473334 = (238.3991189424393 - 216.00749949290062) / 238.3991189424393 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Feb 14, 2023 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.ContainsKeyTrue<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentDictionary - Duration of single invocation 21.23 μs 13.13 μs 0.62 0.09 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue&lt;String, String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyTrue<String, String>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.133442069151485 < 20.20246077215608.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 76.38874046389476 (T) = (0 -11961.240447415958) / Math.Sqrt((1574495.9840099155 / (299)) + (215372.84351683408 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.443099178612283 = (21478.223748369164 - 11961.240447415958) / 21478.223748369164 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Feb 14, 2023 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in Microsoft.Extensions.Logging.LoggingOverhead

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TwoArguments - Duration of single invocation 85.15 ns 67.89 ns 0.80 0.19 False Trace Trace
TwoArguments_FilteredByLevel - Duration of single invocation 71.46 ns 54.88 ns 0.77 0.09 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.LoggingOverhead*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.LoggingOverhead.TwoArguments

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 67.88624329897917 < 80.84637984701433.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 2/1/2023 12:50:52 AM, 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 36.60022040842786 (T) = (0 -67.50901640721972) / Math.Sqrt((49.50399703518141 / (299)) + (3.533741624919118 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.23852999877138592 = (88.6561733204138 - 67.50901640721972) / 88.6561733204138 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; Microsoft.Extensions.Logging.LoggingOverhead.TwoArguments()
       push      rdi
       push      rsi
       push      rbx
       sub       rsp,40
       xor       eax,eax
       mov       [rsp+30],rax
       mov       rsi,[rcx+8]
       mov       rcx,26CCC0068F8
       mov       rdi,[rcx]
       mov       rcx,offset MT_System.Object[]
       mov       edx,2
       call      CORINFO_HELP_NEWARR_1_OBJ
       mov       rbx,rax
       mov       rcx,offset MT_System.Int32
       call      CORINFO_HELP_NEWSFAST
       mov       dword ptr [rax+8],1
       lea       rcx,[rbx+10]
       mov       rdx,rax
       call      CORINFO_HELP_ASSIGN_REF
       mov       rcx,2AD609AB078
       mov       [rbx+18],rcx
       mov       rcx,rsi
       xor       r8d,r8d
       mov       [rsp+30],r8
       mov       [rsp+38],r8d
       mov       r8,2AD609AB0A0
       mov       [rsp+20],r8
       mov       [rsp+28],rbx
       lea       r8,[rsp+30]
       mov       r9,rdi
       mov       edx,4
       call      qword ptr [7FFA62DF1288]; Microsoft.Extensions.Logging.LoggerExtensions.Log(Microsoft.Extensions.Logging.ILogger, Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.Exception, System.String, System.Object[])
       nop
       add       rsp,40
       pop       rbx
       pop       rsi
       pop       rdi
       ret
; Total bytes of code 166

; Microsoft.Extensions.Logging.LoggerExtensions.Log(Microsoft.Extensions.Logging.ILogger, Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.Exception, System.String, System.Object[])
       push      rdi
       push      rsi
       push      rbp
       push      rbx
       sub       rsp,68
       xor       eax,eax
       mov       [rsp+38],rax
       vxorps    xmm4,xmm4,xmm4
       vmovdqa   xmmword ptr [rsp+40],xmm4
       vmovdqa   xmmword ptr [rsp+50],xmm4
       mov       [rsp+60],rax
       mov       rdi,rcx
       mov       ebp,edx
       mov       rsi,r8
       mov       rbx,r9
       test      rdi,rdi
       je        near ptr M01_L00
       lea       rcx,[rsp+50]
       mov       rdx,[rsp+0B0]
       mov       r8,[rsp+0B8]
       call      qword ptr [7FFA62DF55D0]; Microsoft.Extensions.Logging.FormattedLogValues..ctor(System.String, System.Object[])
       mov       rcx,rdi
       mov       rdx,offset MT_Microsoft.Extensions.Logging.ILogger
       mov       r8,offset MD_Microsoft.Extensions.Logging.ILogger.Log[[Microsoft.Extensions.Logging.FormattedLogValues, Microsoft.Extensions.Logging.Abstractions]](Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, Microsoft.Extensions.Logging.FormattedLogValues, System.Exception, System.Func`3<Microsoft.Extensions.Logging.FormattedLogValues,System.Exception,System.String>)
       call      CORINFO_HELP_VIRTUAL_FUNC_PTR
       mov       rcx,rdi
       mov       edx,ebp
       mov       r8,rsi
       mov       r9,[rsp+50]
       mov       [rsp+38],r9
       mov       r9,[rsp+58]
       mov       [rsp+40],r9
       mov       r9,[rsp+60]
       mov       [rsp+48],r9
       mov       [rsp+20],rbx
       mov       r9,26CCC008588
       mov       r9,[r9]
       mov       [rsp+28],r9
       lea       r9,[rsp+38]
       call      rax
       nop
       add       rsp,68
       pop       rbx
       pop       rbp
       pop       rsi
       pop       rdi
       ret
M01_L00:
       mov       ecx,191
       mov       rdx,7FFA62CEC168
       call      CORINFO_HELP_STRCNS
       mov       rcx,rax
       call      qword ptr [7FFA62DF1E70]
       int       3
; Total bytes of code 219

Compare Jit Disasm

; Microsoft.Extensions.Logging.LoggingOverhead.TwoArguments()
       push      rdi
       push      rsi
       push      rbx
       sub       rsp,40
       xor       eax,eax
       mov       [rsp+30],rax
       mov       rsi,[rcx+8]
       mov       rcx,25B1A8068F8
       mov       rdi,[rcx]
       mov       rcx,offset MT_System.Object[]
       mov       edx,2
       call      CORINFO_HELP_NEWARR_1_OBJ
       mov       rbx,rax
       mov       rcx,offset MT_System.Int32
       call      CORINFO_HELP_NEWSFAST
       mov       dword ptr [rax+8],1
       lea       rcx,[rbx+10]
       mov       rdx,rax
       call      CORINFO_HELP_ASSIGN_REF
       mov       rcx,29BAF25C6C0
       mov       [rbx+18],rcx
       mov       rcx,rsi
       xor       r8d,r8d
       mov       [rsp+30],r8
       mov       [rsp+38],r8d
       mov       r8,29BAF25C6E8
       mov       [rsp+20],r8
       mov       [rsp+28],rbx
       lea       r8,[rsp+30]
       mov       r9,rdi
       mov       edx,4
       call      qword ptr [7FF84CEF1288]; Microsoft.Extensions.Logging.LoggerExtensions.Log(Microsoft.Extensions.Logging.ILogger, Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.Exception, System.String, System.Object[])
       nop
       add       rsp,40
       pop       rbx
       pop       rsi
       pop       rdi
       ret
; Total bytes of code 166
; Microsoft.Extensions.Logging.LoggerExtensions.Log(Microsoft.Extensions.Logging.ILogger, Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, System.Exception, System.String, System.Object[])
       push      rdi
       push      rsi
       push      rbp
       push      rbx
       sub       rsp,68
       xor       eax,eax
       mov       [rsp+38],rax
       vxorps    xmm4,xmm4,xmm4
       vmovdqa   xmmword ptr [rsp+40],xmm4
       vmovdqa   xmmword ptr [rsp+50],xmm4
       mov       [rsp+60],rax
       mov       rdi,rcx
       mov       ebp,edx
       mov       rsi,r8
       mov       rbx,r9
       test      rdi,rdi
       je        near ptr M01_L00
       lea       rcx,[rsp+50]
       mov       rdx,[rsp+0B0]
       mov       r8,[rsp+0B8]
       call      qword ptr [7FF84CEF55D0]; Microsoft.Extensions.Logging.FormattedLogValues..ctor(System.String, System.Object[])
       mov       rcx,rdi
       mov       rdx,offset MT_Microsoft.Extensions.Logging.ILogger
       mov       r8,offset MD_Microsoft.Extensions.Logging.ILogger.Log[[Microsoft.Extensions.Logging.FormattedLogValues, Microsoft.Extensions.Logging.Abstractions]](Microsoft.Extensions.Logging.LogLevel, Microsoft.Extensions.Logging.EventId, Microsoft.Extensions.Logging.FormattedLogValues, System.Exception, System.Func`3<Microsoft.Extensions.Logging.FormattedLogValues,System.Exception,System.String>)
       call      CORINFO_HELP_VIRTUAL_FUNC_PTR
       mov       rcx,rdi
       mov       edx,ebp
       mov       r8,rsi
       mov       r9,[rsp+50]
       mov       [rsp+38],r9
       mov       r9,[rsp+58]
       mov       [rsp+40],r9
       mov       r9,[rsp+60]
       mov       [rsp+48],r9
       mov       [rsp+20],rbx
       mov       r9,25B1A808598
       mov       r9,[r9]
       mov       [rsp+28],r9
       lea       r9,[rsp+38]
       call      rax
       nop
       add       rsp,68
       pop       rbx
       pop       rbp
       pop       rsi
       pop       rdi
       ret
M01_L00:
       mov       ecx,191
       mov       rdx,7FF84CDED368
       call      CORINFO_HELP_STRCNS
       mov       rcx,rax
       call      qword ptr [7FF84CEF1E70]
       int       3
; Total bytes of code 219

Microsoft.Extensions.Logging.LoggingOverhead.TwoArguments_FilteredByLevel

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 54.881817512292116 < 68.80172902092117.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 2/1/2023 8:39:12 AM, 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 77.46962930006974 (T) = (0 -53.466901063077344) / Math.Sqrt((11.911891027932452 / (299)) + (0.7068635511354318 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.2820295181616041 = (74.46949758459836 - 53.466901063077344) / 74.46949758459836 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.TryGetValueFalse<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentDictionary - Duration of single invocation 16.81 μs 9.54 μs 0.57 0.06 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;String, String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<String, String>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.543521956087824 < 16.43183911232388.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 128.4785783758654 (T) = (0 -8833.92551556139) / Math.Sqrt((579606.8821893515 / (299)) + (58909.263084579914 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.5004231158631719 = (17682.814790009146 - 8833.92551556139) / 17682.814790009146 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Fill - Duration of single invocation 3.31 ns 2.23 ns 0.67 0.28 False
Reverse - Duration of single invocation 35.38 ns 25.74 ns 0.73 0.34 False
IndexOfAnyFourValues - Duration of single invocation 59.06 ns 53.16 ns 0.90 0.04 False
SequenceEqual - Duration of single invocation 6.53 ns 5.49 ns 0.84 0.05 False
IndexOfValue - Duration of single invocation 29.77 ns 24.58 ns 0.83 0.11 False

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.Fill(Size: 33)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.227304156884574 < 3.1464397093866427.
IsChangePoint: Marked as a change because one of 12/12/2022 5:25:39 PM, 12/14/2022 5:20:21 PM, 12/28/2022 5:13:50 PM, 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 23.657861974294867 (T) = (0 -2.479409959981131) / Math.Sqrt((0.20685365959787527 / (299)) + (0.006943453700442856 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.23377843848046168 = (3.2358916591489146 - 2.479409959981131) / 3.2358916591489146 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.Reverse(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.744226857636058 < 33.58830072007118.
IsChangePoint: Marked as a change because one of 12/1/2022 7:29:59 PM, 12/6/2022 2:12:43 AM, 12/14/2022 12:26:48 AM, 1/13/2023 11:20:20 PM, 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 7.046870192461131 (T) = (0 -28.16724316812059) / Math.Sqrt((9.002830013336512 / (299)) + (16.051580383366865 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.18233371510496443 = (34.44833630597407 - 28.16724316812059) / 34.44833630597407 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfAnyFourValues(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 53.155099025810934 < 55.82108335719541.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 1/27/2023 7:12:54 PM, 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 18.09579595683358 (T) = (0 -54.044239031415536) / Math.Sqrt((6.006753015931907 / (299)) + (0.13474675473178582 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.051694974755610024 = (56.990353939638425 - 54.044239031415536) / 56.990353939638425 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.SequenceEqual(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.486073244188161 < 6.197194697651115.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 1/27/2023 7:12:54 PM, 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 6.833569229374707 (T) = (0 -5.630495688177016) / Math.Sqrt((0.19174198290500585 / (299)) + (0.03851803234256024 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.05694638338245194 = (5.970493712088103 - 5.630495688177016) / 5.970493712088103 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfValue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.578027712967547 < 28.125468004916303.
IsChangePoint: Marked as a change because one of 12/1/2022 11:48:13 PM, 1/3/2023 6:29:42 AM, 2/7/2023 8:22:47 AM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 16.598697462931682 (T) = (0 -24.846804959930434) / Math.Sqrt((1.8185721623663467 / (299)) + (1.5737993424885108 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.13914506284209016 = (28.862940650560144 - 24.846804959930434) / 28.862940650560144 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Factory - Duration of single invocation 42.12 ns 32.34 ns 0.77 0.28 False
ServiceProvider - Duration of single invocation 29.59 ns 25.03 ns 0.85 0.21 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.Factory

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.33905577131704 < 32.79088214062375.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 15.953785334369638 (T) = (0 -31.559397618208852) / Math.Sqrt((2.9432410712971486 / (299)) + (0.25916925535368 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.07002245053015024 = (33.935655367379404 - 31.559397618208852) / 33.935655367379404 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.Factory()
       sub       rsp,28
       mov       r8,rcx
       mov       rax,[r8+10]
       mov       rcx,[rax+8]
       mov       rdx,[r8+8]
       mov       r8,[r8+18]
       call      qword ptr [rax+18]
       mov       r8,rax
       test      r8,r8
       je        short M00_L00
       mov       rcx,offset MT_Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark+TypeToBeActivated
       cmp       [r8],rcx
       je        short M00_L00
       mov       rdx,rax
       call      qword ptr [7FFA7569D8B8]
       mov       r8,rax
M00_L00:
       mov       rax,r8
       add       rsp,28
       ret
; Total bytes of code 69

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.ServiceProvider

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.027363432359802 < 25.815913503733412.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 20.365849447225493 (T) = (0 -25.142132378920945) / Math.Sqrt((0.9821196551480663 / (299)) + (0.3064046879710572 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.09771714575526873 = (27.86502288128541 - 25.142132378920945) / 27.86502288128541 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentDictionary - Duration of single invocation 3.06 μs 2.10 μs 0.69 0.17 False
ImmutableSortedDictionary - Duration of single invocation 39.82 μs 37.50 μs 0.94 0.01 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<Int32, Int32>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.102234807623157 < 2.909003461485527.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 72.36977964437641 (T) = (0 -2129.129539780468) / Math.Sqrt((9959.805980456758 / (299)) + (2684.565991130544 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.30142493375842566 = (3047.8178261292155 - 2129.129539780468) / 3047.8178261292155 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;.ImmutableSortedDictionary(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 37.500524322169056 < 38.08085985696735.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 1/18/2023 11:30:09 AM, 1/27/2023 7:12:54 PM, 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 22.77008314018888 (T) = (0 -35043.411026088026) / Math.Sqrt((1655030.4061856891 / (299)) + (691835.1972113732 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.11305068042365081 = (39510.04894262334 - 35043.411026088026) / 39510.04894262334 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Memory.ReadOnlySequence

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Slice_StartPosition - Duration of single invocation 13.43 ns 12.18 ns 0.91 0.29 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySequence*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySequence.Slice_StartPosition(Segment: Multiple)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.180518633624423 < 13.078926695689109.
IsChangePoint: Marked as a change because one of 12/3/2022 7:24:45 AM, 12/17/2022 10:44:50 PM, 12/21/2022 9:23:17 AM, 12/28/2022 6:20:56 PM, 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 7.519434693706289 (T) = (0 -12.36639026157022) / Math.Sqrt((0.6203706243226941 / (299)) + (0.15271351982122972 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.055522309848672506 = (13.093364078921583 - 12.36639026157022) / 13.093364078921583 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ElementAt - Duration of single invocation 8.75 ns 7.61 ns 0.87 0.07 False
TakeLastHalf - Duration of single invocation 1.49 μs 1.37 μs 0.92 0.02 False
Min - Duration of single invocation 551.23 ns 494.14 ns 0.90 0.00 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.ElementAt(input: IList)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.607218207727617 < 8.230024905812245.
IsChangePoint: Marked as a change because one of 12/3/2022 7:24:45 AM, 12/10/2022 11:25:25 PM, 12/28/2022 5:13:50 PM, 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 23.695801579602616 (T) = (0 -7.587138010512372) / Math.Sqrt((0.29200875346569277 / (299)) + (4.09307004611871E-05 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.08900318700791181 = (8.328391386565986 - 7.587138010512372) / 8.328391386565986 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.TakeLastHalf(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3725652006747875 < 1.4102764405610377.
IsChangePoint: Marked as a change because one of 12/5/2022 8:20:45 AM, 12/10/2022 11:25:25 PM, 1/13/2023 11:01:49 AM, 2/9/2023 4:27:10 AM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 27.81002161762859 (T) = (0 -1375.9531633323359) / Math.Sqrt((543.6221647382323 / (299)) + (126.43438272030603 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.052663620454027005 = (1452.4441297100664 - 1375.9531633323359) / 1452.4441297100664 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Min(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 494.13949672576535 < 522.8001700510706.
IsChangePoint: Marked as a change because one of 1/11/2023 8:19:46 AM, 2/9/2023 4:27:10 AM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 102.04518365812217 (T) = (0 -491.4862310103045) / Math.Sqrt((60.08145549031597 / (299)) + (1.403442002387733 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.09651746249385573 = (543.9908472021376 - 491.4862310103045) / 543.9908472021376 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Feb 14, 2023 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.Concurrent.Count<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 14.48 μs 212.13 ns 0.01 0.42 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.Count&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.Count<String>.Dictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 212.1336794810077 < 13.690310045834124.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 55.82370124414083 (T) = (0 -214.57680607284783) / Math.Sqrt((13905039.974119212 / (299)) + (15.280202630592523 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.9824879314365589 = (12253.081655973334 - 214.57680607284783) / 12253.081655973334 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentDictionary - Duration of single invocation 4.04 μs 2.80 μs 0.69 0.14 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyTrue<Int32, Int32>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.803361988986652 < 3.821851736112345.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 34.063512975913135 (T) = (0 -3042.129050588059) / Math.Sqrt((15959.292955432678 / (299)) + (15114.97446412824 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.23742330597609468 = (3989.2761927139277 - 3042.129050588059) / 3989.2761927139277 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.ContainsKeyFalse<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentDictionary - Duration of single invocation 16.88 μs 8.51 μs 0.50 0.11 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse&lt;String, String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<String, String>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.506241480370774 < 16.292118620499316.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 93.24817667547634 (T) = (0 -9648.289682911163) / Math.Sqrt((570083.6637240482 / (299)) + (131677.37205462254 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.46636785447664353 = (18080.413190717103 - 9648.289682911163) / 18080.413190717103 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentDictionary - Duration of single invocation 13.48 μs 7.22 μs 0.54 0.03 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<Int32>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.220227097767513 < 12.867112603498455.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 342.56183091271924 (T) = (0 -6980.706297912891) / Math.Sqrt((42085.79222357836 / (299)) + (4578.413128054525 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.4817317053759254 = (13469.290655675435 - 6980.706297912891) / 13469.290655675435 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentDictionary - Duration of single invocation 3.77 μs 2.08 μs 0.55 0.13 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<Int32, Int32>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.0799437080728764 < 3.584291080694834.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 104.54883839445657 (T) = (0 -2136.3535837155687) / Math.Sqrt((15547.000833767876 / (299)) + (4587.073459343554 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.44591318087191484 = (3855.6296774526227 - 2136.3535837155687) / 3855.6296774526227 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentDictionary - Duration of single invocation 20.86 μs 13.91 μs 0.67 0.40 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<String>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.908232797140304 < 19.544053472629873.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 42.52682834531356 (T) = (0 -13915.508307973178) / Math.Sqrt((4338927.915311004 / (299)) + (61662.09296557585 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.2875852443881084 = (19532.87491360447 - 13915.508307973178) / 19532.87491360447 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.Concurrent.Count<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 14.50 μs 211.45 ns 0.01 0.00 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.Count&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.Count<Int32>.Dictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 211.45434179231887 < 13.787093313396937.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 1521.347673921013 (T) = (0 -211.6803696934187) / Math.Sqrt((26317.45952598255 / (299)) + (1.2281235447755932 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.9853906819962439 = (14489.408036637622 - 211.6803696934187) / 14489.408036637622 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.CopyTo<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableArray - Duration of single invocation 167.34 ns 129.12 ns 0.77 0.21 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CopyTo&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CopyTo<Int32>.ImmutableArray(Size: 2048)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 129.1176561520668 < 157.0541288268459.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 12.888072345722508 (T) = (0 -130.96660176001276) / Math.Sqrt((257.32832146847784 / (299)) + (1.990656532057264 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.08774852554818026 = (143.56414369043557 - 130.96660176001276) / 143.56414369043557 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.TryGetValueTrue<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentDictionary - Duration of single invocation 21.61 μs 11.64 μs 0.54 0.08 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue&lt;String, String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueTrue<String, String>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.635406600780868 < 20.157580082380562.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 89.13909469432177 (T) = (0 -12461.560850222104) / Math.Sqrt((1737033.3363682246 / (299)) + (76783.02810632135 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.41035810612820045 = (21134.117130645245 - 12461.560850222104) / 21134.117130645245 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in Microsoft.Extensions.DependencyInjection.ScopeValidation

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Transient - Duration of single invocation 29.50 ns 26.13 ns 0.89 0.21 False
TransientWithScopeValidation - Duration of single invocation 39.57 ns 32.03 ns 0.81 0.34 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ScopeValidation*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.ScopeValidation.Transient

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.128357287476735 < 28.75936271741767.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 29.86633740685058 (T) = (0 -26.11635599281404) / Math.Sqrt((1.0295364833245588 / (299)) + (0.2382661371180171 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.12209313266978664 = (29.74843569937665 - 26.11635599281404) / 29.74843569937665 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.ScopeValidation.TransientWithScopeValidation

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.028108864526644 < 37.12799459993386.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 46.636221763294515 (T) = (0 -32.514063194176615) / Math.Sqrt((3.0195913772075036 / (299)) + (0.25722964965219314 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.17656370659041457 = (39.485827202911246 - 32.514063194176615) / 39.485827202911246 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.TryAddGiventSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentDictionary - Duration of single invocation 50.48 μs 43.54 μs 0.86 0.02 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddGiventSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryAddGiventSize<String>.ConcurrentDictionary(Count: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.53744747899159 < 48.49486539331653.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 75.41303499074469 (T) = (0 -44176.291885551094) / Math.Sqrt((1470225.2104788502 / (299)) + (90523.2700519532 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.14088281477369682 = (51420.5659544739 - 44176.291885551094) / 51420.5659544739 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare 2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff Diff

Improvements in System.Collections.AddGivenSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentDictionary - Duration of single invocation 51.66 μs 43.15 μs 0.84 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<String>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.149016111332806 < 48.978425258322495.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 2/9/2023 12:40:40 PM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 66.88874364677453 (T) = (0 -44011.42162453501) / Math.Sqrt((1389186.3945440438 / (299)) + (164231.84663493535 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.14507453002766207 = (51479.8344070379 - 44011.42162453501) / 51479.8344070379 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@EgorBo EgorBo mentioned this issue Feb 21, 2023
Merged
@EgorBo EgorBo closed this as completed Feb 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@EgorBo EgorBo

Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-improvement runtime-coreclr untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@EgorBo