Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Regression in Single/Double parsing #58027

Closed
performanceautofiler bot opened this issue Aug 24, 2021 · 8 comments
Closed

Regression in Single/Double parsing #58027

performanceautofiler bot opened this issue Aug 24, 2021 · 8 comments
Labels
area-System.Runtime tenet-performance-benchmarks Issue from performance benchmark
Milestone
6.0.0

Comments

@performanceautofiler
Copy link

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 4cb2874d70a99a0512cf667f916a8b5b0c6eba69
Compare 52c6dcb45cc1e8e98d267c07f10fe42d89c71656
Diff Diff

Regressions in System.MathBenchmarks.Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ScaleB - Duration of single invocation 12.51 μs 14.78 μs 1.18 0.24 False

graph
Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Double*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Double.ScaleB

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 4cb2874d70a99a0512cf667f916a8b5b0c6eba69
Compare 52c6dcb45cc1e8e98d267c07f10fe42d89c71656
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseSingle - Duration of single invocation 31.41 ns 35.46 ns 1.13 0.03 False

graph
Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: 12345)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 4cb2874d70a99a0512cf667f916a8b5b0c6eba69
Compare 52c6dcb45cc1e8e98d267c07f10fe42d89c71656
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SequenceEqual - Duration of single invocation 10.35 ns 12.95 ns 1.25 0.28 False
BinarySearch - Duration of single invocation 10.29 ns 11.51 ns 1.12 0.09 False

graph
graph
Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.SequenceEqual(Size: 512)

System.Memory.Span<Byte>.BinarySearch(Size: 512)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 4cb2874d70a99a0512cf667f916a8b5b0c6eba69
Compare 52c6dcb45cc1e8e98d267c07f10fe42d89c71656
Diff Diff

Regressions in System.Collections.Sort<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
List - Duration of single invocation 4.29 μs 4.84 μs 1.13 0.34 False

graph
Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<Int32>.List(Size: 512)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 4cb2874d70a99a0512cf667f916a8b5b0c6eba69
Compare 52c6dcb45cc1e8e98d267c07f10fe42d89c71656
Diff Diff

Regressions in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ScaleB - Duration of single invocation 12.49 μs 15.20 μs 1.22 0.08 False

graph
Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Single*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Single.ScaleB

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 4cb2874d70a99a0512cf667f916a8b5b0c6eba69
Compare 52c6dcb45cc1e8e98d267c07f10fe42d89c71656
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DataContractJsonSerializer_ - Duration of single invocation 972.04 μs 2.12 ms 2.18 0.07 False

graph
Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>.DataContractJsonSerializer_

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 4cb2874d70a99a0512cf667f916a8b5b0c6eba69
Compare 52c6dcb45cc1e8e98d267c07f10fe42d89c71656
Diff Diff

Regressions in System.Tests.Perf_Array

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfShort - Duration of single invocation 9.87 ns 13.79 ns 1.40 0.24 False

graph
Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Array*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Array.IndexOfShort

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@kunalspathak kunalspathak changed the title [Perf] Changes at 8/20/2021 8:14:59 PM Regression in Single/Double parsing Aug 24, 2021
@kunalspathak
Copy link
Member

Possibly due to #56501

@kunalspathak kunalspathak transferred this issue from dotnet/perf-autofiling-issues Aug 24, 2021
@dotnet-issue-labeler dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label Aug 24, 2021
@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@kunalspathak
Copy link
Member

Also regression in dotnet/perf-autofiling-issues#732

@ghost
Copy link

ghost commented Aug 25, 2021

Tagging subscribers to this area: @dotnet/area-system-runtime
See info in area-owners.md if you want to be subscribed.

Issue Details

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 4cb2874d70a99a0512cf667f916a8b5b0c6eba69
Compare 52c6dcb45cc1e8e98d267c07f10fe42d89c71656
Diff Diff

Regressions in System.MathBenchmarks.Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ScaleB - Duration of single invocation 12.51 μs 14.78 μs 1.18 0.24 False

graph
Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Double*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Double.ScaleB

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 4cb2874d70a99a0512cf667f916a8b5b0c6eba69
Compare 52c6dcb45cc1e8e98d267c07f10fe42d89c71656
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseSingle - Duration of single invocation 31.41 ns 35.46 ns 1.13 0.03 False

graph
Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: 12345)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 4cb2874d70a99a0512cf667f916a8b5b0c6eba69
Compare 52c6dcb45cc1e8e98d267c07f10fe42d89c71656
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SequenceEqual - Duration of single invocation 10.35 ns 12.95 ns 1.25 0.28 False
BinarySearch - Duration of single invocation 10.29 ns 11.51 ns 1.12 0.09 False

graph
graph
Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.SequenceEqual(Size: 512)

System.Memory.Span<Byte>.BinarySearch(Size: 512)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 4cb2874d70a99a0512cf667f916a8b5b0c6eba69
Compare 52c6dcb45cc1e8e98d267c07f10fe42d89c71656
Diff Diff

Regressions in System.Collections.Sort<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
List - Duration of single invocation 4.29 μs 4.84 μs 1.13 0.34 False

graph
Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<Int32>.List(Size: 512)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 4cb2874d70a99a0512cf667f916a8b5b0c6eba69
Compare 52c6dcb45cc1e8e98d267c07f10fe42d89c71656
Diff Diff

Regressions in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ScaleB - Duration of single invocation 12.49 μs 15.20 μs 1.22 0.08 False

graph
Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Single*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Single.ScaleB

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 4cb2874d70a99a0512cf667f916a8b5b0c6eba69
Compare 52c6dcb45cc1e8e98d267c07f10fe42d89c71656
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DataContractJsonSerializer_ - Duration of single invocation 972.04 μs 2.12 ms 2.18 0.07 False

graph
Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>.DataContractJsonSerializer_

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 4cb2874d70a99a0512cf667f916a8b5b0c6eba69
Compare 52c6dcb45cc1e8e98d267c07f10fe42d89c71656
Diff Diff

Regressions in System.Tests.Perf_Array

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfShort - Duration of single invocation 9.87 ns 13.79 ns 1.40 0.24 False

graph
Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Array*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Array.IndexOfShort

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author: performanceautofiler[bot]
Assignees: -
Labels:

area-System.Runtime, untriaged
Milestone: -

@EgorBo
Copy link
Member

EgorBo commented Aug 25, 2021

Doesn't look like #56501 - I tried to revert it locally but it had no effect on ScaleB benchmark.

@jeffhandley jeffhandley added this to the 6.0.0 milestone Sep 30, 2021
@jeffhandley jeffhandley added needs-further-triage Issue has been initially triaged, but needs deeper consideration or reconsideration tenet-performance-benchmarks Issue from performance benchmark and removed untriaged New issue has not been triaged by the area owner labels Sep 30, 2021
@jeffhandley
Copy link
Member

@EgorBo / @kunalspathak / @tannergooding Can one of you self-assign to indicate that you're going to further investigate this? Thanks.

@AndyAyersMS
Copy link
Member

We forked 6.0 off at 82f7144 (Aug 17) so this is presumably not a 6.0 issue.

@tannergooding
Copy link
Member

This also looks to be simply noise. We're back to sitting at 31-32ns in the latest runs and there are no assembly diffs between the given commits.

@ghost ghost locked as resolved and limited conversation to collaborators Nov 4, 2021
@tannergooding tannergooding removed the needs-further-triage Issue has been initially triaged, but needs deeper consideration or reconsideration label Jun 24, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
area-System.Runtime tenet-performance-benchmarks Issue from performance benchmark
Projects
None yet
Milestone
6.0.0
Development

No branches or pull requests
6 participants
@EgorBo @jeffhandley @jeffschwMSFT @AndyAyersMS @tannergooding @kunalspathak