-
Notifications
You must be signed in to change notification settings - Fork 145
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add additional exclusions to for obsolete libraries #6231
Conversation
Datadog ReportBranch report: ✅ 0 Failed, 367579 Passed, 2105 Skipped, 15h 5m 15.14s Total Time |
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing the following branches/commits: Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6231) - mean (71ms) : 68, 73
. : milestone, 71,
master - mean (70ms) : 68, 72
. : milestone, 70,
section CallTarget+Inlining+NGEN
This PR (6231) - mean (1,118ms) : 1092, 1144
. : milestone, 1118,
master - mean (1,119ms) : 1098, 1141
. : milestone, 1119,
gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6231) - mean (110ms) : 105, 114
. : milestone, 110,
master - mean (110ms) : 107, 114
. : milestone, 110,
section CallTarget+Inlining+NGEN
This PR (6231) - mean (778ms) : 757, 799
. : milestone, 778,
master - mean (782ms) : 763, 801
. : milestone, 782,
gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6231) - mean (93ms) : 90, 96
. : milestone, 93,
master - mean (93ms) : 90, 96
. : milestone, 93,
section CallTarget+Inlining+NGEN
This PR (6231) - mean (740ms) : 722, 758
. : milestone, 740,
master - mean (738ms) : 722, 753
. : milestone, 738,
gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6231) - mean (196ms) : 192, 200
. : milestone, 196,
master - mean (190ms) : 188, 193
. : milestone, 190,
section CallTarget+Inlining+NGEN
This PR (6231) - mean (1,262ms) : 1238, 1285
. : milestone, 1262,
master - mean (1,225ms) : 1202, 1248
. : milestone, 1225,
gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6231) - mean (285ms) : 279, 290
. : milestone, 285,
master - mean (287ms) : 257, 317
. : milestone, 287,
section CallTarget+Inlining+NGEN
This PR (6231) - mean (982ms) : 964, 1001
. : milestone, 982,
master - mean (952ms) : 931, 973
. : milestone, 952,
gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6231) - mean (273ms) : 268, 278
. : milestone, 273,
master - mean (265ms) : 261, 269
. : milestone, 265,
section CallTarget+Inlining+NGEN
This PR (6231) - mean (963ms) : 946, 981
. : milestone, 963,
master - mean (953ms) : 917, 990
. : milestone, 953,
|
Benchmarks Report for tracer 🐌Benchmarks for #6231 compared to master:
The following thresholds were used for comparing the benchmark speeds:
Allocation changes below 0.5% are ignored. Benchmark detailsBenchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.GraphQLBenchmark - Slower
|
Benchmark | diff/base | Base Median (ns) | Diff Median (ns) | Modality |
---|---|---|---|---|
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net6.0 | 1.143 | 1,245.90 | 1,424.11 |
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | ExecuteAsync |
net6.0 | 1.24μs | 2.78ns | 10.8ns | 0.0136 | 0 | 0 | 952 B |
master | ExecuteAsync |
netcoreapp3.1 | 1.64μs | 0.808ns | 3.02ns | 0.013 | 0 | 0 | 952 B |
master | ExecuteAsync |
net472 | 1.81μs | 0.974ns | 3.77ns | 0.145 | 0 | 0 | 915 B |
#6231 | ExecuteAsync |
net6.0 | 1.42μs | 1.59ns | 5.94ns | 0.0135 | 0 | 0 | 952 B |
#6231 | ExecuteAsync |
netcoreapp3.1 | 1.65μs | 2.31ns | 8.96ns | 0.0123 | 0 | 0 | 952 B |
#6231 | ExecuteAsync |
net472 | 1.69μs | 0.978ns | 3.79ns | 0.145 | 0 | 0 | 915 B |
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | SendAsync |
net6.0 | 4.3μs | 2.98ns | 11.2ns | 0.0322 | 0 | 0 | 2.31 KB |
master | SendAsync |
netcoreapp3.1 | 5.18μs | 1.81ns | 6.52ns | 0.0365 | 0 | 0 | 2.85 KB |
master | SendAsync |
net472 | 7.34μs | 2.41ns | 9.35ns | 0.495 | 0 | 0 | 3.12 KB |
#6231 | SendAsync |
net6.0 | 4.33μs | 3.48ns | 13.5ns | 0.0305 | 0 | 0 | 2.31 KB |
#6231 | SendAsync |
netcoreapp3.1 | 5.26μs | 1.82ns | 6.81ns | 0.0393 | 0 | 0 | 2.85 KB |
#6231 | SendAsync |
net472 | 7.35μs | 1.82ns | 7.04ns | 0.492 | 0 | 0 | 3.12 KB |
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | EnrichedLog |
net6.0 | 1.55μs | 0.617ns | 2.39ns | 0.0232 | 0 | 0 | 1.64 KB |
master | EnrichedLog |
netcoreapp3.1 | 2.26μs | 1.23ns | 4.42ns | 0.0215 | 0 | 0 | 1.64 KB |
master | EnrichedLog |
net472 | 2.59μs | 1.57ns | 6.09ns | 0.249 | 0 | 0 | 1.57 KB |
#6231 | EnrichedLog |
net6.0 | 1.56μs | 1.22ns | 4.55ns | 0.0233 | 0 | 0 | 1.64 KB |
#6231 | EnrichedLog |
netcoreapp3.1 | 2.35μs | 9.09ns | 35.2ns | 0.022 | 0 | 0 | 1.64 KB |
#6231 | EnrichedLog |
net472 | 2.57μs | 0.947ns | 3.54ns | 0.25 | 0 | 0 | 1.57 KB |
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | EnrichedLog |
net6.0 | 118μs | 101ns | 390ns | 0 | 0 | 0 | 4.28 KB |
master | EnrichedLog |
netcoreapp3.1 | 122μs | 200ns | 775ns | 0 | 0 | 0 | 4.28 KB |
master | EnrichedLog |
net472 | 151μs | 210ns | 815ns | 0.679 | 0.226 | 0 | 4.46 KB |
#6231 | EnrichedLog |
net6.0 | 116μs | 144ns | 558ns | 0.0575 | 0 | 0 | 4.28 KB |
#6231 | EnrichedLog |
netcoreapp3.1 | 120μs | 326ns | 1.26μs | 0 | 0 | 0 | 4.28 KB |
#6231 | EnrichedLog |
net472 | 152μs | 164ns | 637ns | 0.672 | 0.224 | 0 | 4.46 KB |
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | EnrichedLog |
net6.0 | 3.14μs | 2.51ns | 9.72ns | 0.0304 | 0 | 0 | 2.2 KB |
master | EnrichedLog |
netcoreapp3.1 | 4.27μs | 1.42ns | 5.52ns | 0.0299 | 0 | 0 | 2.2 KB |
master | EnrichedLog |
net472 | 4.97μs | 1.93ns | 7.47ns | 0.321 | 0 | 0 | 2.02 KB |
#6231 | EnrichedLog |
net6.0 | 3.23μs | 0.879ns | 3.29ns | 0.0306 | 0 | 0 | 2.2 KB |
#6231 | EnrichedLog |
netcoreapp3.1 | 4.23μs | 0.998ns | 3.73ns | 0.0296 | 0 | 0 | 2.2 KB |
#6231 | EnrichedLog |
net472 | 4.87μs | 1ns | 3.74ns | 0.32 | 0 | 0 | 2.02 KB |
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | SendReceive |
net6.0 | 1.4μs | 0.596ns | 2.23ns | 0.016 | 0 | 0 | 1.14 KB |
master | SendReceive |
netcoreapp3.1 | 1.91μs | 0.738ns | 2.76ns | 0.0153 | 0 | 0 | 1.14 KB |
master | SendReceive |
net472 | 2.06μs | 0.991ns | 3.84ns | 0.183 | 0 | 0 | 1.16 KB |
#6231 | SendReceive |
net6.0 | 1.45μs | 0.937ns | 3.5ns | 0.0159 | 0 | 0 | 1.14 KB |
#6231 | SendReceive |
netcoreapp3.1 | 1.73μs | 0.885ns | 3.31ns | 0.0153 | 0 | 0 | 1.14 KB |
#6231 | SendReceive |
net472 | 2.11μs | 0.645ns | 2.5ns | 0.183 | 0 | 0 | 1.16 KB |
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | EnrichedLog |
net6.0 | 2.74μs | 4.16ns | 16.1ns | 0.023 | 0 | 0 | 1.6 KB |
master | EnrichedLog |
netcoreapp3.1 | 3.74μs | 1.95ns | 7.28ns | 0.0224 | 0 | 0 | 1.65 KB |
master | EnrichedLog |
net472 | 4.38μs | 3.41ns | 13.2ns | 0.324 | 0 | 0 | 2.04 KB |
#6231 | EnrichedLog |
net6.0 | 2.64μs | 1.02ns | 3.95ns | 0.0226 | 0 | 0 | 1.6 KB |
#6231 | EnrichedLog |
netcoreapp3.1 | 3.89μs | 4.12ns | 16ns | 0.0214 | 0 | 0 | 1.65 KB |
#6231 | EnrichedLog |
net472 | 4.38μs | 1.75ns | 6.76ns | 0.323 | 0 | 0 | 2.04 KB |
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | StartFinishSpan |
net6.0 | 391ns | 0.263ns | 1.02ns | 0.00812 | 0 | 0 | 576 B |
master | StartFinishSpan |
netcoreapp3.1 | 618ns | 1.42ns | 5.51ns | 0.00776 | 0 | 0 | 576 B |
master | StartFinishSpan |
net472 | 681ns | 1.22ns | 4.71ns | 0.0916 | 0 | 0 | 578 B |
master | StartFinishScope |
net6.0 | 519ns | 0.269ns | 1.01ns | 0.00968 | 0 | 0 | 696 B |
master | StartFinishScope |
netcoreapp3.1 | 750ns | 0.682ns | 2.64ns | 0.00952 | 0 | 0 | 696 B |
master | StartFinishScope |
net472 | 937ns | 0.986ns | 3.82ns | 0.104 | 0 | 0 | 658 B |
#6231 | StartFinishSpan |
net6.0 | 401ns | 0.339ns | 1.22ns | 0.00802 | 0 | 0 | 576 B |
#6231 | StartFinishSpan |
netcoreapp3.1 | 625ns | 2.98ns | 11.5ns | 0.00758 | 0 | 0 | 576 B |
#6231 | StartFinishSpan |
net472 | 667ns | 0.58ns | 2.25ns | 0.0915 | 0 | 0 | 578 B |
#6231 | StartFinishScope |
net6.0 | 475ns | 0.309ns | 1.12ns | 0.00978 | 0 | 0 | 696 B |
#6231 | StartFinishScope |
netcoreapp3.1 | 713ns | 0.421ns | 1.58ns | 0.00925 | 0 | 0 | 696 B |
#6231 | StartFinishScope |
net472 | 927ns | 0.976ns | 3.78ns | 0.104 | 0 | 0 | 658 B |
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | RunOnMethodBegin |
net6.0 | 584ns | 0.432ns | 1.67ns | 0.0097 | 0 | 0 | 696 B |
master | RunOnMethodBegin |
netcoreapp3.1 | 897ns | 0.798ns | 3.09ns | 0.00931 | 0 | 0 | 696 B |
master | RunOnMethodBegin |
net472 | 1.14μs | 0.912ns | 3.53ns | 0.104 | 0 | 0 | 658 B |
#6231 | RunOnMethodBegin |
net6.0 | 619ns | 0.434ns | 1.68ns | 0.00995 | 0 | 0 | 696 B |
#6231 | RunOnMethodBegin |
netcoreapp3.1 | 970ns | 0.719ns | 2.79ns | 0.00927 | 0 | 0 | 696 B |
#6231 | RunOnMethodBegin |
net472 | 1.15μs | 0.836ns | 3.24ns | 0.104 | 0 | 0 | 658 B |
Throughput/Crank Report ⚡Throughput results for AspNetCoreSimpleController comparing the following branches/commits: Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red. Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards! gantt
title Throughput Linux x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (6231) (11.179M) : 0, 11178866
master (11.136M) : 0, 11136226
benchmarks/2.9.0 (11.038M) : 0, 11038036
section Automatic
This PR (6231) (7.439M) : 0, 7438733
master (7.217M) : 0, 7216745
benchmarks/2.9.0 (7.724M) : 0, 7723503
section Trace stats
master (7.668M) : 0, 7667600
section Manual
master (11.106M) : 0, 11105865
section Manual + Automatic
This PR (6231) (6.829M) : 0, 6829183
master (6.682M) : 0, 6681799
section DD_TRACE_ENABLED=0
master (10.290M) : 0, 10290155
gantt
title Throughput Linux arm64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (6231) (9.697M) : 0, 9697235
master (9.651M) : 0, 9651400
benchmarks/2.9.0 (9.579M) : 0, 9578713
section Automatic
This PR (6231) (6.287M) : 0, 6287094
master (6.324M) : 0, 6324303
section Trace stats
master (6.773M) : 0, 6772736
section Manual
master (9.439M) : 0, 9439110
section Manual + Automatic
This PR (6231) (6.054M) : 0, 6054229
master (6.084M) : 0, 6084499
section DD_TRACE_ENABLED=0
master (8.868M) : 0, 8868156
gantt
title Throughput Windows x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (6231) (9.967M) : 0, 9966644
master (9.799M) : 0, 9799358
benchmarks/2.9.0 (9.814M) : 0, 9814080
section Automatic
This PR (6231) (6.320M) : 0, 6319533
master (6.534M) : 0, 6534314
benchmarks/2.9.0 (7.108M) : 0, 7108110
section Trace stats
master (7.102M) : 0, 7101962
section Manual
master (9.484M) : 0, 9483918
section Manual + Automatic
This PR (6231) (5.992M) : 0, 5991844
master (6.076M) : 0, 6075800
section DD_TRACE_ENABLED=0
master (9.299M) : 0, 9298888
|
Summary of changes
Silence some more warnings we don't care about
Reason for change
These are test apps, testing old libraries etc. We don't care about them being obsolete
Implementation details
NETSDK1215
silences a warning about usingnetstandard1.0
because it's not recommended (we don't care)Test coverage
No change
Other details
These are flagged by the .NET 9 SDK, but there's no harm in adding the exclusions now