Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add initial support for .NET 9 #6265

Merged
merged 24 commits into from
Nov 13, 2024
Merged

Add initial support for .NET 9 #6265

merged 24 commits into from
Nov 13, 2024
+6,945 −771

Conversation

andrewlock
Copy link
Member

@andrewlock andrewlock commented Nov 8, 2024 •
edited
Loading

Summary of changes

Updates as much as possible to support the .NET 9 RC2 release (with a view to update to GA ASAP)

Reason for change

.NET 9 is released very soon, and we want to support it

Implementation details

As you may be able to tell, this was pretty painful, with a lot of small cuts.
There are some things that I chose to skip rather than address.
Some of these are small... some of them are not 🙈

It's probably best to review commits one at a time, so you can skip the "boilerplate" ones

At a high level

  • Updated all references to 8.0.100 SDK to the latest .NET 9 SDK ♻️
  • Use different VM pools for safety 🦺
  • Add net9.0 targets to all the apps that should support it ➕
  • Fix a bunch of build warnings 🏗️
  • Exclude new experimental activities added in .NET 9 so we don't get duplicate spans 🛑
  • Bump all the built-in integrations to use .NET 9 (introduced a shared constant to make bumping that easier in the future) 🎉
  • Update guard rails to allow injection into .NET 9 (could delay this a little until we have ironed out the kinks) ❓
  • Add support for new supported method overloads for IAST 🆕
    • File has new overloads
    • Type has a new overload
  • Skip unsupported method overloads for IAST tests 🦕
    • There are new overloads for string and StringBuilder that use ReadOnlySpan<> - we don't currently support instrumenting those
    • Additionally, due to the new overload selection in C#13, more methods are selected that use ReadOnlySpan<T> instead of string which means overall we have less coverage
    • This is also annoying because it means the "wrong" methods are selected in expressions, which causes compiler errors, so we have to switch to explicitly passing arrays
    • We should prioritise adding support for ReadOnlySpan for strings if possible
  • Skipped tests that can't be easily fixed (see below) ⚠️

Test coverage

  • Integration tests are all running on .NET 9
  • Some smoke tests (See details below)

Other details

We have some outstanding issues to address ASAP after this is merged. These are also marked with FIXME: .NET 9

  • Our Linux crash tracking smoke tests hang on .NET 9 on x64
  • Our Linux arm64 smoke tests crash (I believe this is related to running on docker compose v2, so I've updated the VMs)
    • Updating the VMs to use docker v2 fixed the issue. I will look to update all our VMs to use v2 later, but that's non-blocking
  • Debugger ProbesTests are failing in .NET 7+ - it looks like the .NET 9 SDK generates slightly different code, this is probably only an issue for the snapshots, so the snapshots should be updated and the test unskipped
  • .NET 9 changed how WebRequest works significantly. We will need to update our integrations, but it's not clear how best to do that without more exploration. This only affects the case where AllowWriteStreamBuffering=false (which is not the default) so changed the tests to use that for now instead.
  • ManualInstrumentation on Windows .NET 9, x86 only are failing because they're failing to find the runtime. I think this is a niche build issue that will be solved with the GA, so just skipped for now
  • TrimmingTests on Linux debian, arm64 .NET 9 only are crashing while trying to add the startup hook. Struggling to get any useful logs or crash dumps right now, so skipping and will look into it closer (potentially on mac arm64 where easier to test)
    • Seems to be fixed in GA
  • Profiler execution tests are currently broken - they need to have the .NET 9 SDK installed, we could probably safely wait for the GA release seeing as that's only 2 days away...
    • Done, installed .NET 9 GA release
  • Gitlab build will be broken... I need to follow up and fix that ASAP now

@andrewlock andrewlock requested review from a team as code owners November 8, 2024 16:51
@andrewlock
Copy link
Member Author

andrewlock commented Nov 8, 2024 •
edited
Loading

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6265) - mean (71ms)  : 64, 79
     .   : milestone, 71,
    master - mean (69ms)  : 67, 70
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (6265) - mean (1,109ms)  : 1085, 1133
     .   : milestone, 1109,
    master - mean (1,107ms)  : 1090, 1124
     .   : milestone, 1107,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6265) - mean (108ms)  : 106, 110
     .   : milestone, 108,
    master - mean (108ms)  : 106, 110
     .   : milestone, 108,

    section CallTarget+Inlining+NGEN
    This PR (6265) - mean (769ms)  : 755, 783
     .   : milestone, 769,
    master - mean (777ms)  : 757, 796
     .   : milestone, 777,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6265) - mean (91ms)  : 90, 93
     .   : milestone, 91,
    master - mean (91ms)  : 89, 94
     .   : milestone, 91,

    section CallTarget+Inlining+NGEN
    This PR (6265) - mean (719ms)  : 706, 733
     .   : milestone, 719,
    master - mean (732ms)  : 716, 748
     .   : milestone, 732,
Loading

@andrewlock
Copy link
Member Author

andrewlock commented Nov 8, 2024 •
edited
Loading

Benchmarks Report for appsec 🐌

Benchmarks for #6265 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.132
  • 2 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6265

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorMoreComplexBody‑net472 1.132 4,358.01 3,851.22

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 72.3μs 67.8ns 263ns 0.0716 0 0 6 KB
master AllCycleSimpleBody netcoreapp3.1 62.5μs 58.7ns 220ns 0.0938 0 0 6.95 KB
master AllCycleSimpleBody net472 49.1μs 72ns 279ns 1.31 0 0 8.33 KB
master AllCycleMoreComplexBody net6.0 78.4μs 55ns 213ns 0.117 0 0 9.51 KB
master AllCycleMoreComplexBody netcoreapp3.1 69.4μs 56.4ns 211ns 0.139 0 0 10.36 KB
master AllCycleMoreComplexBody net472 56.5μs 85.2ns 330ns 1.87 0.0283 0 11.85 KB
master ObjectExtractorSimpleBody net6.0 145ns 0.12ns 0.448ns 0.00395 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 209ns 0.137ns 0.494ns 0.00369 0 0 272 B
master ObjectExtractorSimpleBody net472 171ns 0.0861ns 0.334ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 3.04μs 2.64ns 9.86ns 0.0533 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 3.91μs 3.08ns 11.9ns 0.0507 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 4.36μs 1.92ns 7.44ns 0.601 0.00656 0 3.8 KB
#6265 AllCycleSimpleBody net6.0 74.3μs 197ns 764ns 0.0725 0 0 6 KB
#6265 AllCycleSimpleBody netcoreapp3.1 64.2μs 109ns 423ns 0.0632 0 0 6.95 KB
#6265 AllCycleSimpleBody net472 50.8μs 112ns 432ns 1.32 0 0 8.34 KB
#6265 AllCycleMoreComplexBody net6.0 78.7μs 241ns 932ns 0.117 0 0 9.51 KB
#6265 AllCycleMoreComplexBody netcoreapp3.1 70.2μs 158ns 610ns 0.141 0 0 10.37 KB
#6265 AllCycleMoreComplexBody net472 58.3μs 90.7ns 351ns 1.86 0.0287 0 11.85 KB
#6265 ObjectExtractorSimpleBody net6.0 143ns 0.16ns 0.599ns 0.00393 0 0 280 B
#6265 ObjectExtractorSimpleBody netcoreapp3.1 200ns 0.664ns 2.57ns 0.00371 0 0 272 B
#6265 ObjectExtractorSimpleBody net472 166ns 0.54ns 2.09ns 0.0446 0 0 281 B
#6265 ObjectExtractorMoreComplexBody net6.0 3.08μs 0.958ns 3.45ns 0.0525 0 0 3.78 KB
#6265 ObjectExtractorMoreComplexBody netcoreapp3.1 3.93μs 2.62ns 10.1ns 0.051 0 0 3.69 KB
#6265 ObjectExtractorMoreComplexBody net472 3.85μs 3.93ns 14.7ns 0.602 0.00577 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 38.2μs 26.2ns 102ns 0.454 0 0 32.4 KB
master EncodeArgs netcoreapp3.1 54.3μs 36.7ns 142ns 0.432 0 0 32.4 KB
master EncodeArgs net472 66.4μs 27.9ns 104ns 5.15 0.0661 0 32.5 KB
master EncodeLegacyArgs net6.0 76.2μs 407ns 2.11μs 0 0 0 2.14 KB
master EncodeLegacyArgs netcoreapp3.1 109μs 254ns 983ns 0 0 0 2.14 KB
master EncodeLegacyArgs net472 154μs 97.1ns 376ns 0.31 0 0 2.15 KB
#6265 EncodeArgs net6.0 36.8μs 22.3ns 86.2ns 0.454 0 0 32.4 KB
#6265 EncodeArgs netcoreapp3.1 54μs 16.4ns 63.4ns 0.431 0 0 32.4 KB
#6265 EncodeArgs net472 66.2μs 41.3ns 160ns 5.14 0.0663 0 32.5 KB
#6265 EncodeLegacyArgs net6.0 73.1μs 149ns 633ns 0 0 0 2.14 KB
#6265 EncodeLegacyArgs netcoreapp3.1 105μs 119ns 462ns 0 0 0 2.15 KB
#6265 EncodeLegacyArgs net472 151μs 131ns 506ns 0.301 0 0 2.15 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 185μs 60.9ns 236ns 0 0 0 2.44 KB
master RunWafRealisticBenchmark netcoreapp3.1 198μs 238ns 921ns 0 0 0 2.39 KB
master RunWafRealisticBenchmark net472 210μs 141ns 545ns 0.315 0 0 2.46 KB
master RunWafRealisticBenchmarkWithAttack net6.0 122μs 106ns 381ns 0 0 0 1.47 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 130μs 155ns 598ns 0 0 0 1.46 KB
master RunWafRealisticBenchmarkWithAttack net472 138μs 42.5ns 153ns 0.208 0 0 1.49 KB
#6265 RunWafRealisticBenchmark net6.0 183μs 190ns 735ns 0 0 0 2.44 KB
#6265 RunWafRealisticBenchmark netcoreapp3.1 196μs 299ns 1.16μs 0 0 0 2.39 KB
#6265 RunWafRealisticBenchmark net472 209μs 194ns 725ns 0.312 0 0 2.46 KB
#6265 RunWafRealisticBenchmarkWithAttack net6.0 121μs 136ns 528ns 0 0 0 1.47 KB
#6265 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 130μs 52.1ns 180ns 0 0 0 1.46 KB
#6265 RunWafRealisticBenchmarkWithAttack net472 138μs 50.1ns 180ns 0.207 0 0 1.49 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #6265

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 255.27 KB 264.09 KB 8.82 KB 3.45%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472 272.91 KB 278.53 KB 5.62 KB 2.06%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 52.9μs 209ns 782ns 0 0 0 43.44 KB
master StringConcatBenchmark netcoreapp3.1 54.2μs 287ns 1.38μs 0 0 0 42.64 KB
master StringConcatBenchmark net472 38μs 165ns 616ns 0 0 0 60.49 KB
master StringConcatAspectBenchmark net6.0 295μs 4.71μs 43.7μs 0 0 0 253.95 KB
master StringConcatAspectBenchmark netcoreapp3.1 354μs 1.97μs 12.1μs 0 0 0 255.27 KB
master StringConcatAspectBenchmark net472 278μs 6.14μs 58.9μs 0 0 0 272.91 KB
#6265 StringConcatBenchmark net6.0 58.7μs 837ns 8.32μs 0 0 0 43.44 KB
#6265 StringConcatBenchmark netcoreapp3.1 59.7μs 778ns 7.7μs 0 0 0 42.64 KB
#6265 StringConcatBenchmark net472 37.8μs 191ns 834ns 0 0 0 60.38 KB
#6265 StringConcatAspectBenchmark net6.0 313μs 1.77μs 13μs 0 0 0 254.53 KB
#6265 StringConcatAspectBenchmark netcoreapp3.1 310μs 5.68μs 55.1μs 0 0 0 264.09 KB
#6265 StringConcatAspectBenchmark net472 327μs 10.9μs 108μs 0 0 0 278.53 KB

@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Nov 8, 2024 •
edited
Loading

Datadog Report

Branch report: andrew/dotnet-9
Commit report: 5e70959
Test service: dd-trace-dotnet

✅ 0 Failed, 575951 Passed, 5274 Skipped, 49h 58m 46.61s Total Time

@andrewlock
Copy link
Member Author

andrewlock commented Nov 8, 2024 •
edited
Loading

Benchmarks Report for tracer 🐌

Benchmarks for #6265 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.140
  • 2 benchmarks are slower, with geometric mean 1.178
  • 1 benchmarks have fewer allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.05μs 43.6ns 235ns 0.0158 0.00792 0 5.61 KB
master StartStopWithChild netcoreapp3.1 10.1μs 54.9ns 296ns 0.021 0.0105 0 5.8 KB
master StartStopWithChild net472 16.4μs 43.9ns 170ns 1.03 0.303 0.0901 6.22 KB
#6265 StartStopWithChild net6.0 8.2μs 46.6ns 330ns 0.0161 0.00403 0 5.61 KB
#6265 StartStopWithChild netcoreapp3.1 10.2μs 55.1ns 297ns 0.0207 0.0104 0 5.8 KB
#6265 StartStopWithChild net472 16.5μs 45.7ns 177ns 1.04 0.306 0.0965 6.2 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 486μs 222ns 799ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 646μs 247ns 955ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 843μs 580ns 2.25μs 0.419 0 0 3.3 KB
#6265 WriteAndFlushEnrichedTraces net6.0 469μs 179ns 646ns 0 0 0 2.7 KB
#6265 WriteAndFlushEnrichedTraces netcoreapp3.1 632μs 307ns 1.19μs 0 0 0 2.7 KB
#6265 WriteAndFlushEnrichedTraces net472 827μs 543ns 2.1μs 0.414 0 0 3.3 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 198μs 1.16μs 10.9μs 0.2 0 0 18.73 KB
master SendRequest netcoreapp3.1 229μs 1.42μs 14μs 0.206 0 0 20.89 KB
master SendRequest net472 8.3E‑05ns 6.89E‑05ns 0.000248ns 0 0 0 0 b
#6265 SendRequest net6.0 199μs 1.1μs 9.63μs 0.191 0 0 18.73 KB
#6265 SendRequest netcoreapp3.1 219μs 1.23μs 8.43μs 0.213 0 0 20.89 KB
#6265 SendRequest net472 0.00114ns 0.000519ns 0.00201ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #6265

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 41.82 KB 41.48 KB -347 B -0.83%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 602μs 3.43μs 30.9μs 0.558 0 0 41.82 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 676μs 3.26μs 12.6μs 0.349 0 0 41.78 KB
master WriteAndFlushEnrichedTraces net472 867μs 3.74μs 14.5μs 8.25 2.6 0.434 53.27 KB
#6265 WriteAndFlushEnrichedTraces net6.0 556μs 2.26μs 8.46μs 0.556 0 0 41.48 KB
#6265 WriteAndFlushEnrichedTraces netcoreapp3.1 683μs 1.52μs 5.88μs 0.342 0 0 41.74 KB
#6265 WriteAndFlushEnrichedTraces net472 907μs 3.5μs 13.6μs 8.27 2.3 0.46 53.32 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.23μs 1.52ns 5.87ns 0.0145 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 1.77μs 0.875ns 3.27ns 0.0133 0 0 1.02 KB
master ExecuteNonQuery net472 2.11μs 1.33ns 4.97ns 0.156 0.00106 0 987 B
#6265 ExecuteNonQuery net6.0 1.2μs 0.84ns 3.26ns 0.0144 0 0 1.02 KB
#6265 ExecuteNonQuery netcoreapp3.1 1.77μs 1.03ns 3.99ns 0.0133 0 0 1.02 KB
#6265 ExecuteNonQuery net472 2.14μs 3.24ns 12.5ns 0.157 0.00107 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.23μs 0.403ns 1.51ns 0.0136 0 0 976 B
master CallElasticsearch netcoreapp3.1 1.55μs 0.864ns 3.35ns 0.0131 0 0 976 B
master CallElasticsearch net472 2.48μs 2.6ns 10.1ns 0.157 0 0 995 B
master CallElasticsearchAsync net6.0 1.24μs 0.543ns 2.1ns 0.013 0 0 952 B
master CallElasticsearchAsync netcoreapp3.1 1.6μs 0.683ns 2.55ns 0.0137 0 0 1.02 KB
master CallElasticsearchAsync net472 2.61μs 2.81ns 10.9ns 0.167 0 0 1.05 KB
#6265 CallElasticsearch net6.0 1.22μs 0.644ns 2.41ns 0.0136 0 0 976 B
#6265 CallElasticsearch netcoreapp3.1 1.5μs 0.472ns 1.7ns 0.0134 0 0 976 B
#6265 CallElasticsearch net472 2.59μs 2.79ns 10.8ns 0.157 0 0 995 B
#6265 CallElasticsearchAsync net6.0 1.27μs 0.497ns 1.86ns 0.0134 0 0 952 B
#6265 CallElasticsearchAsync netcoreapp3.1 1.67μs 0.636ns 2.47ns 0.0143 0 0 1.02 KB
#6265 CallElasticsearchAsync net472 2.6μs 2.63ns 10.2ns 0.167 0 0 1.05 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.29μs 0.685ns 2.56ns 0.0135 0 0 952 B
master ExecuteAsync netcoreapp3.1 1.67μs 1.73ns 6.47ns 0.0125 0 0 952 B
master ExecuteAsync net472 1.84μs 0.699ns 2.61ns 0.145 0 0 915 B
#6265 ExecuteAsync net6.0 1.27μs 0.545ns 1.96ns 0.0134 0 0 952 B
#6265 ExecuteAsync netcoreapp3.1 1.62μs 0.51ns 1.91ns 0.013 0 0 952 B
#6265 ExecuteAsync net472 1.8μs 0.817ns 3.06ns 0.145 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.46μs 1.22ns 4.57ns 0.0313 0 0 2.31 KB
master SendAsync netcoreapp3.1 5.2μs 1.63ns 6.3ns 0.039 0 0 2.85 KB
master SendAsync net472 7.52μs 2.64ns 10.2ns 0.495 0 0 3.12 KB
#6265 SendAsync net6.0 4.34μs 2.68ns 10ns 0.0328 0 0 2.31 KB
#6265 SendAsync netcoreapp3.1 5.4μs 1.96ns 7.08ns 0.038 0 0 2.85 KB
#6265 SendAsync net472 7.38μs 2.16ns 8.35ns 0.493 0 0 3.12 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.48μs 1ns 3.76ns 0.0231 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 2.32μs 1.88ns 7.05ns 0.022 0 0 1.64 KB
master EnrichedLog net472 2.64μs 2ns 7.74ns 0.249 0 0 1.57 KB
#6265 EnrichedLog net6.0 1.6μs 0.845ns 3.16ns 0.0232 0 0 1.64 KB
#6265 EnrichedLog netcoreapp3.1 2.28μs 1.26ns 4.89ns 0.0217 0 0 1.64 KB
#6265 EnrichedLog net472 2.74μs 1.25ns 4.69ns 0.249 0 0 1.57 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 119μs 95.3ns 369ns 0.0594 0 0 4.28 KB
master EnrichedLog netcoreapp3.1 122μs 175ns 632ns 0 0 0 4.28 KB
master EnrichedLog net472 154μs 177ns 685ns 0.688 0.229 0 4.46 KB
#6265 EnrichedLog net6.0 119μs 159ns 616ns 0 0 0 4.28 KB
#6265 EnrichedLog netcoreapp3.1 124μs 460ns 1.78μs 0 0 0 4.28 KB
#6265 EnrichedLog net472 156μs 288ns 1.12μs 0.698 0.233 0 4.46 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 3.04μs 13.8ns 55.2ns 0.0302 0 0 2.2 KB
master EnrichedLog netcoreapp3.1 4.23μs 3.92ns 15.2ns 0.0295 0 0 2.2 KB
master EnrichedLog net472 4.91μs 1.62ns 6.27ns 0.32 0 0 2.02 KB
#6265 EnrichedLog net6.0 3.01μs 0.769ns 2.98ns 0.0303 0 0 2.2 KB
#6265 EnrichedLog netcoreapp3.1 4.25μs 1.33ns 5.14ns 0.0292 0 0 2.2 KB
#6265 EnrichedLog net472 4.94μs 1.17ns 4.54ns 0.319 0 0 2.02 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.45μs 0.771ns 2.99ns 0.0159 0 0 1.14 KB
master SendReceive netcoreapp3.1 1.78μs 0.775ns 3ns 0.0151 0 0 1.14 KB
master SendReceive net472 2.17μs 4.16ns 16.1ns 0.183 0 0 1.16 KB
#6265 SendReceive net6.0 1.38μs 0.829ns 3.21ns 0.0159 0 0 1.14 KB
#6265 SendReceive netcoreapp3.1 1.75μs 0.571ns 2.21ns 0.0156 0 0 1.14 KB
#6265 SendReceive net472 2.06μs 1.35ns 5.22ns 0.183 0 0 1.16 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.8μs 1.58ns 5.9ns 0.0224 0 0 1.6 KB
master EnrichedLog netcoreapp3.1 3.9μs 2.96ns 11.5ns 0.0214 0 0 1.65 KB
master EnrichedLog net472 4.39μs 2.57ns 9.96ns 0.323 0 0 2.04 KB
#6265 EnrichedLog net6.0 2.82μs 1.24ns 4.79ns 0.0225 0 0 1.6 KB
#6265 EnrichedLog netcoreapp3.1 3.86μs 1.18ns 4.57ns 0.0212 0 0 1.65 KB
#6265 EnrichedLog net472 4.34μs 4.53ns 17.5ns 0.322 0 0 2.04 KB
Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6265

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0 1.117 405.27 452.52

Faster 🎉 in #6265

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑netcoreapp3.1 1.140 639.46 560.88

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 406ns 0.284ns 1.1ns 0.00803 0 0 576 B
master StartFinishSpan netcoreapp3.1 640ns 0.846ns 3.28ns 0.00779 0 0 576 B
master StartFinishSpan net472 733ns 0.76ns 2.63ns 0.0916 0 0 578 B
master StartFinishScope net6.0 492ns 0.575ns 2.07ns 0.00976 0 0 696 B
master StartFinishScope netcoreapp3.1 736ns 0.596ns 2.31ns 0.00911 0 0 696 B
master StartFinishScope net472 873ns 1.52ns 5.48ns 0.104 0 0 658 B
#6265 StartFinishSpan net6.0 452ns 0.423ns 1.64ns 0.00806 0 0 576 B
#6265 StartFinishSpan netcoreapp3.1 560ns 0.587ns 2.27ns 0.00774 0 0 576 B
#6265 StartFinishSpan net472 671ns 0.491ns 1.84ns 0.0917 0 0 578 B
#6265 StartFinishScope net6.0 480ns 0.255ns 0.918ns 0.00965 0 0 696 B
#6265 StartFinishScope netcoreapp3.1 772ns 0.718ns 2.59ns 0.00927 0 0 696 B
#6265 StartFinishScope net472 907ns 1.29ns 4.99ns 0.104 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6265

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 1.242 599.99 745.48

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 600ns 0.444ns 1.72ns 0.00965 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 954ns 1.8ns 6.98ns 0.00911 0 0 696 B
master RunOnMethodBegin net472 1.12μs 0.714ns 2.76ns 0.104 0 0 658 B
#6265 RunOnMethodBegin net6.0 745ns 0.713ns 2.76ns 0.00977 0 0 696 B
#6265 RunOnMethodBegin netcoreapp3.1 925ns 1.09ns 4.21ns 0.00923 0 0 696 B
#6265 RunOnMethodBegin net472 1.16μs 1.36ns 5.25ns 0.104 0 0 658 B

bouwkast
bouwkast approved these changes Nov 8, 2024
View reviewed changes
Copy link
Contributor

@bouwkast bouwkast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you 🫡

.azure-pipelines/ultimate-pipeline.yml Show resolved Hide resolved
tracer/build/_build/docker/centos7.dockerfile Show resolved Hide resolved
tracer/src/Datadog.Trace/Activity/Handlers/IgnoreActivityHandler.cs Show resolved Hide resolved
tracer/src/Datadog.Trace/ClrProfiler/AutoInstrumentation/Http/WebRequest/WebRequestCommon.cs Show resolved Hide resolved
...rProfiler/AutoInstrumentation/AspNetCore/DefaultModelBindingContext_SetResult_Integration.cs Show resolved Hide resolved
tracer/src/Datadog.Trace/ClrProfiler/AutoInstrumentation/SupportedVersions.cs Show resolved Hide resolved
tracer/test/test-applications/integrations/Samples.OracleMDA/Samples.OracleMDA.csproj Outdated Show resolved Hide resolved
tracer/test/test-applications/integrations/Samples.WebRequest/RequestHelpers.cs Show resolved Hide resolved
@andrewlock
Copy link
Member Author

andrewlock commented Nov 8, 2024 •
edited
Loading

Throughput/Crank Report ⚡

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6265) (11.082M)   : 0, 11082150
    master (11.088M)   : 0, 11088207
    benchmarks/2.9.0 (11.033M)   : 0, 11032866

    section Automatic
    This PR (6265) (7.272M)   : 0, 7272245
    master (7.158M)   : 0, 7157708
    benchmarks/2.9.0 (7.786M)   : 0, 7785853

    section Trace stats
    master (7.480M)   : 0, 7479543

    section Manual
    master (11.166M)   : 0, 11165777

    section Manual + Automatic
    This PR (6265) (6.710M)   : 0, 6710498
    master (6.594M)   : 0, 6593575

    section DD_TRACE_ENABLED=0
    master (10.250M)   : 0, 10250493
Loading 
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6265) (9.562M)   : 0, 9561939
    master (9.364M)   : 0, 9363648
    benchmarks/2.9.0 (9.495M)   : 0, 9494821

    section Automatic
    This PR (6265) (6.442M)   : 0, 6441929
    master (6.450M)   : 0, 6450198

    section Trace stats
    master (6.679M)   : 0, 6679205

    section Manual
    master (9.587M)   : 0, 9587094

    section Manual + Automatic
    This PR (6265) (5.994M)   : 0, 5994391
    master (6.057M)   : 0, 6056858

    section DD_TRACE_ENABLED=0
    master (8.809M)   : 0, 8809017
Loading 
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6265) (9.751M)   : 0, 9750963
    master (9.954M)   : 0, 9953951
    benchmarks/2.9.0 (10.020M)   : 0, 10019592

    section Automatic
    This PR (6265) (5.896M)   : crit ,0, 5896431
    master (6.558M)   : 0, 6558012
    benchmarks/2.9.0 (7.255M)   : 0, 7255257

    section Trace stats
    master (7.152M)   : 0, 7152123

    section Manual
    master (9.703M)   : 0, 9703448

    section Manual + Automatic
    This PR (6265) (5.435M)   : crit ,0, 5434500
    master (6.043M)   : 0, 6042606

    section DD_TRACE_ENABLED=0
    master (9.169M)   : 0, 9169184
Loading

NachoEchevarria
NachoEchevarria reviewed Nov 11, 2024
View reviewed changes
tracer/src/Datadog.Trace/Iast/Aspects/System.IO/FileAspect.cs
@@ -30,20 +30,36 @@ public class FileAspect
[AspectMethodInsertBefore("System.IO.File::AppendAllLines(System.String,System.Collections.Generic.IEnumerable`1<System.String>,System.Text.Encoding)", 2)]
[AspectMethodInsertBefore("System.IO.File::AppendAllText(System.String,System.String)", 1)]
[AspectMethodInsertBefore("System.IO.File::AppendAllText(System.String,System.String,System.Text.Encoding)", 2)]
#if NET6_0_OR_GREATER // .NET 9 actually
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

NachoEchevarria
NachoEchevarria reviewed Nov 11, 2024
View reviewed changes
...lications/integrations/Samples.InstrumentedTests/Vulnerabilities/String/StringConcatTests.cs
@@ -265,7 +265,7 @@ public void GivenATaintedString_WhenCallingConcatWithStringNullParam_ResultIsTai
[Fact]
public void GivenATaintedString_WhenCallingConcat_ResultIsTainted()
{
AssertTaintedFormatWithOriginalCallCheck(":+-tainted-+:", String.Concat(taintedValue), () => String.Concat(taintedValue));
AssertTaintedFormatWithOriginalCallCheck(":+-tainted-+:", String.Concat(taintedValue), () => String.Concat(new[] { taintedValue }));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this change required?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, yes 😢 With the new overload rules in C#13, this selects the Concat(ReadOnlySpan<char>) overload. And then the compiler fails to compile it, because you can't have ReadOnlySpan<> in an Expression (or something like that). That was the cause of all the changes in this commit: 4b7dd3f

daniel-romano-DD
daniel-romano-DD approved these changes Nov 11, 2024
View reviewed changes
Copy link
Contributor

@daniel-romano-DD daniel-romano-DD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IAST wise, it looks fine. Thanks a lot

dudikeleti
dudikeleti approved these changes Nov 11, 2024
View reviewed changes
Copy link
Contributor

@dudikeleti dudikeleti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The debugger changes LGTM. Thanks for the update to support .NET 9

NachoEchevarria
NachoEchevarria approved these changes Nov 12, 2024
View reviewed changes
Copy link
Contributor

@NachoEchevarria NachoEchevarria left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!!!

gleocadie
gleocadie approved these changes Nov 12, 2024
View reviewed changes
Copy link
Collaborator

@gleocadie gleocadie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andrewlock andrewlock mentioned this pull request Nov 12, 2024
Open
@andrewlock
Update guard rails to allow instrumenting .NET 9 by default
ebffd00
@andrewlock
Add .NET 9 to generated package versions
1c8a7e4
@andrewlock
Update integrations using built-in types to 9.*.*
749dd0d 
Introduced a new SupportedVersions type to make it slightly easier to update in the future
@andrewlock
Add IAST aspect for new Type.GetMethod() and File overloads
2a0036b
@andrewlock
Update source gen
f907e8e
@andrewlock
Ignore new experimental activities added in .NET 9
b2edd58
@andrewlock
Bump package version in source generators to prevent complaints
882e1e6
@andrewlock
Allow the .NET tool to run on .NET 9 in the pipeline monitor
8d983dc
@andrewlock
"Fix" integration tests by changing the AllowWriteStreamBuffering val…
59ef73d 
…ue we use
@andrewlock
Add smoke tests for .NET 9
5bc5d13
@andrewlock
Add exclusions for overloads that IAST doesn't support yet
32e9e3c
@andrewlock
Downgrade the lang version of the IAST instrumented tests, to avoid u…
0cd47e6 
…sing span overloads (which are uninstrumented)
@andrewlock andrewlock merged commit 9c46186 into master Nov 13, 2024
84 of 88 checks passed
@andrewlock andrewlock deleted the andrew/dotnet-9 branch November 13, 2024 11:27
@github-actions github-actions bot added this to the vNext-v3 milestone Nov 13, 2024
@andrewlock andrewlock mentioned this pull request Nov 13, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@daniel-romano-DD daniel-romano-DD daniel-romano-DD approved these changes

@NachoEchevarria NachoEchevarria NachoEchevarria approved these changes

@gleocadie gleocadie gleocadie approved these changes

@bouwkast bouwkast bouwkast approved these changes

@dudikeleti dudikeleti dudikeleti approved these changes

@lucaspimentel lucaspimentel lucaspimentel approved these changes

Assignees
No one assigned
Labels
type:new-feature
Projects
None yet
Milestone
vNext-v3
Development

Successfully merging this pull request may close these issues.
7 participants
@andrewlock @lucaspimentel @dudikeleti @bouwkast @gleocadie @NachoEchevarria @daniel-romano-DD