Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ASM] Native CallTarget definitions #6252

Merged
merged 22 commits into from
Nov 22, 2024
Merged

Conversation

daniel-romano-DD
Copy link
Contributor

@daniel-romano-DD daniel-romano-DD commented Nov 6, 2024

Summary of changes

Moved CallTarget definitions to the Native library so we can spare the marshalling.
As from V3, there will not be a situation where a Managed V3 will have to specify its definitions to a newer Native library, we can remove the definitions from the managed part.
We must still keep the ability to receive definitions from a V2 managed library.

Reason for change

Marshalling of definitions from managed to native is taxing. Also, embedding the definitions in each managed library takes space, precious in Serverless scenarions

Implementation details

Moved the generation of the CallTargets definitions to a nuke step, removing them from the managed library.
Created a json file in dependabot folder with all the call target definitions declared. This file is read by the UpdatePackageVersions build step, instead of retrieving them by reflection from the compiled assemblies.
Removed uneeded exports from the native library.

Test coverage

Other details

@daniel-romano-DD daniel-romano-DD changed the base branch from master to dani/apm/NativeCallSites November 6, 2024 21:59
Copy link
Contributor Author

daniel-romano-DD commented Nov 6, 2024

@andrewlock
Copy link
Member

andrewlock commented Nov 6, 2024

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6252) - mean (72ms)  : 65, 80
     .   : milestone, 72,
    master - mean (71ms)  : 64, 78
     .   : milestone, 71,

    section CallTarget+Inlining+NGEN
    This PR (6252) - mean (1,066ms)  : 931, 1201
     .   : milestone, 1066,
    master - mean (1,109ms)  : 1083, 1135
     .   : milestone, 1109,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6252) - mean (128ms)  : 119, 136
     .   : milestone, 128,
    master - mean (109ms)  : 106, 112
     .   : milestone, 109,

    section CallTarget+Inlining+NGEN
    This PR (6252) - mean (773ms)  : 727, 820
     .   : milestone, 773,
    master - mean (772ms)  : 752, 791
     .   : milestone, 772,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6252) - mean (108ms)  : 101, 116
     .   : milestone, 108,
    master - mean (92ms)  : 91, 94
     .   : milestone, 92,

    section CallTarget+Inlining+NGEN
    This PR (6252) - mean (741ms)  : 652, 829
     .   : milestone, 741,
    master - mean (726ms)  : 711, 741
     .   : milestone, 726,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6252) - mean (191ms)  : 186, 195
     .   : milestone, 191,
    master - mean (192ms)  : 186, 198
     .   : milestone, 192,

    section CallTarget+Inlining+NGEN
    This PR (6252) - mean (1,094ms)  : 1066, 1123
     .   : milestone, 1094,
    master - mean (1,214ms)  : 1192, 1237
     .   : milestone, 1214,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6252) - mean (277ms)  : 272, 282
     .   : milestone, 277,
    master - mean (278ms)  : 272, 284
     .   : milestone, 278,

    section CallTarget+Inlining+NGEN
    This PR (6252) - mean (874ms)  : 831, 918
     .   : milestone, 874,
    master - mean (949ms)  : 933, 965
     .   : milestone, 949,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6252) - mean (266ms)  : 261, 271
     .   : milestone, 266,
    master - mean (266ms)  : 262, 270
     .   : milestone, 266,

    section CallTarget+Inlining+NGEN
    This PR (6252) - mean (858ms)  : 832, 885
     .   : milestone, 858,
    master - mean (931ms)  : 914, 948
     .   : milestone, 931,

Loading

@andrewlock
Copy link
Member

andrewlock commented Nov 6, 2024

Benchmarks Report for tracer 🐌

Benchmarks for #6252 compared to master:

  • 3 benchmarks are faster, with geometric mean 1.271
  • 2 benchmarks have fewer allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 7.86μs 43.8ns 297ns 0.0152 0.00758 0 5.61 KB
master StartStopWithChild netcoreapp3.1 9.92μs 55.2ns 362ns 0.0204 0.0102 0 5.8 KB
master StartStopWithChild net472 16.3μs 33.8ns 126ns 1.04 0.294 0.0952 6.21 KB
#6252 StartStopWithChild net6.0 7.8μs 43.4ns 298ns 0.0149 0.00746 0 5.62 KB
#6252 StartStopWithChild netcoreapp3.1 10.2μs 53.9ns 314ns 0.0203 0.0102 0 5.8 KB
#6252 StartStopWithChild net472 16.4μs 57.9ns 224ns 1.05 0.305 0.0907 6.21 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 486μs 371ns 1.44μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 672μs 329ns 1.19μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 871μs 612ns 2.37μs 0.431 0 0 3.3 KB
#6252 WriteAndFlushEnrichedTraces net6.0 511μs 503ns 1.95μs 0 0 0 2.7 KB
#6252 WriteAndFlushEnrichedTraces netcoreapp3.1 645μs 626ns 2.42μs 0 0 0 2.7 KB
#6252 WriteAndFlushEnrichedTraces net472 854μs 592ns 2.21μs 0.425 0 0 3.3 KB
Benchmarks.Trace.AspNetCoreBenchmark - Faster 🎉 Fewer allocations 🎉

Faster 🎉 in #6252

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.AspNetCoreBenchmark.SendRequest‑net6.0 1.352 202,753.17 149,919.63 several?
Benchmarks.Trace.AspNetCoreBenchmark.SendRequest‑netcoreapp3.1 1.338 223,666.70 167,205.01

Fewer allocations 🎉 in #6252

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.AspNetCoreBenchmark.SendRequest‑netcoreapp3.1 20.89 KB 17.27 KB -3.62 KB -17.32%
Benchmarks.Trace.AspNetCoreBenchmark.SendRequest‑net6.0 18.73 KB 14.47 KB -4.26 KB -22.74%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 203μs 1.12μs 9.11μs 0.203 0 0 18.73 KB
master SendRequest netcoreapp3.1 226μs 1.29μs 10.4μs 0.215 0 0 20.89 KB
master SendRequest net472 0.000253ns 0.000174ns 0.000626ns 0 0 0 0 b
#6252 SendRequest net6.0 151μs 877ns 7.39μs 0.138 0 0 14.47 KB
#6252 SendRequest netcoreapp3.1 167μs 939ns 7.69μs 0.159 0 0 17.27 KB
#6252 SendRequest net472 0.000344ns 0.000242ns 0.000937ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 567μs 2.79μs 11.8μs 0.551 0 0 41.62 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 685μs 3.58μs 18.6μs 0.34 0 0 41.81 KB
master WriteAndFlushEnrichedTraces net472 861μs 3.96μs 15.4μs 8.33 2.5 0.417 53.28 KB
#6252 WriteAndFlushEnrichedTraces net6.0 575μs 3.04μs 15.2μs 0.541 0 0 41.65 KB
#6252 WriteAndFlushEnrichedTraces netcoreapp3.1 678μs 3.19μs 13.9μs 0.331 0 0 41.84 KB
#6252 WriteAndFlushEnrichedTraces net472 852μs 3.85μs 14.9μs 8.08 2.55 0.425 53.29 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.39μs 1.29ns 4.98ns 0.0146 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 1.78μs 1.11ns 4.15ns 0.0133 0 0 1.02 KB
master ExecuteNonQuery net472 1.97μs 2.06ns 7.98ns 0.156 0.000983 0 987 B
#6252 ExecuteNonQuery net6.0 1.3μs 1.59ns 6.15ns 0.0139 0 0 1.02 KB
#6252 ExecuteNonQuery netcoreapp3.1 1.76μs 1.58ns 5.92ns 0.0141 0 0 1.02 KB
#6252 ExecuteNonQuery net472 2.12μs 1.45ns 5.61ns 0.156 0.00105 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.28μs 0.995ns 3.72ns 0.0136 0 0 976 B
master CallElasticsearch netcoreapp3.1 1.55μs 1.31ns 5.06ns 0.0133 0 0 976 B
master CallElasticsearch net472 2.59μs 1.32ns 4.93ns 0.158 0 0 995 B
master CallElasticsearchAsync net6.0 1.36μs 0.847ns 3.17ns 0.0129 0 0 952 B
master CallElasticsearchAsync netcoreapp3.1 1.55μs 0.998ns 3.87ns 0.0139 0 0 1.02 KB
master CallElasticsearchAsync net472 2.55μs 1.24ns 4.8ns 0.167 0 0 1.05 KB
#6252 CallElasticsearch net6.0 1.23μs 1.07ns 3.99ns 0.0135 0 0 976 B
#6252 CallElasticsearch netcoreapp3.1 1.63μs 0.497ns 1.86ns 0.0131 0 0 976 B
#6252 CallElasticsearch net472 2.52μs 1.2ns 4.49ns 0.157 0 0 995 B
#6252 CallElasticsearchAsync net6.0 1.24μs 0.724ns 2.71ns 0.013 0 0 952 B
#6252 CallElasticsearchAsync netcoreapp3.1 1.65μs 0.568ns 2.13ns 0.0137 0 0 1.02 KB
#6252 CallElasticsearchAsync net472 2.67μs 1.07ns 4.15ns 0.167 0 0 1.05 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.25μs 0.326ns 1.18ns 0.0131 0 0 952 B
master ExecuteAsync netcoreapp3.1 1.59μs 1.07ns 4.15ns 0.0127 0 0 952 B
master ExecuteAsync net472 1.82μs 0.702ns 2.72ns 0.145 0 0 915 B
#6252 ExecuteAsync net6.0 1.26μs 0.395ns 1.42ns 0.0133 0 0 952 B
#6252 ExecuteAsync netcoreapp3.1 1.6μs 0.779ns 2.92ns 0.0127 0 0 952 B
#6252 ExecuteAsync net472 1.78μs 1.02ns 3.95ns 0.145 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.37μs 1.68ns 6.49ns 0.0328 0 0 2.31 KB
master SendAsync netcoreapp3.1 5.36μs 1.96ns 7.05ns 0.0373 0 0 2.85 KB
master SendAsync net472 7.34μs 1.64ns 6.37ns 0.493 0 0 3.12 KB
#6252 SendAsync net6.0 4.34μs 1.31ns 4.55ns 0.0327 0 0 2.31 KB
#6252 SendAsync netcoreapp3.1 5.27μs 1.5ns 5.62ns 0.0369 0 0 2.85 KB
#6252 SendAsync net472 7.2μs 1.21ns 4.7ns 0.493 0 0 3.12 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.49μs 1.2ns 4.5ns 0.0231 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 2.22μs 2.7ns 10.5ns 0.022 0 0 1.64 KB
master EnrichedLog net472 2.52μs 1.01ns 3.78ns 0.25 0 0 1.57 KB
#6252 EnrichedLog net6.0 1.55μs 1.55ns 5.99ns 0.0232 0 0 1.64 KB
#6252 EnrichedLog netcoreapp3.1 2.31μs 2.06ns 7.7ns 0.0218 0 0 1.64 KB
#6252 EnrichedLog net472 2.6μs 0.899ns 3.48ns 0.249 0 0 1.57 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 119μs 135ns 523ns 0.0596 0 0 4.28 KB
master EnrichedLog netcoreapp3.1 125μs 143ns 534ns 0 0 0 4.28 KB
master EnrichedLog net472 153μs 84.7ns 328ns 0.685 0.228 0 4.46 KB
#6252 EnrichedLog net6.0 120μs 134ns 518ns 0.0598 0 0 4.28 KB
#6252 EnrichedLog netcoreapp3.1 124μs 140ns 523ns 0 0 0 4.28 KB
#6252 EnrichedLog net472 151μs 175ns 654ns 0.683 0.228 0 4.46 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 3.08μs 0.73ns 2.83ns 0.0308 0 0 2.2 KB
master EnrichedLog netcoreapp3.1 4.14μs 1.48ns 5.53ns 0.029 0 0 2.2 KB
master EnrichedLog net472 4.9μs 1.42ns 5.51ns 0.32 0 0 2.02 KB
#6252 EnrichedLog net6.0 3.09μs 1.18ns 4.55ns 0.0309 0 0 2.2 KB
#6252 EnrichedLog netcoreapp3.1 4.04μs 1.03ns 4.01ns 0.0283 0 0 2.2 KB
#6252 EnrichedLog net472 5.15μs 1.1ns 4.27ns 0.32 0 0 2.02 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.31μs 0.858ns 3.32ns 0.0159 0 0 1.14 KB
master SendReceive netcoreapp3.1 1.82μs 1.1ns 4.25ns 0.0154 0 0 1.14 KB
master SendReceive net472 2.14μs 0.844ns 3.27ns 0.183 0 0 1.16 KB
#6252 SendReceive net6.0 1.42μs 0.58ns 2.17ns 0.0157 0 0 1.14 KB
#6252 SendReceive netcoreapp3.1 1.81μs 1.01ns 3.91ns 0.0154 0 0 1.14 KB
#6252 SendReceive net472 2.07μs 1.97ns 7.38ns 0.183 0 0 1.16 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.75μs 0.776ns 2.9ns 0.022 0 0 1.6 KB
master EnrichedLog netcoreapp3.1 3.99μs 1.56ns 6.05ns 0.0219 0 0 1.65 KB
master EnrichedLog net472 4.32μs 1.49ns 5.56ns 0.323 0 0 2.04 KB
#6252 EnrichedLog net6.0 2.73μs 3.38ns 12.7ns 0.0218 0 0 1.6 KB
#6252 EnrichedLog netcoreapp3.1 3.94μs 1.44ns 5.58ns 0.0217 0 0 1.65 KB
#6252 EnrichedLog net472 4.34μs 3.1ns 12ns 0.323 0 0 2.04 KB
Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6252

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1 1.134 793.55 699.68

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 408ns 0.525ns 2.03ns 0.00802 0 0 576 B
master StartFinishSpan netcoreapp3.1 542ns 0.746ns 2.89ns 0.00778 0 0 576 B
master StartFinishSpan net472 651ns 0.765ns 2.96ns 0.0917 0 0 578 B
master StartFinishScope net6.0 501ns 0.739ns 2.86ns 0.00971 0 0 696 B
master StartFinishScope netcoreapp3.1 793ns 0.938ns 3.38ns 0.00942 0 0 696 B
master StartFinishScope net472 934ns 1.18ns 4.4ns 0.105 0 0 658 B
#6252 StartFinishSpan net6.0 395ns 0.428ns 1.66ns 0.00801 0 0 576 B
#6252 StartFinishSpan netcoreapp3.1 583ns 0.799ns 3.09ns 0.00778 0 0 576 B
#6252 StartFinishSpan net472 666ns 1.52ns 5.87ns 0.0917 0 0 578 B
#6252 StartFinishScope net6.0 540ns 0.741ns 2.87ns 0.00971 0 0 696 B
#6252 StartFinishScope netcoreapp3.1 701ns 0.866ns 3.36ns 0.00925 0 0 696 B
#6252 StartFinishScope net472 863ns 1.76ns 6.81ns 0.104 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 652ns 0.805ns 3.12ns 0.00988 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 929ns 1.69ns 6.54ns 0.00932 0 0 696 B
master RunOnMethodBegin net472 1.11μs 1.83ns 7.07ns 0.104 0 0 658 B
#6252 RunOnMethodBegin net6.0 664ns 1.06ns 4.11ns 0.00972 0 0 696 B
#6252 RunOnMethodBegin netcoreapp3.1 1.03μs 1.55ns 5.57ns 0.00917 0 0 696 B
#6252 RunOnMethodBegin net472 1.13μs 1.57ns 6.08ns 0.104 0 0 658 B

@andrewlock
Copy link
Member

andrewlock commented Nov 6, 2024

Throughput/Crank Report ⚡

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6252) (11.163M)   : 0, 11162819
    master (11.155M)   : 0, 11154721
    benchmarks/2.9.0 (11.033M)   : 0, 11032866

    section Automatic
    This PR (6252) (7.227M)   : 0, 7227309
    master (7.297M)   : 0, 7297200
    benchmarks/2.9.0 (7.786M)   : 0, 7785853

    section Trace stats
    master (7.684M)   : 0, 7684145

    section Manual
    master (11.069M)   : 0, 11068901

    section Manual + Automatic
    This PR (6252) (6.778M)   : 0, 6778057
    master (6.786M)   : 0, 6785523

    section DD_TRACE_ENABLED=0
    master (10.249M)   : 0, 10249493

Loading
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6252) (9.413M)   : 0, 9413240
    master (9.602M)   : 0, 9601778
    benchmarks/2.9.0 (9.495M)   : 0, 9494821

    section Automatic
    This PR (6252) (6.435M)   : 0, 6435246
    master (6.210M)   : 0, 6209560

    section Trace stats
    master (6.728M)   : 0, 6728350

    section Manual
    master (9.437M)   : 0, 9436743

    section Manual + Automatic
    This PR (6252) (6.006M)   : 0, 6005837
    master (5.810M)   : 0, 5809535

    section DD_TRACE_ENABLED=0
    master (8.705M)   : 0, 8705485

Loading
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6252) (10.225M)   : 0, 10224745
    master (10.329M)   : 0, 10328534
    benchmarks/2.9.0 (10.020M)   : 0, 10019592

    section Automatic
    This PR (6252) (5.723M)   : crit ,0, 5723439
    master (6.617M)   : 0, 6616689
    benchmarks/2.9.0 (7.255M)   : 0, 7255257

    section Trace stats
    master (7.166M)   : 0, 7165623

    section Manual
    master (10.093M)   : 0, 10093300

    section Manual + Automatic
    This PR (6252) (5.187M)   : crit ,0, 5186760
    master (6.311M)   : 0, 6310782

    section DD_TRACE_ENABLED=0
    master (9.459M)   : 0, 9459125

Loading

@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Nov 6, 2024

Datadog Report

Branch report: dani/apm/NativeCallTargets
Commit report: 35ca88a
Test service: dd-trace-dotnet

✅ 0 Failed, 462509 Passed, 3594 Skipped, 32h 32m 28.28s Total Time
❄️ 1 New Flaky

New Flaky Tests (1)

  • SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.StackExchangeRedisTests - Last Failure

    Expand for error
     Results do not match.
     Differences:
     Received: StackExchangeRedisTests.Latest.SchemaV1.received.txt
     Verified: StackExchangeRedisTests.Latest.SchemaV1.verified.txt
     Received Content:
     [
       {
         TraceId: Id_1,
         SpanId: Id_2,
         Name: redis.command,
     ...
    

@daniel-romano-DD daniel-romano-DD force-pushed the dani/apm/NativeCallTargets branch 2 times, most recently from c803d7d to afed6e3 Compare November 7, 2024 13:21
@daniel-romano-DD daniel-romano-DD added area:native-library Automatic instrumentation native C++ code (Datadog.Trace.ClrProfiler.Native) area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) labels Nov 7, 2024
@andrewlock
Copy link
Member

andrewlock commented Nov 8, 2024

Benchmarks Report for appsec 🐌

Benchmarks for #6252 compared to master:

  • All benchmarks have the same speed
  • 2 benchmarks have fewer allocations
  • 1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 72.3μs 67.8ns 263ns 0.0716 0 0 6 KB
master AllCycleSimpleBody netcoreapp3.1 62.5μs 58.7ns 220ns 0.0938 0 0 6.95 KB
master AllCycleSimpleBody net472 49.1μs 72ns 279ns 1.31 0 0 8.33 KB
master AllCycleMoreComplexBody net6.0 78.4μs 55ns 213ns 0.117 0 0 9.51 KB
master AllCycleMoreComplexBody netcoreapp3.1 69.4μs 56.4ns 211ns 0.139 0 0 10.36 KB
master AllCycleMoreComplexBody net472 56.5μs 85.2ns 330ns 1.87 0.0283 0 11.85 KB
master ObjectExtractorSimpleBody net6.0 145ns 0.12ns 0.448ns 0.00395 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 209ns 0.137ns 0.494ns 0.00369 0 0 272 B
master ObjectExtractorSimpleBody net472 171ns 0.0861ns 0.334ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 3.04μs 2.64ns 9.86ns 0.0533 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 3.91μs 3.08ns 11.9ns 0.0507 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 4.36μs 1.92ns 7.44ns 0.601 0.00656 0 3.8 KB
#6252 AllCycleSimpleBody net6.0 72.2μs 53.8ns 208ns 0.0721 0 0 6 KB
#6252 AllCycleSimpleBody netcoreapp3.1 63.8μs 36.3ns 126ns 0.0958 0 0 6.95 KB
#6252 AllCycleSimpleBody net472 47.7μs 55.8ns 216ns 1.31 0 0 8.34 KB
#6252 AllCycleMoreComplexBody net6.0 78.6μs 98ns 367ns 0.118 0 0 9.51 KB
#6252 AllCycleMoreComplexBody netcoreapp3.1 69μs 60.6ns 227ns 0.138 0 0 10.37 KB
#6252 AllCycleMoreComplexBody net472 54.8μs 52.6ns 204ns 1.88 0.0272 0 11.85 KB
#6252 ObjectExtractorSimpleBody net6.0 146ns 0.273ns 1.02ns 0.00392 0 0 280 B
#6252 ObjectExtractorSimpleBody netcoreapp3.1 206ns 0.285ns 1.07ns 0.00371 0 0 272 B
#6252 ObjectExtractorSimpleBody net472 165ns 0.124ns 0.462ns 0.0446 0 0 281 B
#6252 ObjectExtractorMoreComplexBody net6.0 3.14μs 3.38ns 12.6ns 0.053 0 0 3.78 KB
#6252 ObjectExtractorMoreComplexBody netcoreapp3.1 3.9μs 1.8ns 6.72ns 0.0509 0 0 3.69 KB
#6252 ObjectExtractorMoreComplexBody net472 4.34μs 3.23ns 12.5ns 0.603 0.0065 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 38.2μs 26.2ns 102ns 0.454 0 0 32.4 KB
master EncodeArgs netcoreapp3.1 54.3μs 36.7ns 142ns 0.432 0 0 32.4 KB
master EncodeArgs net472 66.4μs 27.9ns 104ns 5.15 0.0661 0 32.5 KB
master EncodeLegacyArgs net6.0 76.2μs 407ns 2.11μs 0 0 0 2.14 KB
master EncodeLegacyArgs netcoreapp3.1 109μs 254ns 983ns 0 0 0 2.14 KB
master EncodeLegacyArgs net472 154μs 97.1ns 376ns 0.31 0 0 2.15 KB
#6252 EncodeArgs net6.0 38.4μs 26.9ns 104ns 0.441 0 0 32.4 KB
#6252 EncodeArgs netcoreapp3.1 54.7μs 24.9ns 96.6ns 0.439 0 0 32.4 KB
#6252 EncodeArgs net472 66.4μs 41.8ns 162ns 5.14 0.0663 0 32.5 KB
#6252 EncodeLegacyArgs net6.0 78.7μs 374ns 1.45μs 0 0 0 2.14 KB
#6252 EncodeLegacyArgs netcoreapp3.1 105μs 158ns 612ns 0 0 0 2.14 KB
#6252 EncodeLegacyArgs net472 152μs 92.5ns 358ns 0.306 0 0 2.15 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 185μs 60.9ns 236ns 0 0 0 2.44 KB
master RunWafRealisticBenchmark netcoreapp3.1 198μs 238ns 921ns 0 0 0 2.39 KB
master RunWafRealisticBenchmark net472 210μs 141ns 545ns 0.315 0 0 2.46 KB
master RunWafRealisticBenchmarkWithAttack net6.0 122μs 106ns 381ns 0 0 0 1.47 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 130μs 155ns 598ns 0 0 0 1.46 KB
master RunWafRealisticBenchmarkWithAttack net472 138μs 42.5ns 153ns 0.208 0 0 1.49 KB
#6252 RunWafRealisticBenchmark net6.0 185μs 65.7ns 246ns 0 0 0 2.44 KB
#6252 RunWafRealisticBenchmark netcoreapp3.1 194μs 195ns 757ns 0 0 0 2.39 KB
#6252 RunWafRealisticBenchmark net472 212μs 119ns 461ns 0.314 0 0 2.46 KB
#6252 RunWafRealisticBenchmarkWithAttack net6.0 124μs 70.8ns 274ns 0 0 0 1.47 KB
#6252 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 130μs 213ns 825ns 0 0 0 1.46 KB
#6252 RunWafRealisticBenchmarkWithAttack net472 139μs 47.3ns 177ns 0.207 0 0 1.48 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #6252

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472 272.91 KB 278.53 KB 5.62 KB 2.06%

Fewer allocations 🎉 in #6252

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 255.27 KB 253.38 KB -1.89 KB -0.74%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472 60.49 KB 59.04 KB -1.45 KB -2.39%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 52.9μs 209ns 782ns 0 0 0 43.44 KB
master StringConcatBenchmark netcoreapp3.1 54.2μs 287ns 1.38μs 0 0 0 42.64 KB
master StringConcatBenchmark net472 38μs 165ns 616ns 0 0 0 60.49 KB
master StringConcatAspectBenchmark net6.0 295μs 4.71μs 43.7μs 0 0 0 253.95 KB
master StringConcatAspectBenchmark netcoreapp3.1 354μs 1.97μs 12.1μs 0 0 0 255.27 KB
master StringConcatAspectBenchmark net472 278μs 6.14μs 58.9μs 0 0 0 272.91 KB
#6252 StringConcatBenchmark net6.0 61.1μs 769ns 7.53μs 0 0 0 43.44 KB
#6252 StringConcatBenchmark netcoreapp3.1 62.5μs 869ns 8.65μs 0 0 0 42.64 KB
#6252 StringConcatBenchmark net472 37.6μs 119ns 429ns 0 0 0 59.04 KB
#6252 StringConcatAspectBenchmark net6.0 310μs 1.41μs 5.46μs 0 0 0 255.08 KB
#6252 StringConcatAspectBenchmark netcoreapp3.1 334μs 1.7μs 10.9μs 0 0 0 253.38 KB
#6252 StringConcatAspectBenchmark net472 283μs 6.3μs 61.4μs 0 0 0 278.53 KB

@daniel-romano-DD daniel-romano-DD force-pushed the dani/apm/NativeCallSites branch from 5f466a9 to 6463601 Compare November 8, 2024 18:52
@daniel-romano-DD daniel-romano-DD force-pushed the dani/apm/NativeCallTargets branch from bc03048 to faade40 Compare November 8, 2024 18:53
@daniel-romano-DD daniel-romano-DD marked this pull request as ready for review November 9, 2024 11:35
@daniel-romano-DD daniel-romano-DD requested review from a team as code owners November 9, 2024 11:35
@daniel-romano-DD daniel-romano-DD requested a review from a team as a code owner November 11, 2024 11:39
@daniel-romano-DD daniel-romano-DD force-pushed the dani/apm/NativeCallTargets branch from 0e2e051 to fd471c7 Compare November 11, 2024 11:39
Copy link
Member

@andrewlock andrewlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work 👍

Mostly just some small suggestions e.g. be more aggressive about validation and error cases in the generator file - we can just throw as soon as we spot anything invalid, better to break the build than have something invalid in there.

A bigger concern is the changing of the native interface - I don't think we should change the native method that is called from the managed size, just add a new one?

tracer/build/_build/Build.Steps.cs Outdated Show resolved Hide resolved
tracer/build/_build/Build.Utilities.cs Outdated Show resolved Hide resolved
tracer/build/_build/CodeGenerators/CallTargetsGenerator.cs Outdated Show resolved Hide resolved
tracer/build/_build/CodeGenerators/CallTargetsGenerator.cs Outdated Show resolved Hide resolved
tracer/src/Datadog.Tracer.Native/cor_profiler.h Outdated Show resolved Hide resolved
Copy link
Member

@andrewlock andrewlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Nice Work!

@daniel-romano-DD daniel-romano-DD force-pushed the dani/apm/NativeCallTargets branch from 02b9bdf to 7effd69 Compare November 22, 2024 14:46
@daniel-romano-DD daniel-romano-DD force-pushed the dani/apm/NativeCallTargets branch from 7effd69 to 35ca88a Compare November 22, 2024 18:34
@daniel-romano-DD daniel-romano-DD merged commit a698ef0 into master Nov 22, 2024
66 of 70 checks passed
@daniel-romano-DD daniel-romano-DD deleted the dani/apm/NativeCallTargets branch November 22, 2024 21:20
@github-actions github-actions bot added this to the vNext-v3 milestone Nov 22, 2024
veerbia pushed a commit that referenced this pull request Dec 16, 2024
## Summary of changes
Moved CallTarget definitions to the Native library so we can spare the
marshalling.
As from V3, there will not be a situation where a Managed V3 will have
to specify its definitions to a newer Native library, we can remove the
definitions from the managed part.
We must still keep the ability to receive definitions from a V2 managed
library.

## Reason for change
Marshalling of definitions from managed to native is taxing. Also,
embedding the definitions in each managed library takes space, precious
in Serverless scenarions

## Implementation details
Moved the generation of the `CallTargets` definitions to a nuke step,
removing them from the managed library.
Created a json file in dependabot folder with all the call target
definitions declared. This file is read by the UpdatePackageVersions
build step, instead of retrieving them by reflection from the compiled
assemblies.
Removed uneeded exports from the native library.

## Test coverage

## Other details
<!-- Fixes #{issue} -->

<!-- ⚠️ Note: where possible, please obtain 2 approvals prior to
merging. Unless CODEOWNERS specifies otherwise, for external teams it is
typically best to have one review from a team member, and one review
from apm-dotnet. Trivial changes do not require 2 reviews. -->

---------

Co-authored-by: Andrew Lock <andrew.lock@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:asm area:native-library Automatic instrumentation native C++ code (Datadog.Trace.ClrProfiler.Native) area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants