feat(trace-exporter): setup boilerplate for data-pipeline integration

[ganeshnj]

Summary of changes

Use native TraceExporter to send traces.

Reason for change

As a strategic goal to move common stuff across the trace SDKs to native components, this PR targets the Trace exporter component.

Goal for the PR is to get into merging state so when the blocking changes are fixed. We can speed the merging.

Implementation details

TraceExporter implements the IApi interface which is responsible to send the traces to the agent. It has no-op for stats since the stats computation happens inside the native side when enabled.

Remaining work (will be different PRs)

• Conditionally enable data pipeline on Azure App Services only
• Improve logging

⚠️ This change targets a feature branch since we can't merge this in current form because some ad-hoc hacking done to get the CI working.

⚠️ reviewers must focus on changes in tracer dir only, rest of changes are just boilerplate for making CI work and it will be removed after vcpkg integration.

Test coverage

• More tests are added to make sure TraceExporter works with TCP, UDP and Windows named pipes
• Tests are updated to exercise both when data pipeline is enabled/disabled.

Other details

[datadog-ddstaging]

Datadog Report

Branch report: ganeshnj/feature/datapipeline-bindings
Commit report: d982e5f
Test service: dd-trace-dotnet

❌ 926 Failed (0 Known Flaky), 56736 Passed, 208 Skipped, 5h 47m 35.51s Total Time

❌ Failed Tests (926)

This report shows up to 5 failed tests.

• StartStopWithChild - Benchmarks.Trace.ActivityBenchmark - Details

• StartStopWithChild - Benchmarks.Trace.ActivityBenchmark - Details

• StartStopWithChild - Benchmarks.Trace.ActivityBenchmark - Details

• AzureFunctionsTests+InProcessRuntimeV4.SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests - Details

  Expand for error

   Expected collection to contain at least 21 item(s) because we want to ensure that we don't timeout while waiting for spans from the mock tracer agent, but found 0: {empty}.
  

• AzureFunctionsTests+IsolatedRuntimeV4SdkV1.SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests - Details

  Expand for error

   Expected collection to contain at least 21 item(s) because we want to ensure that we don't timeout while waiting for spans from the mock tracer agent, but found 0: {empty}.
  

[andrewlock]

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

• This PR (6314)
• master

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

• Welch test with statistical test for significance of 5%
• Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6314) - mean (68ms)  : 65, 71
     .   : milestone, 68,
    master - mean (68ms)  : 66, 69
     .   : milestone, 68,

    section CallTarget+Inlining+NGEN
    This PR (6314) - mean (1,018ms)  : 1002, 1034
     .   : milestone, 1018,
    master - mean (1,007ms)  : 985, 1028
     .   : milestone, 1007,

Loading

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6314) - mean (102ms)  : 100, 105
     .   : milestone, 102,
    master - mean (102ms)  : 100, 104
     .   : milestone, 102,

    section CallTarget+Inlining+NGEN
    This PR (6314) - mean (695ms)  : 682, 707
     .   : milestone, 695,
    master - mean (714ms)  : 644, 784
     .   : milestone, 714,

Loading

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6314) - mean (89ms)  : 87, 91
     .   : milestone, 89,
    master - mean (90ms)  : 87, 92
     .   : milestone, 90,

    section CallTarget+Inlining+NGEN
    This PR (6314) - mean (664ms)  : 648, 681
     .   : milestone, 664,
    master - mean (659ms)  : 636, 682
     .   : milestone, 659,

Loading

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6314) - mean (190ms)  : 186, 194
     .   : milestone, 190,
    master - mean (190ms)  : 186, 193
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (6314) - mean (1,164ms)  : 1143, 1184
     .   : milestone, 1164,
    master - mean (1,114ms)  : 1084, 1143
     .   : milestone, 1114,

Loading

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6314) - mean (269ms)  : 265, 272
     .   : milestone, 269,
    master - mean (269ms)  : 265, 274
     .   : milestone, 269,

    section CallTarget+Inlining+NGEN
    This PR (6314) - mean (927ms)  : 916, 938
     .   : milestone, 927,
    master - mean (882ms)  : 843, 922
     .   : milestone, 882,

Loading

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6314) - mean (262ms)  : 259, 265
     .   : milestone, 262,
    master - mean (261ms)  : 257, 264
     .   : milestone, 261,

    section CallTarget+Inlining+NGEN
    This PR (6314) - mean (928ms)  : crit, 905, 951
     .   : crit, milestone, 928,
    master - mean (870ms)  : 844, 896
     .   : milestone, 870,

Loading

[andrewlock]

Nice work 👍

[GreenMatan]

Reviewed Debugger's related changes and LGTM