[Profiler] Cleanup repository #6547

chrisnas · 2025-01-15T10:46:56Z

Summary of changes

Cleanup the documentation and fix compilation errors in non demo applications

Reason for change

Cleanup repository

Implementation details

Test coverage

Other details

datadog-ddstaging · 2025-01-15T11:10:38Z

Datadog Report

Branch report: chrisnas/update_profiler_doc
Commit report: 9c0d298
Test service: dd-trace-dotnet

❌ 1 Failed (0 Known Flaky), 243375 Passed, 2459 Skipped, 20h 54m 48.28s Total Time

❌ Failed Tests (1)

Computer01Test.CheckGenericsForNewCpuProfiler - Datadog.Profiler.SmokeTests - Details

Expand for error

 Exit code of "Samples.Computer01" should be 0 instead of 134 (= 0x86)
 Expected: True
 Actual:   False

andrewlock · 2025-01-15T12:17:02Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6547) - mean (69ms)  : 66, 72
     .   : milestone, 69,
    master - mean (69ms)  : 66, 72
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (6547) - mean (984ms)  : 952, 1015
     .   : milestone, 984,
    master - mean (975ms)  : 950, 1001
     .   : milestone, 975,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6547) - mean (108ms)  : 105, 110
     .   : milestone, 108,
    master - mean (108ms)  : 105, 110
     .   : milestone, 108,

    section CallTarget+Inlining+NGEN
    This PR (6547) - mean (677ms)  : 661, 693
     .   : milestone, 677,
    master - mean (679ms)  : 664, 694
     .   : milestone, 679,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6547) - mean (91ms)  : 89, 93
     .   : milestone, 91,
    master - mean (91ms)  : 90, 93
     .   : milestone, 91,

    section CallTarget+Inlining+NGEN
    This PR (6547) - mean (630ms)  : 615, 644
     .   : milestone, 630,
    master - mean (635ms)  : 619, 651
     .   : milestone, 635,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6547) - mean (191ms)  : 186, 195
     .   : milestone, 191,
    master - mean (194ms)  : 189, 198
     .   : milestone, 194,

    section CallTarget+Inlining+NGEN
    This PR (6547) - mean (1,091ms)  : 1066, 1115
     .   : milestone, 1091,
    master - mean (1,094ms)  : 1067, 1122
     .   : milestone, 1094,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6547) - mean (276ms)  : 272, 280
     .   : milestone, 276,
    master - mean (277ms)  : 273, 281
     .   : milestone, 277,

    section CallTarget+Inlining+NGEN
    This PR (6547) - mean (870ms)  : 839, 900
     .   : milestone, 870,
    master - mean (871ms)  : 849, 892
     .   : milestone, 871,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6547) - mean (264ms)  : 260, 268
     .   : milestone, 264,
    master - mean (267ms)  : 263, 271
     .   : milestone, 267,

    section CallTarget+Inlining+NGEN
    This PR (6547) - mean (842ms)  : 810, 873
     .   : milestone, 842,
    master - mean (855ms)  : 820, 889
     .   : milestone, 855,

andrewlock · 2025-01-16T02:40:08Z

Benchmarks Report for tracer 🐌

Benchmarks for #6547 compared to master:

1 benchmarks are slower, with geometric mean 1.249
All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	8.32μs	44.7ns	253ns	0.0165	0.00824	0	5.62 KB
master	`StartStopWithChild`	netcoreapp3.1	10.2μs	50.7ns	287ns	0.0199	0.00993	0	5.8 KB
master	`StartStopWithChild`	net472	16.3μs	35.3ns	137ns	1.06	0.307	0.102	6.21 KB
#6547	`StartStopWithChild`	net6.0	7.99μs	44.6ns	275ns	0.017	0.00848	0	5.61 KB
#6547	`StartStopWithChild`	netcoreapp3.1	9.85μs	55.1ns	370ns	0.0246	0.00984	0	5.8 KB
#6547	`StartStopWithChild`	net472	16.1μs	37.6ns	146ns	1.05	0.318	0.103	6.21 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	470μs	385ns	1.49μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	662μs	269ns	1.04μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	859μs	722ns	2.79μs	0.428	3.3 KB
#6547	`WriteAndFlushEnrichedTraces`	net6.0	477μs	328ns	1.23μs	0	2.7 KB
#6547	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	676μs	528ns	2.04μs	0	2.7 KB
#6547	`WriteAndFlushEnrichedTraces`	net472	856μs	482ns	1.87μs	0.428	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	157μs	923ns	8.71μs	0.147	14.47 KB
master	`SendRequest`	netcoreapp3.1	175μs	1.12μs	11.1μs	0.191	17.27 KB
master	`SendRequest`	net472	0.00137ns	0.000761ns	0.00285ns	0	0 b
#6547	`SendRequest`	net6.0	153μs	865ns	8.11μs	0.205	14.47 KB
#6547	`SendRequest`	netcoreapp3.1	177μs	1.17μs	11.7μs	0.173	17.27 KB
#6547	`SendRequest`	net472	0.000924ns	0.0005ns	0.00194ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	571μs	2.67μs	10μs	0.584	0	0	41.6 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	695μs	3.86μs	23.5μs	0.353	0	0	41.76 KB
master	`WriteAndFlushEnrichedTraces`	net472	825μs	2.97μs	10.7μs	8.5	2.55	0.425	53.27 KB
#6547	`WriteAndFlushEnrichedTraces`	net6.0	561μs	2.87μs	14.9μs	0.558	0	0	41.55 KB
#6547	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	682μs	3.38μs	18.5μs	0.336	0	0	41.66 KB
#6547	`WriteAndFlushEnrichedTraces`	net472	824μs	2.13μs	7.99μs	8.41	2.4	0.401	53.27 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	1.32μs	1ns	3.87ns	0.0147	0	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.77μs	1.85ns	6.92ns	0.0133	0	1.02 KB
master	`ExecuteNonQuery`	net472	2.06μs	3.08ns	11.9ns	0.156	0.00102	987 B
#6547	`ExecuteNonQuery`	net6.0	1.45μs	1.28ns	4.8ns	0.014	0	1.02 KB
#6547	`ExecuteNonQuery`	netcoreapp3.1	1.84μs	1.51ns	5.86ns	0.0137	0	1.02 KB
#6547	`ExecuteNonQuery`	net472	2.11μs	2.66ns	10.3ns	0.156	0.00104	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.26μs	1.05ns	3.94ns	0.0133	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.57μs	1.18ns	4.58ns	0.0132	976 B
master	`CallElasticsearch`	net472	2.64μs	1.83ns	7.08ns	0.157	995 B
master	`CallElasticsearchAsync`	net6.0	1.28μs	0.793ns	2.97ns	0.0133	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.59μs	0.636ns	2.38ns	0.0135	1.02 KB
master	`CallElasticsearchAsync`	net472	2.65μs	2.23ns	8.62ns	0.166	1.05 KB
#6547	`CallElasticsearch`	net6.0	1.27μs	0.749ns	2.8ns	0.0134	976 B
#6547	`CallElasticsearch`	netcoreapp3.1	1.58μs	0.746ns	2.79ns	0.0135	976 B
#6547	`CallElasticsearch`	net472	2.58μs	6.47ns	25ns	0.157	995 B
#6547	`CallElasticsearchAsync`	net6.0	1.38μs	0.509ns	1.97ns	0.0131	952 B
#6547	`CallElasticsearchAsync`	netcoreapp3.1	1.72μs	0.895ns	3.47ns	0.014	1.02 KB
#6547	`CallElasticsearchAsync`	net472	2.59μs	1.47ns	5.7ns	0.167	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.23μs	0.583ns	2.18ns	0.0132	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.57μs	2.12ns	7.92ns	0.0126	952 B
master	`ExecuteAsync`	net472	1.83μs	0.494ns	1.91ns	0.145	915 B
#6547	`ExecuteAsync`	net6.0	1.31μs	0.606ns	2.18ns	0.0132	952 B
#6547	`ExecuteAsync`	netcoreapp3.1	1.7μs	1.14ns	4.26ns	0.0125	952 B
#6547	`ExecuteAsync`	net472	1.79μs	1.06ns	4.09ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.4μs	2.07ns	7.76ns	0.0325	2.31 KB
master	`SendAsync`	netcoreapp3.1	5.21μs	1.27ns	4.76ns	0.039	2.85 KB
master	`SendAsync`	net472	7.35μs	2.08ns	8.06ns	0.493	3.12 KB
#6547	`SendAsync`	net6.0	4.37μs	1.15ns	4.3ns	0.0326	2.31 KB
#6547	`SendAsync`	netcoreapp3.1	5.48μs	2.05ns	7.68ns	0.0384	2.85 KB
#6547	`SendAsync`	net472	7.4μs	1.15ns	4.3ns	0.493	3.12 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.5μs	0.724ns	2.8ns	0.0233	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.24μs	5.12ns	19.8ns	0.022	1.64 KB
master	`EnrichedLog`	net472	2.63μs	1.34ns	5.19ns	0.25	1.57 KB
#6547	`EnrichedLog`	net6.0	1.46μs	0.801ns	3ns	0.0233	1.64 KB
#6547	`EnrichedLog`	netcoreapp3.1	2.35μs	0.795ns	2.75ns	0.0221	1.64 KB
#6547	`EnrichedLog`	net472	2.59μs	1.08ns	4.05ns	0.249	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	117μs	178ns	689ns	0.0579	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	123μs	186ns	722ns	0	0	4.28 KB
master	`EnrichedLog`	net472	150μs	117ns	452ns	0.684	0.228	4.46 KB
#6547	`EnrichedLog`	net6.0	115μs	111ns	428ns	0	0	4.28 KB
#6547	`EnrichedLog`	netcoreapp3.1	120μs	204ns	792ns	0	0	4.28 KB
#6547	`EnrichedLog`	net472	150μs	129ns	501ns	0.677	0.226	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.98μs	0.65ns	2.52ns	0.0299	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.33μs	2.1ns	8.15ns	0.0281	2.2 KB
master	`EnrichedLog`	net472	4.88μs	1.59ns	6.17ns	0.319	2.02 KB
#6547	`EnrichedLog`	net6.0	3.16μs	1.36ns	5.25ns	0.0301	2.2 KB
#6547	`EnrichedLog`	netcoreapp3.1	4.17μs	1.97ns	7.63ns	0.0294	2.2 KB
#6547	`EnrichedLog`	net472	5.01μs	1.93ns	7.48ns	0.318	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.3μs	0.604ns	2.34ns	0.0162	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.78μs	1.7ns	6.38ns	0.0154	1.14 KB
master	`SendReceive`	net472	2.11μs	1.65ns	6.4ns	0.183	1.16 KB
#6547	`SendReceive`	net6.0	1.39μs	1.14ns	4.42ns	0.0161	1.14 KB
#6547	`SendReceive`	netcoreapp3.1	1.87μs	1.14ns	4.43ns	0.0159	1.14 KB
#6547	`SendReceive`	net472	2.21μs	2.35ns	9.12ns	0.183	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.82μs	1.48ns	5.72ns	0.0225	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.91μs	2.99ns	11.6ns	0.0214	1.65 KB
master	`EnrichedLog`	net472	4.35μs	2.51ns	9.73ns	0.322	2.04 KB
#6547	`EnrichedLog`	net6.0	2.71μs	1.33ns	5.14ns	0.0218	1.6 KB
#6547	`EnrichedLog`	netcoreapp3.1	3.78μs	1.44ns	5.57ns	0.0208	1.65 KB
#6547	`EnrichedLog`	net472	4.22μs	3.24ns	12.6ns	0.323	2.04 KB

Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6547

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0	1.249	397.43	496.47

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	398ns	0.155ns	0.602ns	0.00819	576 B
master	`StartFinishSpan`	netcoreapp3.1	615ns	0.451ns	1.75ns	0.00767	576 B
master	`StartFinishSpan`	net472	700ns	0.374ns	1.4ns	0.0917	578 B
master	`StartFinishScope`	net6.0	542ns	0.325ns	1.22ns	0.00975	696 B
master	`StartFinishScope`	netcoreapp3.1	718ns	0.412ns	1.49ns	0.00946	696 B
master	`StartFinishScope`	net472	884ns	0.661ns	2.56ns	0.104	658 B
#6547	`StartFinishSpan`	net6.0	496ns	0.299ns	1.16ns	0.00799	576 B
#6547	`StartFinishSpan`	netcoreapp3.1	603ns	0.591ns	2.29ns	0.0078	576 B
#6547	`StartFinishSpan`	net472	665ns	0.528ns	2.05ns	0.0916	578 B
#6547	`StartFinishScope`	net6.0	489ns	0.84ns	3.14ns	0.0097	696 B
#6547	`StartFinishScope`	netcoreapp3.1	770ns	0.443ns	1.66ns	0.00919	696 B
#6547	`StartFinishScope`	net472	872ns	0.46ns	1.72ns	0.105	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	667ns	0.353ns	1.32ns	0.00969	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	876ns	0.706ns	2.74ns	0.00922	696 B
master	`RunOnMethodBegin`	net472	1.07μs	0.659ns	2.55ns	0.104	658 B
#6547	`RunOnMethodBegin`	net6.0	648ns	0.4ns	1.55ns	0.00975	696 B
#6547	`RunOnMethodBegin`	netcoreapp3.1	927ns	0.648ns	2.43ns	0.00917	696 B
#6547	`RunOnMethodBegin`	net472	1.14μs	0.651ns	2.52ns	0.104	658 B

Update doc and fix compilation errors

9c0d298

chrisnas requested a review from a team as a code owner January 15, 2025 10:46

chrisnas changed the title ~~[Profiler] Cleanup~~ [Profiler] Cleanup repository Jan 15, 2025

github-actions bot added the area:profiler Issues related to the continous-profiler label Jan 15, 2025

gleocadie approved these changes Jan 17, 2025

View reviewed changes

chrisnas merged commit f25f11f into master Jan 17, 2025
70 of 74 checks passed

chrisnas deleted the chrisnas/update_profiler_doc branch January 17, 2025 14:34

github-actions bot added this to the vNext-v3 milestone Jan 17, 2025

andrewlock added the type:cleanup Minor code clean up label Jan 22, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Profiler] Cleanup repository #6547

[Profiler] Cleanup repository #6547

chrisnas commented Jan 15, 2025

datadog-ddstaging bot commented Jan 15, 2025 •

edited

Loading

andrewlock commented Jan 15, 2025

andrewlock commented Jan 16, 2025

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6547

Raw results

Raw results

[Profiler] Cleanup repository #6547

[Profiler] Cleanup repository #6547

Conversation

chrisnas commented Jan 15, 2025

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

datadog-ddstaging bot commented Jan 15, 2025 • edited Loading

Datadog Report

❌ Failed Tests (1)

andrewlock commented Jan 15, 2025

Execution-Time Benchmarks Report ⏱️

andrewlock commented Jan 16, 2025

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6547

Raw results

Raw results

datadog-ddstaging bot commented Jan 15, 2025 •

edited

Loading