Optionally record activities in the service

[safesparrow]

Follow up on @baronfel 's initial work.

This PR adds activity recording to the FCS service.

It adds a dependency on System.Diagnostics.DiagnosticSource but nothing else.

Recording and exporting of activities can be enabled in user code (eg. in the IDE or external tests) - example.

Sample traces

Sample traces produced can be found in https://github.com/safesparrow/fsharp-benchmark-generator/tree/main/trace_samples
They can be opened with Jaeger UI.

Jaeger UI screenshots:

Fantomas sample (codebase)

stress_huge_dense sample (codebase)

Sample traces for perf analysis of #13521

The driver for this PR is the need to easily verify the behaviour of the compiler/service, especially w.r.t. multi-threaded work.

With that in mind, I produced the following traces showcasing the behaviour of parallel project analysis:

50_leaves sample, parallel analysis on, server GC

50_leaves sample, parallel analysis on, workstation GC

50_leaves sample, parallel analysis off, server GC

50_leaves sample, parallel analysis off, workstation GC

Generating traces

Sample traces can be generated with the following tools:

• Jaeger as tracing backend and UI, and
• https://github.com/safesparrow/fsharp-benchmark-generator/ as a way to run benchmarks that produce traces
  using the following script:

# Run Jaeger docker image - command from https://www.jaegertracing.io/docs/1.37/getting-started/
docker run -d --name jaeger -e COLLECTOR_ZIPKIN_HOST_PORT=:9411 -e COLLECTOR_OTLP_ENABLED=true   -p 6831:6831/udp   -p 6832:6832/udp   -p 5778:5778   -p 16686:16686   -p 4317:4317   -p 4318:4318   -p 14250:14250   -p 14268:14268   -p 14269:14269   -p 9411:9411   jaegertracing/all-in-one:1.37

# Install 'fcs-benchmark' global dotnet tool - https://github.com/safesparrow/fsharp-benchmark-generator
dotnet tool install fcsbenchmark --global --prerelease

# Run a benchmark of a sample solution, using local F# codebase, and 4 different GC&threading configurations
fcs-benchmark --sample 50_leaves --local=C:/projekty/fsharp/fsharp-otel -c C:/.artifacts --gc=Compare --parallel-analysis=Compare --record-otel-jaeger -w 0 -n 1

Notes

Names of the activities and tags are likely wrong - please do suggest better names🙂

To do:

• Update surface baseline
• Fix formatting
• Attach traces for parallel vs sequential project analysis (Parallel project analysis behind a feature flag #13521 )
• Verify activity and tag names
• More

[baronfel]

Nice - I think it might demonstrate the value better if you could attach trace views to show the results.

[safesparrow]

Have attached Jaeger screenshots now and removed any dependencies except for System.Diagnostics.DiagnosticSource.

[vzarytovskii]

We can merge after @KevinRansom looks at it

[KevinRansom]

Thanks for this, it looks good. However, we will need to do some due diligence internally to ensure that this satisfies our telemetry and privacy policies. We will add this to our November milestone to see if we can get it approved.

Planning issue here: #14210

[safesparrow]

Thanks for this, it looks good. However, we will need to do some due diligence internally to ensure that this satisfies our telemetry and privacy policies. We will add this to our November milestone to see if we can get it approved.

Planning issue here: #14210

Thanks for looking.

OOI what makes this more sensitive than any other existing logging? Also it doesn't depend on open telemetry and is off by default. Is there some increased risk of user data leakage or something like that?

[safesparrow]

@vzarytovskii If this gets merged, how difficult will it be to add new activities for other areas of the compiler in future PRs? I'm looking into optimization and having some detailed tracing there would help (eg. a span for each file being optimized, similar to how parsing & type-checking work atm).

Will it require new approvals from the policy PoV?

[vzarytovskii]

@vzarytovskii Vlad Zarytovskii FTE If this gets merged, how difficult will it be to add new activities for other areas of the compiler in future PRs? I'm looking into optimization and having some detailed tracing there would help (eg. a span for each file being optimized, similar to how parsing & type-checking work atm).

Will it require new approvals from the policy PoV?

Yep, it got approved in principle. Since compiler doesn't work with PII or EUII per se, we can just add new metrics, as long as we don't opt-in them by default (and we don't for neither old or new telemetry).

Apprved

[vzarytovskii]

@safesparrow It's ready to merge, let me know if you want any more changes here, or if we can merge it now.

[T-Gro]

That is great news, let's merge it in so that new ones can build on this module.

[safesparrow]

I'm happy to have it merged, and we can iterate easily in future. Thanks for help all 👍

[vzarytovskii]

This needs to be reverted: #14413

[vzarytovskii]

Revert has to also go directly to 17.5

[vzarytovskii]

Or, as Kevin pointed out, that might be solved by adding package to vsix.