Skip to content

OpenTelemetry instrumentation for xUnit tests #18230

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
psfinaki merged 4 commits into from
Jan 13, 2025
Merged

OpenTelemetry instrumentation for xUnit tests #18230

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
fa14c59
otel instrumentation for xunit test runs
majocha Jan 12, 2025
3caf0bf
wip
majocha Jan 12, 2025
ce82a51
sort out deps version conflict
majocha Jan 12, 2025
9e4add0
disable for desktop
majocha Jan 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 0 additions & 11 deletions tests/FSharp.Compiler.ComponentTests/FSharpChecker/CommonWorkflows.fs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,14 +143,6 @@ let ``Using getSource and notifications instead of filesystem`` () =

[<Fact>]
let GetAllUsesOfAllSymbols() =
let traceProvider =
Sdk.CreateTracerProviderBuilder()
.AddSource("fsc")
.SetResourceBuilder(ResourceBuilder.CreateDefault().AddService(serviceName="F#", serviceVersion = "1"))
.AddJaegerExporter()
.Build()

use _ = Activity.start "GetAllUsesOfAllSymbols" [ ]

let result =
async {
Expand All @@ -162,9 +154,6 @@ let GetAllUsesOfAllSymbols() =
return checkProjectResults.GetAllUsesOfAllSymbols()
} |> Async.RunSynchronously

traceProvider.ForceFlush() |> ignore
traceProvider.Dispose()

if result.Length <> 79 then failwith $"Expected 81 symbolUses, got {result.Length}:\n%A{result}"

[<Fact>]
Expand Down
7 changes: 0 additions & 7 deletions tests/FSharp.Compiler.ComponentTests/FSharpChecker/TransparentCompiler.fs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -556,13 +556,6 @@ let fuzzingTest seed (project: SyntheticProject) = task {
do! Task.Delay (rng.Next maxCheckingDelayMs)
}

use _tracerProvider =
Sdk.CreateTracerProviderBuilder()
.AddSource("fsc")
.SetResourceBuilder(ResourceBuilder.CreateDefault().AddService(serviceName="F# Fuzzing", serviceVersion = "1"))
.AddJaegerExporter()
.Build()

use _ = Activity.start $"Fuzzing {project.Name}" [ Activity.Tags.project, project.Name; "seed", seed.ToString() ]

do! task {
Expand Down
2 changes: 1 addition & 1 deletion tests/FSharp.Test.Utilities/FSharp.Test.Utilities.fsproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@
<PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="$(MicrosoftCodeAnalysisCSharpVersion)" />
<PackageReference Include="Microsoft.CodeAnalysis.Test.Resources.Proprietary" Version="$(MicrosoftCodeAnalysisTestResourcesProprietaryVersion)" />
<PackageReference Include="Microsoft.NETCore.App.Ref" Version="6.0.0" IncludeAssets="none" PrivateAssets="all" GeneratePathProperty="true" />
<PackageReference Include="OpenTelemetry.Exporter.Jaeger" Version="1.4.0" />
<PackageReference Include="OpenTelemetry.Exporter.OpenTelemetryProtocol" Version="1.9.0" />
<PackageReference Include="Newtonsoft.Json" Version="$(NewtonsoftJsonVersion)" />
</ItemGroup>

Expand Down
11 changes: 0 additions & 11 deletions tests/FSharp.Test.Utilities/ProjectGeneration.fs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -961,7 +961,6 @@ type ProjectWorkflowBuilder

let mutable latestProject = initialProject
let mutable activity = None
let mutable tracerProvider = None

let getSource f = f |> getSourceText latestProject :> ISourceText |> Some |> async.Return

Expand Down Expand Up @@ -1010,13 +1009,6 @@ type ProjectWorkflowBuilder

member this.Yield _ = async {
let! ctx = getInitialContext()
tracerProvider <-
Sdk.CreateTracerProviderBuilder()
.AddSource("fsc")
.SetResourceBuilder(ResourceBuilder.CreateDefault().AddService(serviceName="F#", serviceVersion = "1"))
.AddJaegerExporter()
.Build()
|> Some
activity <- Activity.start ctx.Project.Name [ Activity.Tags.project, ctx.Project.Name; "UsingTransparentCompiler", useTransparentCompiler.ToString() ] |> Some
return ctx
}
Expand All @@ -1032,9 +1024,6 @@ type ProjectWorkflowBuilder
if initialContext.IsNone && not isExistingProject then
this.DeleteProjectDir()
activity |> Option.iter (fun x -> if not (isNull x) then x.Dispose())
tracerProvider |> Option.iter (fun x ->
x.ForceFlush() |> ignore
x.Dispose())

member this.Run(workflow: Async<WorkflowContext>) =
if autoStart then
Expand Down
22 changes: 22 additions & 0 deletions tests/FSharp.Test.Utilities/XunitHelpers.fs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,12 @@ open Xunit.Abstractions

open TestFramework

open FSharp.Compiler.Diagnostics

open OpenTelemetry
open OpenTelemetry.Resources
open OpenTelemetry.Trace

/// Disables custom internal parallelization added with XUNIT_EXTRAS.
/// Execute test cases in a class or a module one by one instead of all at once. Allow other collections to run simultaneously.
[<AttributeUsage(AttributeTargets.Class ||| AttributeTargets.Method, AllowMultiple = false)>]
Expand Down Expand Up @@ -45,6 +51,7 @@ type ConsoleCapturingTestRunner(test, messageBus, testClass, constructorArgument
override this.InvokeTestAsync (aggregator: ExceptionAggregator) =
task {
use capture = new TestConsole.ExecutionCapture()
use _ = Activity.start test.DisplayName [ ]
let! executionTime = this.BaseInvokeTestMethodAsync aggregator
let output =
seq {
Expand Down Expand Up @@ -139,9 +146,24 @@ type FSharpXunitFramework(sink: IMessageSink) =
log "FSharpXunitFramework with XUNIT_EXTRAS installing TestConsole redirection"
TestConsole.install()

// TODO: Currently does not work with Desktop .NET Framework. Upcoming OpenTelemetry 1.11.0 may change it.
Copy link
Member

@baronfel baronfel Jan 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

out of curiosity, what makes you think 1.11 will enable .NET Framework support for OTel? I don't see anything in the release notes or changelog to suggest that.

Copy link
Member

@baronfel baronfel Jan 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it this new builder pattern?

Copy link
Contributor Author

@majocha majocha Jan 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They say they're dropping the quasi-deprecated gRpc.Core. When I tried it here, grpc just fails silently (exception can be observed in debug mode) during runtime on desktop target.

Copy link
Contributor Author

@majocha majocha Jan 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May be wishful thinking on my part, though!

Copy link
Contributor Author

@majocha majocha Jan 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just did a quick test with console app. Otel 1.9.0 does work in general with net472, just not here in this repo, for some reason.

#if NETCOREAPP
let traceProvider =
Sdk.CreateTracerProviderBuilder()
.AddSource(ActivityNames.FscSourceName)
.SetResourceBuilder(
ResourceBuilder.CreateDefault().AddService(serviceName="F#", serviceVersion = "1.0.0"))
.AddOtlpExporter()
.Build()
#endif

interface IDisposable with
member _.Dispose() =
cleanUpTemporaryDirectoryOfThisTestRun ()
#if NETCOREAPP
traceProvider.ForceFlush() |> ignore
traceProvider.Dispose()
#endif
base.Dispose()

override this.CreateDiscoverer (assemblyInfo) =
Expand Down
Loading