Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optionally record activities in the service #13835

Merged
merged 58 commits into from
Nov 21, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
0dd5258
first activity plumbing
baronfel Jul 17, 2022
237bf59
very basic spans
baronfel Jul 17, 2022
524cd17
better names
baronfel Jul 17, 2022
c62e89c
Allow parallel project analysis with an environment variable
safesparrow Jul 17, 2022
4431af5
reformat CompilerImports.fs
safesparrow Jul 17, 2022
4618764
more granularity
baronfel Jul 24, 2022
5658f02
Merge remote-tracking branch 'baronfel/activities' into otel
safesparrow Jul 27, 2022
906ed7a
Merge branch 'main' into otel
safesparrow Aug 29, 2022
01e165b
WIP
safesparrow Aug 30, 2022
d4db68a
Merge branch 'main' into otel
safesparrow Aug 30, 2022
30fd578
Don't dispose of the telemetry objects immediately
safesparrow Aug 30, 2022
16fcfd4
Hacks to make telemetry work in incremental builder
safesparrow Aug 31, 2022
0702fc1
More hacks to get nested project information in the trace
safesparrow Aug 31, 2022
c3d84f1
Merge remote-tracking branch 'origin/parallel-project-analysis' into …
safesparrow Aug 31, 2022
f72b570
more hacks
safesparrow Sep 2, 2022
f8170b8
Improve activity logging for the background checker
safesparrow Sep 2, 2022
a1f74c9
* Make the code do nothing if no listeners are hooked up.
safesparrow Sep 3, 2022
ad5d138
Use Jaeger exporter in FSC
safesparrow Sep 3, 2022
57ca46b
Revert "Allow parallel project analysis with an environment variable"
safesparrow Sep 3, 2022
057a110
* Remove references to OpenTelemetry
safesparrow Sep 3, 2022
ca51e57
Fix build
safesparrow Sep 3, 2022
e5a3816
Revert unneeded changes/PR feedback
safesparrow Sep 3, 2022
679d2aa
Remove nuget.org
safesparrow Sep 3, 2022
c86c851
Merge remote-tracking branch 'upstream/main' into otel
safesparrow Sep 3, 2022
bdb546b
Fantomas formatting
safesparrow Sep 4, 2022
3133d0c
Update baseline
safesparrow Sep 4, 2022
eabca75
Fantomas formatting
safesparrow Sep 5, 2022
83dcc97
Make the Activity module internal, remove from the surface
safesparrow Sep 5, 2022
43394e6
Merge remote-tracking branch 'upstream/main' into otel
safesparrow Sep 12, 2022
6783b97
Merge branch 'main' into otel
safesparrow Sep 13, 2022
ea8a7ed
Merge upstream/main
safesparrow Sep 17, 2022
80459f9
Fix System.Diagnostics.DiagnosticSource package version
safesparrow Sep 17, 2022
a776c86
Tracing via dotnet ActivitySource
T-Gro Oct 10, 2022
924097b
Resolving conflicts
T-Gro Oct 10, 2022
6ce22da
Fantomas applied
T-Gro Oct 10, 2022
50fd7b5
Merge remote-tracking branch 'origin/main' into feature/activity-tracing
T-Gro Oct 10, 2022
4e23ca3
Merge pull request #3 from T-Gro/feature/activity-tracing
safesparrow Oct 10, 2022
bca4a45
Use consistent casing in tag names
safesparrow Oct 10, 2022
b71bf73
Remove old code
safesparrow Oct 10, 2022
1122aa6
Merge branch 'main' into otel
safesparrow Oct 10, 2022
b857487
Rename activity file, fix NRE, rename start methods
safesparrow Oct 10, 2022
6684413
Fix build
safesparrow Oct 10, 2022
549b3f8
Don't use dummyDisposable
safesparrow Oct 10, 2022
e1be127
Fantomas formatting
safesparrow Oct 10, 2022
dcfea04
Apply fixes from T-Gro to avoid NRE
safesparrow Oct 11, 2022
8c053c3
Fantomas formatting one more time
safesparrow Oct 11, 2022
2b9210f
Fix CI by nuspec dependency version assignment
T-Gro Oct 11, 2022
6fc7aeb
Merge branch 'otel' of https://github.com/safesparrow/fsharp into pr/…
T-Gro Oct 11, 2022
d502913
Merge branch 'main' into feature/activity-tracking-CI-fix
T-Gro Oct 11, 2022
0c95dc9
Updating DiagnosticSource version and reflecting in vsintegraion proj…
T-Gro Oct 11, 2022
fb61e37
Reflecting newer package version in the code (more overloads available)
T-Gro Oct 11, 2022
ab1b693
Applying Fantomas
T-Gro Oct 11, 2022
03f6760
Merge branch 'main' into otel
safesparrow Oct 12, 2022
ad91f8b
Merge remote-tracking branch 'origin/main' into otel
safesparrow Oct 24, 2022
26ab7f5
Merge remote-tracking branch 'upstream/main' into otel
safesparrow Oct 24, 2022
336b120
Add a comment and move Activity.fs/fsi to Utilities/
safesparrow Oct 24, 2022
8767a5a
Merge branch 'main' into otel
safesparrow Oct 26, 2022
488f8b4
Merge branch 'main' into otel
safesparrow Nov 20, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions eng/Versions.props
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@
<SystemBuffersVersion>4.5.1</SystemBuffersVersion>
<SystemCollectionsImmutableVersion>6.0.0</SystemCollectionsImmutableVersion>
<MicrosoftDiaSymReaderPortablePdbVersion>1.6.0</MicrosoftDiaSymReaderPortablePdbVersion>
<SystemDiagnosticsDiagnosticSourceVersion>6.0.0</SystemDiagnosticsDiagnosticSourceVersion>
<SystemMemoryVersion>4.5.5</SystemMemoryVersion>
<SystemReflectionEmitVersion>4.7.0</SystemReflectionEmitVersion>
<SystemReflectionMetadataVersion>6.0.0</SystemReflectionMetadataVersion>
Expand Down
17 changes: 15 additions & 2 deletions src/Compiler/Checking/CheckDeclarations.fs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ module internal FSharp.Compiler.CheckDeclarations
open System
open System.Collections.Generic

open FSharp.Compiler.Diagnostics
open Internal.Utilities.Collections
open Internal.Utilities.Library
open Internal.Utilities.Library.Extras
Expand Down Expand Up @@ -5289,11 +5290,17 @@ let CheckOneImplFile
env,
rootSigOpt: ModuleOrNamespaceType option,
synImplFile) =

let (ParsedImplFileInput (_, isScript, qualNameOfFile, scopedPragmas, _, implFileFrags, isLastCompiland, _)) = synImplFile
let (ParsedImplFileInput (fileName, isScript, qualNameOfFile, scopedPragmas, _, implFileFrags, isLastCompiland, _)) = synImplFile
let infoReader = InfoReader(g, amap)

cancellable {
use _ =
Activity.start "CheckDeclarations.CheckOneImplFile"
[|
"fileName", fileName
"qualifiedNameOfFile", qualNameOfFile.Text
|]
let cenv =
cenv.Create (g, isScript, amap, thisCcu, false, Option.isSome rootSigOpt,
conditionalDefines, tcSink, (LightweightTcValForUsingInBuildMethodCall g), isInternalTestSpanStackReferring,
Expand Down Expand Up @@ -5421,6 +5428,12 @@ let CheckOneImplFile
/// Check an entire signature file
let CheckOneSigFile (g, amap, thisCcu, checkForErrors, conditionalDefines, tcSink, isInternalTestSpanStackReferring) tcEnv (sigFile: ParsedSigFileInput) =
cancellable {
use _ =
Activity.start "CheckDeclarations.CheckOneSigFile"
[|
"fileName", sigFile.FileName
"qualifiedNameOfFile", sigFile.QualifiedName.Text
|]
let cenv =
cenv.Create
(g, false, amap, thisCcu, true, false, conditionalDefines, tcSink,
Expand Down
11 changes: 6 additions & 5 deletions src/Compiler/Driver/CompilerOptions.fs
Original file line number Diff line number Diff line change
Expand Up @@ -2331,13 +2331,14 @@ let PrintWholeAssemblyImplementation (tcConfig: TcConfig) outfile header expr =
//----------------------------------------------------------------------------

let mutable tPrev: (DateTime * DateTime * float * int[]) option = None
let mutable nPrev: string option = None
let mutable nPrev: (string * IDisposable) option = None

let ReportTime (tcConfig: TcConfig) descr =

match nPrev with
| None -> ()
| Some prevDescr ->
| Some (prevDescr, prevActivity) ->
use _ = prevActivity // Finish the previous diagnostics activity by .Dispose() at the end of this block

if tcConfig.pause then
dprintf "[done '%s', entering '%s'] press <enter> to continue... " prevDescr descr
Console.ReadLine() |> ignore
Expand Down Expand Up @@ -2376,7 +2377,7 @@ let ReportTime (tcConfig: TcConfig) descr =

let tStart =
match tPrev, nPrev with
| Some (tStart, tPrev, utPrev, gcPrev), Some prevDescr ->
| Some (tStart, tPrev, utPrev, gcPrev), Some (prevDescr, _) ->
let spanGC = [| for i in 0..maxGen -> GC.CollectionCount i - gcPrev[i] |]
let t = tNow - tStart
let tDelta = tNow - tPrev
Expand All @@ -2403,7 +2404,7 @@ let ReportTime (tcConfig: TcConfig) descr =

tPrev <- Some(tStart, tNow, utNow, gcNow)

nPrev <- Some descr
nPrev <- Some(descr, Activity.startNoTags descr)

let ignoreFailureOnMono1_1_16 f =
try
Expand Down
6 changes: 4 additions & 2 deletions src/Compiler/Driver/ParseAndCheckInputs.fs
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
module internal FSharp.Compiler.ParseAndCheckInputs

open System
open System.Diagnostics
open System.IO
open System.Collections.Generic

Expand Down Expand Up @@ -1175,6 +1176,9 @@ let CheckOneInputAux

cancellable {
try
use _ =
Activity.start "ParseAndCheckInputs.CheckOneInput" [| "fileName", inp.FileName |]

CheckSimulateException tcConfig

let m = inp.Range
Expand Down Expand Up @@ -1365,10 +1369,8 @@ let CheckMultipleInputsFinish (results, tcState: TcState) =

let CheckOneInputAndFinish (checkForErrors, tcConfig: TcConfig, tcImports, tcGlobals, prefixPathOpt, tcSink, tcState, input) =
cancellable {
Logger.LogBlockStart LogCompilerFunctionId.CompileOps_TypeCheckOneInputAndFinishEventually
let! result, tcState = CheckOneInput(checkForErrors, tcConfig, tcImports, tcGlobals, prefixPathOpt, tcSink, tcState, input, false)
let finishedResult = CheckMultipleInputsFinish([ result ], tcState)
Logger.LogBlockStop LogCompilerFunctionId.CompileOps_TypeCheckOneInputAndFinishEventually
return finishedResult
}

Expand Down
5 changes: 0 additions & 5 deletions src/Compiler/Driver/fsc.fs
Original file line number Diff line number Diff line change
Expand Up @@ -532,7 +532,6 @@ let main1

// Process command line, flags and collect filenames
let sourceFiles =

// The ParseCompilerOptions function calls imperative function to process "real" args
// Rather than start processing, just collect names, then process them.
try
Expand Down Expand Up @@ -710,7 +709,6 @@ let main2
exiter: Exiter,
ilSourceDocs))
=

if tcConfig.typeCheckOnly then
exiter.Exit 0

Expand Down Expand Up @@ -818,7 +816,6 @@ let main3
exiter: Exiter,
ilSourceDocs))
=

// Encode the signature data
ReportTime tcConfig "Encode Interface Data"
let exportRemapping = MakeExportRemapping generatedCcu generatedCcu.Contents
Expand Down Expand Up @@ -914,7 +911,6 @@ let main4
exiter: Exiter,
ilSourceDocs))
=

match tcImportsCapture with
| None -> ()
| Some f -> f tcImports
Expand Down Expand Up @@ -1049,7 +1045,6 @@ let main6
exiter: Exiter,
ilSourceDocs))
=

ReportTime tcConfig "Write .NET Binary"

use _ = UseBuildPhase BuildPhase.Output
Expand Down
8 changes: 5 additions & 3 deletions src/Compiler/FSharp.Compiler.Service.fsproj
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@

<!-- The FSharp.Compiler.Service dll provides a referencable public interface for tool builders -->
<PropertyGroup Condition="'$(Configuration)' != 'Proto'">
<CompressMetadata Condition="'$(CompressAllMetadata)' != 'true'">false</CompressMetadata>
<CompressMetadata Condition="'$(CompressAllMetadata)' != 'true'">false</CompressMetadata>
</PropertyGroup>

<PropertyGroup>
Expand Down Expand Up @@ -57,6 +57,7 @@
<NuspecProperty Include="SystemBuffersPackageVersion=$(SystemBuffersVersion)" />
<NuspecProperty Include="SystemCollectionsImmutablePackageVersion=$(SystemCollectionsImmutableVersion)" />
<NuspecProperty Include="SystemMemoryPackageVersion=$(SystemMemoryVersion)" />
<NuspecProperty Include="SystemDiagnosticsDiagnosticSourcePackageVersion=$(SystemDiagnosticsDiagnosticSourceVersion)" />
<NuspecProperty Include="SystemReflectionEmitPackageVersion=$(SystemReflectionEmitVersion)" />
<NuspecProperty Include="SystemReflectionMetadataPackageVersion=$(SystemReflectionMetadataVersion)" />
<NuspecProperty Include="SystemRuntimeCompilerServicesUnsafePackageVersion=$(SystemRuntimeCompilerServicesUnsafeVersion)" />
Expand Down Expand Up @@ -91,6 +92,8 @@
<Link>FSStrings.resx</Link>
<LogicalName>FSStrings.resources</LogicalName>
</EmbeddedResource>
<Compile Include="Utilities\Activity.fsi" />
<Compile Include="Utilities\Activity.fs" />
<Compile Include="Utilities\sformat.fsi" />
<Compile Include="Utilities\sformat.fs" />
<Compile Include="Utilities\sr.fsi" />
Expand Down Expand Up @@ -131,8 +134,6 @@
<Compile Include="Utilities\range.fsi" />
<Compile Include="Utilities\range.fs" />
<EmbeddedText Include="Facilities\UtilsStrings.txt" />
<Compile Include="Facilities\Logger.fsi" />
<Compile Include="Facilities\Logger.fs" />
<Compile Include="Facilities\LanguageFeatures.fsi" />
<Compile Include="Facilities\LanguageFeatures.fs" />
<Compile Include="Facilities\DiagnosticOptions.fsi" />
Expand Down Expand Up @@ -488,6 +489,7 @@
<PackageReference Include="System.Reflection.Emit" Version="$(SystemReflectionEmitVersion)" />
<PackageReference Include="System.Reflection.Metadata" Version="$(SystemReflectionMetadataVersion)" />
<PackageReference Include="System.Buffers" Version="$(SystemBuffersVersion)" />
<PackageReference Include="System.Diagnostics.DiagnosticSource" Version="$(SystemDiagnosticsDiagnosticSourceVersion)" />
<PackageReference Include="System.Memory" Version="$(SystemMemoryVersion)" />
<PackageReference Include="System.Runtime.CompilerServices.Unsafe" Version="$(SystemRuntimeCompilerServicesUnsafeVersion)" />
</ItemGroup>
Expand Down
1 change: 1 addition & 0 deletions src/Compiler/FSharp.Compiler.Service.nuspec
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
<dependency id="FSharp.Core" version="$FSharpCorePackageVersion$" exclude="Build,Analyzers" />
<dependency id="System.Buffers" version="$SystemBuffersPackageVersion$" exclude="Build,Analyzers" />
<dependency id="System.Collections.Immutable" version="$SystemCollectionsImmutablePackageVersion$" exclude="Build,Analyzers" />
<dependency id="System.Diagnostics.DiagnosticSource" version="$SystemDiagnosticsDiagnosticSourcePackageVersion$" exclude="Build,Analyzers" />
<dependency id="System.Memory" version="$SystemMemoryPackageVersion$" exclude="Build,Analyzers" />
<dependency id="System.Reflection.Emit" version="$SystemReflectionEmitPackageVersion$" exclude="Build,Analyzers" />
<dependency id="System.Reflection.Metadata" version="$SystemReflectionMetadataPackageVersion$" exclude="Build,Analyzers" />
Expand Down
10 changes: 10 additions & 0 deletions src/Compiler/Facilities/BuildGraph.fs
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,16 @@ type NodeCodeBuilder() =
(value :> IDisposable).Dispose()
}
)

[<DebuggerHidden; DebuggerStepThrough>]
member _.Using(value: IDisposable, binder: IDisposable -> NodeCode<'U>) =
Node(
async {
use _ = value
return! binder value |> Async.AwaitNodeCode
}
)


let node = NodeCodeBuilder()

Expand Down
7 changes: 5 additions & 2 deletions src/Compiler/Facilities/BuildGraph.fsi
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
module internal FSharp.Compiler.BuildGraph

open System
open System.Diagnostics
open System.Threading
open System.Threading.Tasks
open FSharp.Compiler.DiagnosticsLogger
Expand Down Expand Up @@ -43,10 +44,12 @@ type NodeCodeBuilder =

member Combine: x1: NodeCode<unit> * x2: NodeCode<'T> -> NodeCode<'T>

/// A limited form 'use' for establishing the compilation globals. (Note
/// that a proper generic 'use' could be implemented but has not currently been necessary)
/// A limited form 'use' for establishing the compilation globals.
member Using: CompilationGlobalsScope * (CompilationGlobalsScope -> NodeCode<'T>) -> NodeCode<'T>

/// A generic 'use' that disposes of the IDisposable at the end of the computation.
member Using: IDisposable * (IDisposable -> NodeCode<'T>) -> NodeCode<'T>

/// Specifies code that can be run as part of the build graph.
val node: NodeCodeBuilder

Expand Down
87 changes: 0 additions & 87 deletions src/Compiler/Facilities/Logger.fs

This file was deleted.

33 changes: 0 additions & 33 deletions src/Compiler/Facilities/Logger.fsi

This file was deleted.

4 changes: 3 additions & 1 deletion src/Compiler/Service/FSharpCheckerResults.fs
Original file line number Diff line number Diff line change
Expand Up @@ -2347,6 +2347,7 @@ module internal ParseAndCheckFile =

let parseFile (sourceText: ISourceText, fileName, options: FSharpParsingOptions, userOpName: string, suggestNamesForErrors: bool) =
Trace.TraceInformation("FCS: {0}.{1} ({2})", userOpName, "parseFile", fileName)
use act = Activity.start "ParseAndCheckFile.parseFile" [| "fileName", fileName |]

let errHandler =
DiagnosticsHandler(true, fileName, options.DiagnosticOptions, sourceText, suggestNamesForErrors)
Expand Down Expand Up @@ -2502,7 +2503,8 @@ module internal ParseAndCheckFile =
) =

cancellable {
use _logBlock = Logger.LogBlock LogCompilerFunctionId.Service_CheckOneFile
use _ =
Activity.start "ParseAndCheckFile.CheckOneFile" [| "fileName", mainInputFileName; "length", sourceText.Length.ToString() |]

let parsedMainInput = parseResults.ParseTree

Expand Down
Loading