Configure FSI to use gcServer

[cartermp]

Inspired by #5381

In some situations gcServer may result in much more throughput for large sets of data being executed in FSI. This should be configurable in settings independently of an IDE, but should also have an option in VS.

[nhirschey]

For context adding illustration of GC consequences when transforming a 30 gigabyte file on disk into an array of records, showing the ~linear speedup on an 18 core cpu.

• ServerGC Run Add contributing guidelines to repo root #1 Real: 00:08:40.133, CPU: 02:45:06.468, GC gen0: 2700, gen1: 885, gen2: 8
• ServerGC Run Label F# PCL projects such that it's obvious which ones support Xamarin #2 Real: 00:08:36.486, CPU: 02:47:22.718, GC gen0: 2908, gen1: 965, gen2: 8
• DefaultGC Run Add contributing guidelines to repo root #1 Real: 03:32:50.024, CPU: 10:49:39.015, GC gen0: 723832, gen1: 134812, gen2: 17
• DefaultGC Run Label F# PCL projects such that it's obvious which ones support Xamarin #2 Real: 02:40:32.485, CPU: 09:22:08.765, GC gen0: 723344, gen1: 135214, gen2: 18

   let getRecords samplePeriod = 
        sasFile.Observations // SAS file loaded via type provider
        |> Seq.toArray
        |> Array.Parallel.choose(fun obs ->
            match inSamplePeriod samplePeriod obs with
            | true -> Some (parseSas obs) // converts SAS file row to record type
            | false -> None)
 
    GC.Collect()
    // Timings based on running below line
    let x = getRecords samplePeriod

[dsyme]

I'm going over open FSI issues.

So the conclusion here was that we decided not to take the change in #5381, and we are using this issue to track the work that is necessary?

Also, is the situation here the same with .NET Core?

[cartermp]

Yeah this one's for @KevinRansom to think a little more about

[KevinRansom]

I'm not sure what the situation is with gcServer and DotNETCore it's going to require some research.

[nhirschey]

Thanks for checking in on this. It is still an issue in .NET Core and it has an enormous impact on PSeq and Array.Parallel data processing performance.

1. Currently, .NET Core fsi is still only distributed with workstation GC. To allow gcServer on .NET Core fsi the fsi.runtimeconfig.json (windows) or dotnet.runtimeconfig.json (mac/Linux) file needs to be edited. More details here.

2. On a related multi-core fsi note, I noticed that at least on windows the .NET Core fsi is restricted to 1 CPU group, which limits fsi on dual processor and > 64 virtual core CPUs. I have successfully modified fsiAnyCpu.exe.config to use all CPUs inside Visual Studio (enabling GCCpuGroup), but I didn't find the right runtime options to get dotnet fsi working across all CPU groups; not sure if it's possible. I think this is the proper reference.

[nhirschey]

Here's a reproducible example using dotnet fsi from ionide, Microsoft (R) F# Interactive version 11.0.0.0 for F# 5.0.

#time 

let x = 
    [| 1.0 .. 10_000_000.0 |]
    |> Array.groupBy(fun x -> x % 100.0)

System.GC.Collect() 

// Timings are based on running this code below only

for i = 1 to 10 do
    x 
    |> Array.Parallel.map(fun (_, xs) -> 
        xs 
        |> Array.map string
        |> Array.map float
        |> Array.sum) 

// .NET Core GcServer: Real: 00:00:00.998, CPU: 00:00:49.921, GC gen0: 1, gen1: 1, gen2: 1
// .NET Core GcWorkstation: Real: 00:00:20.170, CPU: 00:02:42.296, GC gen0: 773, gen1: 273, gen2: 7

[vzarytovskii]

I think we do that now:

cat /usr/local/share/dotnet/sdk/8.0.100/FSharp/*.runtimeconfig.json | grep "GC.Server"
      "System.GC.Server": true,
      "System.GC.Server": true,