Improve performance of lsif generation.

[CyrusNajmabadi]

We would often write out tens of thousands of elements per file, taking locks for each eleemnt we're writing. We can just take the lock once and write out the full batch.

Before time:

Total time spent in the generation phase for all projects, including compilation fetch time: 25.02 seconds

After time:

Total time spent in the generation phase for all projects, including compilation fetch time: 18.67 seconds

This is an improvement of 25%.

[sharwell]

Is there an easy way I can run the generator locally?

[CyrusNajmabadi]

I'm running:

C:\github\roslyn\artifacts\bin\Microsoft.CodeAnalysis.LanguageServerIndexFormat.Generator\Release\net472\Microsoft.CodeAnalysis.LanguageServerIndexFormat.Generator.exe --project C:\github\roslyn\src\Compilers\Core\Portable\Microsoft.CodeAnalysis.csproj --log c:\temp\log.txt --output c:\temp\lsif.json

Note: you'll need #51458 where i add support for just running against a single project. i can also collect full solution time.

[CyrusNajmabadi]

Performance at the full solution goes from:

Total time spent in the generation phase for all projects, including compilation fetch time: 523.66 seconds

to

Total time spent in the generation phase for all projects, including compilation fetch time: 464.94 seconds

Only 12% gain here, but still nice to have.

[sharwell]

❔ Do we still need this one?

[CyrusNajmabadi]

yup. several callers call directly into that one. I could have them wrap into a single-element list, but it seemed wasteful :)

[sharwell]

Performance at the full solution goes from:

You'll get at least 60 seconds gain my moving from MSBuildWorkspace to direct creation of compilations from csc invocations in the binlog. MSBuildWorkspace is extremely terrible for end-to-end performance because there is no parallelism in the MSBuild evaluation.

csc invocations have a secondary benefit of improved accuracy.

[CyrusNajmabadi]

to direct creation of compilations from csc invocations

Definitely agree. I'm just very limited on time to here :)

[ghost]

Auto-approval