Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP, Investigation] Cache project options #2801

Conversation

vasily-kirichenko
Copy link
Contributor

It caches FSharpProjectOptions and it won't work if projects changed. The log shows the following:

05-Apr-17 17:10:52 - FSharp.LanguageService.fsproj - created!!!
05-Apr-17 17:10:52 - FSharp.LanguageService.fsproj - cached
05-Apr-17 17:10:52 - FSharp.LanguageService.fsproj - created!!!
05-Apr-17 17:10:52 - FSharp.LanguageService.fsproj - cached
05-Apr-17 17:10:52 - FSharp.LanguageService.fsproj - created!!!
05-Apr-17 17:10:52 - FSharp.LanguageService.fsproj - cached
05-Apr-17 17:10:52 - FSharp.LanguageService.fsproj - created!!!
05-Apr-17 17:10:52 - FSharp.LanguageService.fsproj - cached
05-Apr-17 17:10:52 - FSharp.LanguageService.fsproj - created!!!
05-Apr-17 17:10:52 - FSharp.LanguageService.fsproj - created!!!
05-Apr-17 17:10:52 - FSharp.LanguageService.fsproj - cached
05-Apr-17 17:10:52 - FSharp.Core.fsproj - cached
05-Apr-17 17:10:52 - FSharp.Core.fsproj - cached
05-Apr-17 17:10:52 - FSharp.LanguageService.Compiler.fsproj - cached
05-Apr-17 17:10:52 - FSharp.LanguageService.Compiler.fsproj - cached
05-Apr-17 17:10:52 - FSharp.VS.FSI.fsproj - cached
05-Apr-17 17:10:52 - FSharp.VS.FSI.fsproj - cached
05-Apr-17 17:10:52 - FSharp.Compiler.fsproj - cached
05-Apr-17 17:10:52 - FSharp.Compiler.fsproj - cached
05-Apr-17 17:10:52 - FSharp.Compiler.Server.Shared.fsproj - cached
05-Apr-17 17:10:52 - FSharp.Compiler.Server.Shared.fsproj - cached
05-Apr-17 17:10:52 - FSharp.Editor.fsproj - cached
05-Apr-17 17:10:52 - FSharp.Editor.fsproj - cached
05-Apr-17 17:10:52 - FSharp.Editor.fsproj - cached
05-Apr-17 17:10:52 - FSharp.Editor.fsproj - cached
05-Apr-17 17:10:52 - FSharp.Editor.fsproj - created!!!
05-Apr-17 17:10:52 - FSharp.Editor.fsproj - cached
05-Apr-17 17:10:53 - ProjectSystem.fsproj - cached
05-Apr-17 17:10:53 - ProjectSystem.fsproj - cached
05-Apr-17 17:10:53 - ProjectSystem.fsproj - cached
05-Apr-17 17:10:53 - ProjectSystem.fsproj - cached
05-Apr-17 17:10:53 - ProjectSystem.fsproj - cached
05-Apr-17 17:10:53 - ProjectSystem.fsproj - created!!!
05-Apr-17 17:10:53 - VisualFSharp.Unittests.fsproj - cached
05-Apr-17 17:10:53 - VisualFSharp.Unittests.fsproj - created!!!
05-Apr-17 17:10:53 - VisualFSharp.Unittests.fsproj - cached
05-Apr-17 17:10:53 - VisualFSharp.Unittests.fsproj - cached
05-Apr-17 17:10:53 - VisualFSharp.Unittests.fsproj - cached
05-Apr-17 17:10:53 - VisualFSharp.Unittests.fsproj - cached
05-Apr-17 17:10:53 - VisualFSharp.Unittests.fsproj - cached
05-Apr-17 17:10:53 - VisualFSharp.Unittests.fsproj - cached
05-Apr-17 17:10:53 - VisualFSharp.Unittests.fsproj - cached
05-Apr-17 17:10:53 - VisualFSharp.Unittests.fsproj - cached
05-Apr-17 17:10:53 - VisualFSharp.Unittests.fsproj - cached
05-Apr-17 17:10:53 - VisualFSharp.Unittests.fsproj - cached
05-Apr-17 17:10:53 - VisualFSharp.Unittests.fsproj - cached
05-Apr-17 17:10:53 - VisualFSharp.Unittests.fsproj - created!!!
05-Apr-17 17:10:53 - VisualFSharp.Unittests.fsproj - created!!!
05-Apr-17 17:10:53 - VisualFSharp.Unittests.fsproj - cached
05-Apr-17 17:10:53 - FSharp.Build.fsproj - cached
05-Apr-17 17:10:53 - FSharp.Build.fsproj - cached
05-Apr-17 17:10:53 - VisualFSharp.Salsa.fsproj - cached
05-Apr-17 17:10:53 - VisualFSharp.Salsa.fsproj - cached
05-Apr-17 17:10:53 - DummyProviderForLanguageServiceTesting.fsproj - cached
05-Apr-17 17:10:53 - DummyProviderForLanguageServiceTesting.fsproj - created!!!
05-Apr-17 17:10:53 - DummyProviderForLanguageServiceTesting.fsproj - cached
05-Apr-17 17:10:53 - EmptyAssembly.fsproj - cached
05-Apr-17 17:10:53 - EmptyAssembly.fsproj - created!!!
05-Apr-17 17:10:53 - EmptyAssembly.fsproj - cached
05-Apr-17 17:10:53 - Fsc.fsproj - cached
05-Apr-17 17:10:53 - Fsc.fsproj - cached
05-Apr-17 17:10:53 - Fsc.fsproj - created!!!
05-Apr-17 17:10:53 - Fsc.fsproj - cached
05-Apr-17 17:10:53 - FSharp.Compiler.Interactive.Settings.fsproj - cached
05-Apr-17 17:10:53 - FSharp.Compiler.Interactive.Settings.fsproj - created!!!
05-Apr-17 17:10:53 - FSharp.Compiler.Interactive.Settings.fsproj - cached
05-Apr-17 17:10:53 - FsiAnyCPU.fsproj - cached
05-Apr-17 17:10:53 - FsiAnyCPU.fsproj - cached
05-Apr-17 17:10:53 - FsiAnyCPU.fsproj - cached
05-Apr-17 17:10:53 - FsiAnyCPU.fsproj - created!!!
05-Apr-17 17:10:53 - FsiAnyCPU.fsproj - cached
05-Apr-17 17:10:53 - Fsi.fsproj - cached
05-Apr-17 17:10:53 - Fsi.fsproj - cached
05-Apr-17 17:10:53 - Fsi.fsproj - cached
05-Apr-17 17:10:53 - Fsi.fsproj - created!!!
05-Apr-17 17:10:53 - Fsi.fsproj - cached
05-Apr-17 17:10:53 - FSharp.Tests.FSharpSuite.fsproj - cached
05-Apr-17 17:10:53 - FSharp.Tests.FSharpSuite.fsproj - cached
05-Apr-17 17:10:53 - FSharp.Tests.FSharpSuite.fsproj - created!!!
05-Apr-17 17:10:53 - FSharp.Tests.FSharpSuite.fsproj - cached
05-Apr-17 17:10:53 - FSharp.Compiler.Unittests.fsproj - cached
05-Apr-17 17:10:53 - FSharp.Compiler.Unittests.fsproj - cached
05-Apr-17 17:10:53 - FSharp.Compiler.Unittests.fsproj - created!!!
05-Apr-17 17:10:53 - FSharp.Compiler.Unittests.fsproj - cached
05-Apr-17 17:10:53 - FSharp.Core.Unittests.fsproj - cached
05-Apr-17 17:10:53 - FSharp.Core.Unittests.fsproj - created!!!
05-Apr-17 17:10:53 - FSharp.Core.Unittests.fsproj - cached
05-Apr-17 17:10:53 - HostedCompilerServer.fsproj - cached
05-Apr-17 17:10:53 - HostedCompilerServer.fsproj - cached
05-Apr-17 17:10:53 - HostedCompilerServer.fsproj - created!!!
05-Apr-17 17:10:53 - HostedCompilerServer.fsproj - cached
05-Apr-17 17:10:53 - ILComparer.fsproj - cached
05-Apr-17 17:10:53 - ILComparer.fsproj - created!!!
05-Apr-17 17:10:53 - ILComparer.fsproj - cached
05-Apr-17 17:10:53 - LanguageServiceProfiling.fsproj - cached
05-Apr-17 17:10:53 - LanguageServiceProfiling.fsproj - cached
05-Apr-17 17:10:53 - LanguageServiceProfiling.fsproj - created!!!
05-Apr-17 17:10:53 - LanguageServiceProfiling.fsproj - cached

So, yes, there is lot redundant work, but all these are done in about 2 seconds, so...

@saul
Copy link
Contributor

saul commented Apr 6, 2017

How does this work on the dense solution?

@vasily-kirichenko
Copy link
Contributor Author

@saul
Master:

  • 2:16 - projects appeared in solution explorer, solution looks like "Solution Dense (loading...)"
  • ---- one CPU core is 100% loaded, devenv.exe process eats memory like crazy until it reached > 2GB and I kelled it
    This PR:
  • 2:16 - project appeared in solution explorer, solution look like "Solution Dense (100 projects)", the UI is not frozen, memory is about 1GB
  • 7:00 - devenv.exe is 200MB, 15% CPU load (8 logical cores). No idea what it's doing.

@vasily-kirichenko
Copy link
Contributor Author

dotTrace on this PR (that 15% CPU load)

image

@vasily-kirichenko
Copy link
Contributor Author

Looks like AreSameForChecking is the problem, it's deeply nested:

image

That's after I've made the cache here https://github.com/Microsoft/visualfsharp/blob/master/src/fsharp/vs/IncrementalBuild.fs#L1671 global, like

static let cache = TimeStampCache(defaultTimeStamp) 

(it's obviously must be invalidated on file changes (watcher?), but it's not the point here)

@saul
Copy link
Contributor

saul commented Apr 6, 2017

That looks a lot better, thanks for investigating this. I think AreSameForChecking doesn't need to be recursive, though. Maybe remove the recursion and see what the performance is like?

@vasily-kirichenko
Copy link
Contributor Author

@saul With no recursion it was loading CPU at 15% for about 4 minutes, then dropped to zero. I'm not sure we can remove recursion there because if a referenced project changed, we won't know that and project options becomes outdated.

@dsyme
Copy link
Contributor

dsyme commented Apr 10, 2017

@saul With no recursion it was loading CPU at 15% for about 4 minutes, then dropped to zero. I'm not sure we can remove recursion there because if a referenced project changed, we won't know that and project options becomes outdated.

There's surely got to be a better way of doing these comparisons (e.g. making the project options type abstract and giving the values a unique stamp, or a SHA1 hash, then just comparing stamps/hashes)

@saul
Copy link
Contributor

saul commented Apr 13, 2017

Any closing thoughts @vasily-kirichenko ?

@vasily-kirichenko
Copy link
Contributor Author

@saul no. I think I'm not going to touch project system in near future. Let's wait until MS port it to CPS (no sarcasm, I'm just tired with this stuff).

@saul
Copy link
Contributor

saul commented Apr 15, 2017

Fair enough - as this is in the language service though I don't think CPS will help us here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants