Memory usage

[adamConnerSax]

I know versions of this have been reported before but I thought this might be a useful data point and maybe a way for me to get things to try...
I'm using ghcide-0.1.0.0, with ghc-8.6.5, from emacs (with recent versions of hie-bios and haskell-lsp) on Mac OSX 10.15.3.
It works fine but memory usage (of ghcide) starts at about 5GB, grows fast and eventually (an hour of editing a few files in the project) makes things too slow. So I kill it and then emacs asks me to restart and it all works fine for a while.
One observation: I was using ghcide 0.0.6 and the same thing would happen but much more slowly. I'd maybe restart once a day instead of once an hour.
Anything I can try to improve this or help debug it?

[mpickering]

@adamConnerSax You want to save your bug report until the work on haskell/ghcide#355 is resolved. Perhaps you can try out that branch?

[adamConnerSax]

Does it work with 8.6.5? Then I'd be happy to. But I am stuck with a dependency that I can't use with 8.8.
I looked at that issue/branch and somehow thought it was 8.8 only.

[mpickering]

There is a branch which works with 8.6.5 which I am planning to work on a bit more tonight.

[adamConnerSax]

If you point me to a commit/branch, I'm happy to try!

[mpickering]

https://github.com/mpickering/ghcide/tree/wip/interface-files-hie-8.6.5

[adamConnerSax]

That's working! And holding at about 2.7GB. I can't always tell if all the features are working...sometimes I get the little window in the upper left with definitions and sometimes I don't, but that was true before as well, I think.
Spoke too soon. Also growing as I open more files in emacs. But more slowly for sure.

[mpickering]

@adamConnerSax In the current design no results of shake rules are ever garbage collected, so over the course of run then it will leak memory. The branch should definitely use less memory overall though, especially on subsequent runs once the interface files have been created.

[LukaHorvat]

@mpickering Is that branch merged somewhere now or should I still use it if I want to test it?

[pepeiborra]

Yes it is, merged to master

[mpardalos]

We've spoken with @mpickering about working on this as part of my GSoC project. We would need to add some kind of garbage collection scheme for the results of the Shake rules.

Would something simple like removing all entries relating to a file when it is closed work?
Also, would performance regressions relating to this kind of change show up in the benchmarks?

[jneira]

@mpardalos hi! anything of interest about this issue? could it be closed?

[mpardalos]

I was actually planning to start work on this. It's still very much an issue. It should stay open.

[jneira]

@mpardalos dont want to bother you too much but gsoc has finished some time ago afaik: do you have some additional insights about this issue? thanks 🙂

[mpardalos]

@jneira my GSoC time ended before I had a chance to address this memory leak, however, I had done work on identifying what exactly was causing the memory leak that was merged into ghcide as haskell/ghcide#922, and that should help in finding a fix

I have discussed some findings in haskell/ghcide#826 (which ended up becoming haskell/ghcide#922) that you might find useful.

[jneira]

not sure if last changes around performance in hls 1.5.0 would affect this

[jneira]

@adamConnerSax hi, there had been lot of changes since the issue and we have newer issues about memory usage, so i am gonna close this one. If you continue expriencing excessive memory usage with the last hls version feel free to comment in any of the newer memory issues or open a new one if no one match your observations