PPU LLVM: Fix memory leaks and protect against the rise of CPU threads in the coming years #15377
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In RPCS3 leaks memory on PPU compilation #15376, vsh.self is being precompiled. And when that happens, RPCS3 attempts to precompile nearly every PPU executable it finds lying inisde the firmware. By itself this is currently the intended behavior, but after some debugging I discovered that the memory of every executable lying inisde the "/dev_flash/sys/" is not being released after it finalizes.
The proper way to do it is: whitelist only file under "/dev_flash/sys/external", of PRX type only and only if being done ingame.
The second problem is that RPCS3 naively allows concurrent loads of executable PPU files and their entire LLVM workloads per CPU thread. This is not only a cause for massive memory usages during precompilation but is also not really needed as each files may get multiple threads to compile it as well depending on its size. So I added a limit which allows to load files of total sum of RAM size / 800. This is not accurate but it does work nicely with the previousl limit for Overlay files of 2 with Metal Gear Solid 4 on 16GB of RAM.
This should protect from most memory shortages in the near future when CPUs have even more threads.
Fix PPU LLVM memory cleanup of modules outside of /dev_flash/sys/external but inside /dev_flash even ingame, it turns out memory cleanup failed because the entry used to store the JIT instance uses a key that was different between creation access and removal access.
Minor SPU LLVM worker memory optimization when SPU LLVM ingame workers are not active.
Fixes #15376