llvm: Use mimalloc for lld/llvm/clang #4591
Draft
+392
−54
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
EXPERIMENTAL DO NOT MERGE
I saw the developer of MOLD mention that using mimalloc with LLD improved performance by about 10%, so I set about seeing how that could be implemented in our package.
There were basically three methods that I could use:
This PR essentially modifies LLVM to create a new LLVM library
libLLVMMimalloc.a
and then statically links that into the clang, lld, and llvm-ar binaries (as those seemed to be where we would most want to see speed benefits). This library essentially causes mimalloc to replace the glibc malloc and uses mimalloc APIs to suppress mimalloc logging output (it's possible to re-enable via envvar). This code is called as close to the start ofmain
for each of these binaries as seemed reasonable. This is basically how MOLD implements it BTW.Note that an earlier attempt at this added this functionality to
LLVMSupport
which is linked intolibLLVM
. An unfortunate consequence of that approach is that it caused everything linked against libLLVM to use mimalloc which wouldn't have been so bad if that didn't include mesa (replacing the memory allocator for mesa seemed unwise to me).Also, this patch includes changes to the bolt source as this patch is intended to be applied to that as well.
Testing
I've done a few test builds with this patch applied and results seem somewhat promising. It shaved a few seconds off of the qt6-base build though it really should be tested with something like qt6-webengine to really put it through its paces.
You can see if this is working by doing something like
MIMALLOC_VERBOSE=1 clang --version
which will print additional information if MIMALLOC is being used.