Provide a way to replace default malloc to investigate different allocators

[SchrodingerZhu]

Hi,
I have seen a previous discussion at #128 and a recent discussion at https://discourse.julialang.org/t/julia-vs-microsofts-rust-inspired-verona-language-and-e-g-snmalloc-allocator/38610.

After investigating a bit at microsoft/snmalloc#176, I think that the error caused by LD_PRELOAD is not the fault of snmalloc(they even tried exposing cfree to fix the problem). Therefore, I think currently there is no appropriate way to intercept low-level memory APIs for Julia correctly (ptrace is not considered here).

Hence, I open this issue hoping to request an investigation on a proper and efficient way to change the global allocator of Julia.

Several years has passed since #128, though memory allocation may not be the first thing to consider in most situations, some modern allocators like snmalloc and mimalloc can considerably improve the performance in some cases (see https://github.com/schrodingerzhu/bench_suite). Especially, these allocators are providing a better support with some special functions for modern languages. Maybe it is not easy to adapt some special features like deferring the memory returning in local free list in mimalloc, but simply apply the allocator can do good directly. Languages like rust also provides some handy interfaces to change the global allocator with a single line of code.

[JeffBezanson]

It's not clear to me why LD_PRELOAD doesn't work. Looks like something inside libgfortran?

[SchrodingerZhu]

@JeffBezanson I do not know the exact reason either. libgfortran calls a free in that case, but it is the version within system libc.

[ufechner7]

Could be useful for example for using Bumper.jl as allocator for a section of code.

[ViralBShah]

Related: #46298 (jemalloc), #47062 (mimalloc)

[giordano]

This ticket is only about providing a mechanism to change the allocator rather than defaulting to a different one, but in case someone was to suggest using jemalloc as default allocator, issues like the ones described in https://social.treehouse.systems/@marcan/113463801684233089 (e.g. jemalloc/jemalloc#467) make very hard to produce portable applications shipping jemalloc.

[gbaraldi]

If anyone wants to play with this I started #47062 a while ago with mimalloc