jemalloc is not getting used coherently on all platforms. #9925
Comments
|
The recent bug with immediates that we found upgrading Servo to the latest Rust also had a stacktrace that started with jemalloc. That was doing a free of an owned pointer inside an enum variant. So it's used in at least some cases. The specific struct was this: https://github.com/mozilla/servo/blob/master/src/components/main/layout/float_context.rs#L49-L52 and the stacktrace is here: https://gist.github.com/metajack/7010750 |
|
This is causing some serious bugs. On OSX, it looks like free/malloc are |
|
@huonw asked if that stack trace was from linux. It was. |
|
Updated the issue title and description with the results of some more investigation. |
As discovered in #9925, it turns out that we weren't using jemalloc on most platforms. Additionally, on some platforms we were using it incorrectly and mismatching the libc version of malloc with the jemalloc version of malloc. Additionally, it's not clear that using jemalloc is indeed a large performance win in particular situtations. This could be due to building jemalloc incorrectly, or possibly due to using jemalloc incorrectly, but it is unclear at this time. Until jemalloc can be confirmed to integrate correctly on all platforms and has verifiable large performance wins on platforms as well, it shouldn't be part of the default build process. It should still be available for use via the LD_PRELOAD trick on various architectures, but using it as the default allocator for everything would require guaranteeing that it works in all situtations, which it currently doesn't. Closes #9925
|
Yikes! |
`strdup` internally uses glibc malloc to allocate the returned char *. This means that if application is using some other allocator, say jemalloc, `free`-ing result of `strdup` is an undefined behavior. References: https://stackoverflow.com/questions/32944390/what-is-the-rationale-for-not-including-strdup-in-the-c-standard jemalloc/jemalloc#365 rust-lang/rust#9925
`strdup` internally uses glibc malloc to allocate the returned char *. This means that if application is using some other allocator, say jemalloc, `free`-ing result of `strdup` is an undefined behavior. References: https://stackoverflow.com/questions/32944390/what-is-the-rationale-for-not-including-strdup-in-the-c-standard jemalloc/jemalloc#365 rust-lang/rust#9925
`strdup` internally uses glibc malloc to allocate the returned char *. This means that if application is using some other allocator, say jemalloc, `free`-ing result of `strdup` is an undefined behavior. References: https://stackoverflow.com/questions/32944390/what-is-the-rationale-for-not-including-strdup-in-the-c-standard jemalloc/jemalloc#365 rust-lang/rust#9925
`strdup` internally uses glibc malloc to allocate the returned char *. This means that if application is using some other allocator, say jemalloc, `free`-ing result of `strdup` is an undefined behavior. References: https://stackoverflow.com/questions/32944390/what-is-the-rationale-for-not-including-strdup-in-the-c-standard jemalloc/jemalloc#365 rust-lang/rust#9925
UPDATED REPORT
I decided to investigate what was happening and why anything was working at all today, and I found some surprising results. My investigation involved breaking on
uv_fs_openand then seeing what happened. One of the first things that this function does is invokestrdup, which is then later deallocated withfree. I followed these calls to figure out which malloc (part of strdup) and free were getting used. This data was all collected by running./$platform/stage0/bin/rustcand just seeing what happened.global_heap::malloc_rawglobal_heap::exchange_freeIt appears that 32-bit linux is the worst off where the allocator is being mismatched, but I'm not certain that it's not being mismatched on other platforms.
ORIGINAL REPORT
I was stepping through i386 linux recently, and got this backtrace:
This appears to indicate that we are not using jemalloc for the exchange or managed heaps. I did see jemalloc get used by libuv though (but jemalloc is linked statically to libuv)
The text was updated successfully, but these errors were encountered: