ASAN reports oom while there is enough memory

[Kayaking-zh]

I have a program which needs around 400G memory. But asan reports oom while cosuming around 300G memory on a server has 700G memory available.

Below is the error message

False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
AddressSanitizer: Out of memory. The process has exhausted 65536MB for size class 96.
==49==AddressSanitizer's allocator is terminating the process instead of returning 0
==49==If you don't like this behavior set allocator_may_return_null=1
==49==AddressSanitizer CHECK failed: ../../../../libsanitizer/sanitizer_common/sanitizer_allocator.cc:216 "((0)) != (0)" (0x0, 0x0)

My program is built by bazel with gcc 8.3.1 and parameters as following

--copt=-fsanitize=address --copt -fno-omit-frame-pointer --linkopt=-fsanitize=address --linkopt=-static-libasan

I have found similar problem in #607, But I can't update gcc or migrate to clang.

Is there any other workarounds?

[Enna1]

The size of ASan PrimaryAllocator address space is 4T, see
https://github.com/llvm/llvm-project/blob/main/compiler-rt/lib/asan/asan_allocator.h#L219
The 4T PrimaryAllocator address space is split into 64 regions, the size of each region is 64G.

In your case, the region for 96-bytes memory-chunk allocation is exhausted.
IMO, there is no workaround on the ASan part, you need to change your code to reduce live 96-bytes chunks to not exceed 64G. You can set environment variable ASAN_OPTIONS='heap_profile=true' to enable heap profiler and check which parts allocate such huge memory.

@vitalybuka Can we change the CompactPtrT type to u64, so we can increate the kAllocatorSize(4T). https://github.com/llvm/llvm-project/blob/main/compiler-rt/lib/sanitizer_common/sanitizer_allocator_primary64.h#L102
Not sure if this will introduce non-negligible overhead.