BOLT: Assertion failed: allocator ran out of memory

[vient]

Hi, I've build PGO+LTO Clang and instrumented it with BOLT. After that I run it on my test codebase, and it fails pretty early on one of files with mentioned Assertion failed: allocator ran out of memory error.

I've found this string in sources — it is affected by MaxSize which can be set externally so it's not clear for me what my next moves can be. Looks like there are no options to deal with it (or are they?), is it a bug in BOLT or conscious limitation which I can try to up or something else?

[llvmbot]

@llvm/issue-subscribers-bolt

[vient]

Solved it by manually increasing default allocator size. Seems to be not ideal, periods of ~10 seconds regularly occur when most time is spent in kernel, but at least no more assertions. Overall, about 10% time is spent in kernel in my case.

The idea is that mmap size can be set arbitrarily large, it won't actually be consumed until touched and your bump allocator looks perfect in that sense. MAP_NORESERVE may be unnecessary, will test later in different combinations with decreased sizes to see if kernel time decreases.

diff --git a/bolt/runtime/instr.cpp b/bolt/runtime/instr.cpp
index 1a8bcf500..2903b9a6c 100644
--- a/bolt/runtime/instr.cpp
+++ b/bolt/runtime/instr.cpp
@@ -141,8 +141,8 @@ public:
 #endif
       StackBase = reinterpret_cast<uint8_t *>(
           __mmap(0, MaxSize, 0x3 /* PROT_READ | PROT_WRITE*/,
-                 Shared ? 0x21 /*MAP_SHARED | MAP_ANONYMOUS*/
-                        : MAP_PRIVATE_MAP_ANONYMOUS /* MAP_PRIVATE | MAP_ANONYMOUS*/,
+                 Shared ? 0x4021 /*MAP_SHARED | MAP_ANONYMOUS | MAP_NORESERVE*/
+                        : 0x4022 /* MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE*/,
                  -1, 0));
       StackSize = 0;
     }
@@ -206,7 +206,7 @@ public:

 private:
   static constexpr uint64_t Magic = 0x1122334455667788ull;
-  uint64_t MaxSize = 0xa00000;
+  uint64_t MaxSize = 0x10000000;
   uint8_t *StackBase{nullptr};
   uint64_t StackSize{0};
   bool Shared{false};

[vient]

Trimmed it down to

diff --git a/bolt/runtime/instr.cpp b/bolt/runtime/instr.cpp
index 1a8bcf500..27a25e1aa 100644
--- a/bolt/runtime/instr.cpp
+++ b/bolt/runtime/instr.cpp
@@ -206,7 +206,7 @@ public:

 private:
   static constexpr uint64_t Magic = 0x1122334455667788ull;
-  uint64_t MaxSize = 0xa00000;
+  uint64_t MaxSize = 0x4000000;
   uint8_t *StackBase{nullptr};
   uint64_t StackSize{0};
   bool Shared{false};

Didn't try smaller sizes but you get the idea.

[vient]

Looks like it was fixed