Infinite loop when dropping specific `Vec` on WASM

[MortenLohne]

If run on wasm32-unknown-unknown, the following code returns a "cursed" 1024-byte Vec<u8>: if it is dropped by the caller, the runtime goes into an apparent infinite loop, consuming CPU forever.

fn get_cursed_vec() -> Vec<u8> {
    // Reserve a 1.5 GiB outer vector, to OOM faster
    let mut test_vector: Vec<Vec<u8>> = Vec::with_capacity(2usize.pow(27));

    // Allocate 1KiB vectors until we run out of memory
    loop {
        let mut inner_vector = vec![];
        if inner_vector.try_reserve_exact(1024).is_err() {
            // Remove the final inner vector. It is cursed, and cannot be dropped.
            return mem::take(test_vector.last_mut().unwrap());
        };
        test_vector.push(inner_vector);
    }
}

Full reproduction repo here, also running at https://rust-wasm-bug.vercel.app/

Quick explanation: The function allocates millions of identical vectors until Vec::try_reserve_exact(1024) fails, then it returns the penultimate vector, i.e. the last one that was allocated successfully. The millions of other vectors are successfully dropped. If the cursed vector is mem::forget()'d, the curse is broken and the problem goes away.

I have not been able to reproduce the issue on different targets. I suspect a bug in dlmalloc, but I could not reproduce the issue by setting dlmalloc as the global allocator and running on i686-unknown-linux-gnu.

Meta

rustc --version --verbose: (Issue also happens on stable)

rustc 1.90.0-nightly (8f08b3a32 2025-07-18)
binary: rustc
commit-hash: 8f08b3a32478b8d0507732800ecb548a76e0fd0c
commit-date: 2025-07-18
host: x86_64-unknown-linux-gnu
release: 1.90.0-nightly
LLVM version: 20.1.8

Stopping the stalled script in Firefox gave the following:

Script terminated by timeout at:
rust_wasm_bug.wasm.dlmalloc::dlmalloc::Dlmalloc<A>::free::h465f83a8a3d89dcd@http://127.0.0.1:8000/rust_wasm_bug_bg.wasm:wasm-function[8]:0x1fcd
rust_wasm_bug.wasm.__rustc[37882050545b4706]::__rdl_dealloc@http://127.0.0.1:8000/rust_wasm_bug_bg.wasm:wasm-function[41]:0x43c4
rust_wasm_bug.wasm.__rustc[37882050545b4706]::__rust_dealloc@http://127.0.0.1:8000/rust_wasm_bug_bg.wasm:wasm-function[82]:0x4a0f
rust_wasm_bug.wasm.rust_wasm_bug::main::h7b80711b02f89ef2@http://127.0.0.1:8000/rust_wasm_bug_bg.wasm:wasm-function[24]:0x38f0
rust_wasm_bug.wasm.main@http://127.0.0.1:8000/rust_wasm_bug_bg.wasm:wasm-function[99]:0x4abf
rust_wasm_bug.wasm.@http://127.0.0.1:8000/rust_wasm_bug_bg.wasm:wasm-function[98]:0x4aba
__wbg_finalize_init@http://127.0.0.1:8000/rust_wasm_bug.js:93:10
__wbg_init@http://127.0.0.1:8000/rust_wasm_bug.js:147:12
async*run@http://127.0.0.1:8000/:15:19
@http://127.0.0.1:8000/:18:9

[MortenLohne]

I've been able to make some progress. If I manually set dlmalloc as the global allocator with

[dependencies]
dlmalloc = { version = "0.2.9", features = ["global"] }

and

#[global_allocator]
static ALLOC: dlmalloc::GlobalDlmalloc = dlmalloc::GlobalDlmalloc;

in lib.rs, and build the project in debug mode, the issue goes away. Neither step is sufficient on their own, dlmalloc override + release mode still results in an infinite loop.

EDIT: Slapping #[inline(never)] on segment_holding() in dlmalloc (code here) "fixes" the issue in release mode as well. dlmalloc is a direct C port and consists almost entirely of unsafe code, so UB seems like a plausible cause. Alternatively, a compiler bug.

[saethlin]

There is definitely some suspicious code in dlmalloc, but the fact that this code is trying to observe allocation failure makes me wary.

This build scenario makes me think that this is just about whether the dlmalloc dependency is optimized.

This still only reproduces on wasm, right?

[MortenLohne]

Yes. I tried running it on wasmtime with wasm32-wasip1 as well, setting dlmalloc as the global allocator, but I was unable to reproduce the issue.

[MortenLohne]

Looks like dlmalloc as a global allocator is unsound under both Stacked Borrows and Tree Borrows. Running cargo miri test reports UB before the test runner can even start the actual tests. This might not even be related to my issue, but I think it makes UB in dlmalloc the more likely culprit.

I've created alexcrichton/dlmalloc-rs#51 to address this.