mmap impl for MemoryInstance

[pepyakin]

When the heap grows from, say, 1GB to 3GB, in naive vector implementation of MemoryInstance we need to reallocate vector, copy 1GB worth of data into the new vector and then fill with zeroes newly allocated 2GB worth of data.

On 64-bit machines (and maybe in some cases on 32-bit ones) we might allocate virtual memory up to memory instance's limit, and then just move the heap_end pointer upon call to grow, thus avoiding memory copying. Maybe it is possible to do zero-filling in a lazy way, i.e upon the first access to the page, however, i'm not sure how to do this in robust and portable manner.

[eira-fransham]

Lazily zero-filling can be done with calloc, but there is no sound way to call it in Rust right now.

[pepyakin]

Yeah, calloc could help with that, but wouldn't it be better to use directly memory mapping instead of dealing with allocator, since we know that we will allocate no less than 64k?

[eira-fransham]

What we can do is make a huge mmap and then madvise WILLNEED on the bits that will be used (i.e. start of heap to end of heap) and then madvise WONTNEED on the rest. That way we don't need to worry about paying the cost of mmaps being paged to disk.

[eira-fransham]

It looks like Rust has a seperate mechanism for allocating zeroed memory, which would imply that it uses calloc at least with alloc_system (jemalloc has calloc but it's unclear if it can access kernel magic like the system allocator, I'd guess not).

[eira-fransham]

Just to come back to this, you can definitely calloc in stable Rust with:

use std::mem;

unsafe fn calloc(size: usize, count: usize) -> *mut u8 {
    let mut allocation: Vec<u8> = vec![0; size * count];
    let ptr = allocation.as_mut_ptr();
    assert!(ptr as usize % mem::size_of::<usize>() == 0);
    mem::forget(allocation);
    ptr
}

[pepyakin]

Notes from working on #190

1. calloc (the one from libsystem/libmalloc.dylib) actually uses bzero on macOS (tested on 10.14) which leads to massive slowdowns on some workloads.
2. I am not sure why would we need madvise tricks if the memory allocated by mmap by default starts in uncommited state which means that this virtual memory is backed by neither swap nor physical memory.

[Robbepop]

On 64-bit machines (and maybe in some cases on 32-bit ones) we might allocate virtual memory up to memory instance's limit, and then just move the heap_end pointer upon call to grow, thus avoiding memory copying. Maybe it is possible to do zero-filling in a lazy way, i.e upon the first access to the page, however, i'm not sure how to do this in robust and portable manner.

#274 implements exactly this on 64-bit platforms. Therefore this issue can be closed as soon as it has been merged.

[Robbepop]

Closed since #274 has been merged.