Clear affine slots when dropping a Module
#5321
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit implements a resource usage optimization for Wasmtime with the pooling instance allocator by ensuring that when a
Module
is dropped its backing virtual memory mappings are all removed. Currently when aModule
is dropped it releases a strong reference to its internal memory image but the memory image may stick around in individual pooling instance allocator slots. When using theRandom
allocation strategy, for example, this means that the memory images could stick around for a long time.While not a pressing issue this has resource usage implications for Wasmtime. Namely removing a
Module
does not guarantee the memfd, if in use for a memory image, is closed and deallocated within the kernel. Unfortunately simply closing the memfd is not sufficient as well as the mappings into the address space additionally all need to be removed for the kernel to release the resources for the memfd. This means that to release all kernel-level resources for aModule
all slots which have the memory image mapped in must have the slot reset.This problem isn't particularly present when using the
NextAvailable
allocation strategy since the number of lingering memfds is proportional to the maximum concurrent size of wasm instances. With theRandom
andReuseAffinity
strategies, however, it's much more prominent because the number of lingering memfds can reach the total number of slots available. This can appear as a leak of kernel-level memory which can cause other system instability.To fix this issue this commit adds necessary instrumentation to
Drop for Module
to purge all references to the module in the pooling instance allocator. All index allocation strategies now maintain affinity tracking to ensure that regardless of the strategy in use a module that is dropped will remove all its memory mappings. A new allocation method was added to the index allocator for allocating an index without setting affinity and only allocating affine slots. This is used to iterate over all the affine slots without holding the global index lock for an unnecessarily long time while mappings are removed.