Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Support memory decommission of instance memory on macOS #9164

Closed
wants to merge 7 commits into from

Conversation

athei
Copy link
Member

@athei athei commented Jun 21, 2021

This PR adds the memory decommission support for macOS that was added for Linux in #8998.

While we don't support running validators on macOS and thus performance is less of a concern we should still aim for feature parity on all systems we support. Otherwise we might run in situations where development on those plattforms will be impeded by high resource consumption or other inconveniences. This can be a slippery slope that can eventually lead to having Linux as the de facto exclusive platform.

Supporting this on macOS is more involved because the madvise API won't work there to remove memory from the resident set. One has to use the low level mach_* interfaces. The memory needs to be allocated in a specific way so that the kernel can track it for decommission. In addition we need to work around some old TODO that limits purgeable memory mappings to 32bit. I filed a radar for that.

I did the following in order to implement this feature:

  • Allocate wasmtime memory using mach_allocate by implementing the MemoryCreator trait of wasmtime
  • Added code to read out memory mappings on macOS and used it to support the resident memory test on macOS
  • Add the actual code to decommission (purge in mach lingo) the memory when the instance is put back into the cache

@athei athei added A0-please_review Pull request needs code review. B5-clientnoteworthy C1-low PR touches the given topic and has a low impact on builders. D2-notlive 💤 PR contains changes in a runtime directory that is not deployed to a chain that requires an audit. labels Jun 21, 2021
@athei athei requested a review from pepyakin June 21, 2021 15:00
@athei
Copy link
Member Author

athei commented Jun 23, 2021

Had a talk with @pepyakin . He is planning to migrate our wasmtime allocation strategy to pooling. That might solve our memory footprint on all operating systems without resorting to operating system specific code in our codebase. We put this on hold until be explored that solution.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A0-please_review Pull request needs code review. C1-low PR touches the given topic and has a low impact on builders. D2-notlive 💤 PR contains changes in a runtime directory that is not deployed to a chain that requires an audit.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant