Use mmap for allocation #190
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bkchr
reviewed
Jul 3, 2019
ordian
reviewed
Jul 3, 2019
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
andresilva
approved these changes
Jul 3, 2019
2 tasks
bkchr
approved these changes
Jul 8, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR uses
mmapfor allocation of linear memory.This can be seen that it effectively reverts #115 because the same is done by the system now and we no longer need it in substrate/polkadot.
A little bit of background:
It turns out that creation of a large Vec can actually be really wasteful. There is an specialization for Rust, that leverages
callocfor creating zeroed arrays. However, my testing shows that on macOS it usesbzero(or simply it just zeroes the whole allocation right away).On the other hand,
mmaphas a good property. The allocated memory starts in uncommited state and it is not backed by any physical or swap memory. Only on the first access to a virtual memory page it is allocated and zeroed (if not is not zeroed already).Related to #45, but doesn't fix it since we can go further and avoid copying on grow. This is however outside of the scope of this PR to optimize it for reviewing.