Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(vmm): only use memfd if no vhost-user-blk devices configured #4498

Merged
merged 6 commits into from
Mar 15, 2024

Conversation

kalyazin
Copy link
Contributor

Changes

Only back guest memory with a memfd if a vhost-user-blk device is configured in the VM, otherwise fall back to anonymous private memory.

This is recovering performance demonstrated before commit 027a992 had been merged.

Reason

Page faults are more expensive for shared memory mapping (like memfd).

License Acceptance

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.

PR Checklist

  • [ ] If a specific issue led to this PR, this PR closes the issue.
  • The description of changes is clear and encompassing.
  • [ ] Any required documentation changes (code and docs) are included in this
    PR.
  • [ ] API changes follow the Runbook for Firecracker API changes.
  • User-facing changes are mentioned in CHANGELOG.md.
  • All added/changed functionality is tested.
  • [ ] New TODOs link to an issue.
  • Commits meet
    contribution quality standards.

  • This functionality cannot be added in rust-vmm.

Copy link

codecov bot commented Mar 13, 2024

Codecov Report

Attention: Patch coverage is 70.00000% with 6 lines in your changes are missing coverage. Please review.

Project coverage is 81.54%. Comparing base (655af13) to head (58c7c56).

Files Patch % Lines
src/vmm/src/builder.rs 70.00% 6 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4498      +/-   ##
==========================================
- Coverage   81.63%   81.54%   -0.10%     
==========================================
  Files         243      243              
  Lines       29825    29839      +14     
==========================================
- Hits        24349    24333      -16     
- Misses       5476     5506      +30     
Flag Coverage Δ
4.14-c5n.metal 78.86% <70.00%> (?)
4.14-c7g.metal ?
4.14-m5d.metal ?
4.14-m5n.metal 78.85% <70.00%> (?)
4.14-m6a.metal 77.99% <70.00%> (-0.11%) ⬇️
4.14-m6g.metal 76.94% <70.00%> (-0.11%) ⬇️
4.14-m6i.metal 78.84% <70.00%> (-0.10%) ⬇️
4.14-m7g.metal 76.94% <70.00%> (?)
5.10-c5n.metal 81.51% <70.00%> (?)
5.10-c7g.metal ?
5.10-m5d.metal ?
5.10-m5n.metal 81.50% <70.00%> (?)
5.10-m6a.metal 80.73% <70.00%> (-0.11%) ⬇️
5.10-m6g.metal 79.82% <70.00%> (-0.11%) ⬇️
5.10-m6i.metal 81.49% <70.00%> (-0.11%) ⬇️
5.10-m7g.metal 79.82% <70.00%> (?)
6.1-c5n.metal 81.51% <70.00%> (?)
6.1-m5n.metal 81.50% <70.00%> (?)
6.1-m6a.metal 80.73% <70.00%> (-0.11%) ⬇️
6.1-m6g.metal 79.82% <70.00%> (-0.11%) ⬇️
6.1-m6i.metal 81.49% <70.00%> (-0.11%) ⬇️
6.1-m7g.metal 79.82% <70.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@kalyazin kalyazin force-pushed the memfd_anon branch 3 times, most recently from 5919aba to f4e4904 Compare March 14, 2024 11:57
Page faults are more expensive for shared memory mapping (like memfd).
For this reason, we only back guest memory with a memfd
if a vhost-user-blk device is configured in the VM, otherwise
we fall back to anonymous private memory.

This is recovering performance demonstrated before commit
027a992 had been merged.

The vhost-user-blk branch in VM creation is not currently
covered by Rust integration tests, because we are looking for
a converging solution, so guest memory is always created
in the same way, and implementing required integration test
infrastructure code (spawing a backend process) would not
be useful long term.

Signed-off-by: Nikita Kalyazin <kalyazin@amazon.com>
This is because guest memory is backed by anon
private mapping in a regular case.

Signed-off-by: Nikita Kalyazin <kalyazin@amazon.com>
Huge pages tests search for memory mapping name,
which is now anon private instead of memfd.

Signed-off-by: Nikita Kalyazin <kalyazin@amazon.com>
This brings the former negative file size limit test
from before 60737eb ,
which is valid for anon private guest memory backing.

Signed-off-by: Nikita Kalyazin <kalyazin@amazon.com>
@kalyazin kalyazin marked this pull request as ready for review March 14, 2024 13:49
@kalyazin kalyazin self-assigned this Mar 14, 2024
@kalyazin kalyazin added the Status: Awaiting review Indicates that a pull request is ready to be reviewed label Mar 14, 2024
roypat
roypat previously approved these changes Mar 14, 2024
bchalios
bchalios previously approved these changes Mar 14, 2024
Copy link
Contributor

@xmarcalx xmarcalx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i have some small suggestions

CHANGELOG.md Outdated Show resolved Hide resolved
docs/api_requests/block-vhost-user.md Outdated Show resolved Hide resolved
docs/api_requests/block-vhost-user.md Show resolved Hide resolved
Mention memfd page fault overhead and host pagecache
usage for virtio block devices compared to vhost-user.

Signed-off-by: Nikita Kalyazin <kalyazin@amazon.com>
Adds a changelog entry that VMs that do not use
vhost-user-blk devices are backed by anon private memory mapping.

Signed-off-by: Nikita Kalyazin <kalyazin@amazon.com>
@kalyazin kalyazin dismissed stale reviews from bchalios and roypat via 58c7c56 March 14, 2024 17:05
@kalyazin kalyazin requested a review from xmarcalx March 14, 2024 17:06
Copy link
Contributor

@xmarcalx xmarcalx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the changes.

@kalyazin kalyazin merged commit a6ee3b5 into firecracker-microvm:main Mar 15, 2024
6 of 7 checks passed
@kalyazin kalyazin deleted the memfd_anon branch March 15, 2024 09:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Awaiting review Indicates that a pull request is ready to be reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants