Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support use of huge pages without CPU pinning #6185

Closed
rsmontero opened this issue Apr 24, 2023 · 1 comment
Closed

Support use of huge pages without CPU pinning #6185

rsmontero opened this issue Apr 24, 2023 · 1 comment
Assignees
@rsmontero
Labels
Category: Core & System Priority: Normal Sponsored Status: Accepted Type: Feature
Milestone
Release 6.6.2

Comments

@rsmontero
Copy link
Member

Description
OpenNebula < 6.6 requires to set a Pin policy for using huge pages. With the implementation of NUMA pinning it should be posible to just specify the desired huge page size, so:

  • OpenNebula will look for a NUMA node with enough free huge pages
  • The scheduling will pick the node with more free pages (i.e. try to balance load across NUMA nodes)
  • The VM will be configured with CPU affinity to the selected NUMA node

Use case
Simply use of huge pages (also easier use of DPDK with Open vSwitch)

Interface Changes
Templates can be defined with just HUGEPAGE_SIZE
@rsmontero rsmontero added this to the Release 6.6.2 milestone Apr 24, 2023
rsmontero added a commit to OpenNebula/docs that referenced this issue Apr 24, 2023
@rsmontero
F OpenNebula/one#6185: Update NUMA hugepage usage
d494b61
rsmontero added a commit to OpenNebula/docs that referenced this issue Apr 24, 2023
@rsmontero
F OpenNebula/one#6185: Resolved issues
05190a3
rsmontero added a commit to OpenNebula/docs that referenced this issue Apr 24, 2023
@rsmontero
F OpenNebula/one#6185: Update NUMA hugepage usage
5ca4c1a 
(cherry picked from commit d494b61)
rsmontero added a commit to OpenNebula/docs that referenced this issue Apr 24, 2023
@rsmontero
F OpenNebula/one#6185: What's new
f588958
rsmontero added a commit that referenced this issue Apr 28, 2023
@rsmontero
F #6185: Support huge pages without CPU pinning
45f28e1 
VM topology can be defined:

- To use a NUMA node and huge pages. Example:
    TOPOLOGY = [ NODE_AFFINITY = 0, HUGEPAGE_SIZE = 2 ]

- To use huge pages without specifying the NUMA node. Example:
    TOPOLOGY = [ HUGEPAGE_SIZE = 2 ]

In any case OpenNebula will:

- look (or check) for a NUMA node with enough free huge pages
- will pick the NUMA node with more free pages (if no NUMA node is specified)
- configure the VM with CPU affinity to the selected NUMA node

(cherry picked from commit 1b3b88e)
feldsam added a commit to FELDSAM-INC/one that referenced this issue Dec 16, 2023
@feldsam
F OpenNebula#6185: Support huge pages without CPU pinning
c6adde2 
Ensure that OpenNebula will: configure the VM with CPU affinity to the auto-selected NUMA node

Signed-off-by: Kristian Feldsam <feldsam@gmail.com>
@feldsam
Copy link
Contributor

feldsam commented Dec 16, 2023 •
edited
Loading

Hello @rsmontero, I created PR for this issue. In the issue description there is "The VM will be configured with CPU affinity to the selected NUMA node", but if no affinity is selected, them huge pages are pinned to some numa node, but vCPUs are not. This PR fix this, so all vCPUs will be pinned to auto-selected numa node.

EDIT:
There is also another drawback of this functionality. When you migrate VM to other host, scheduler auto select new numa node, but it is not actually propagated to running VM. So we ends with some numa node in VM template and other in Libvirt.

EDIT2:
I implemented enhanced migration with better support for VMs with NUMA topology. Will open PR soon!

@feldsam feldsam mentioned this issue Dec 17, 2023
Closed
1 task
@feldsam feldsam mentioned this issue Nov 4, 2024
Open
3 tasks
feldsam pushed a commit to FELDSAM-INC/one that referenced this issue Nov 4, 2024
@rsmontero @feldsam
F OpenNebula#6185: Support huge pages without CPU pinning
99b5602 
VM topology can be defined:

- To use a NUMA node and huge pages. Example:
    TOPOLOGY = [ NODE_AFFINITY = 0, HUGEPAGE_SIZE = 2 ]

- To use huge pages without specifying the NUMA node. Example:
    TOPOLOGY = [ HUGEPAGE_SIZE = 2 ]

In any case OpenNebula will:

- look (or check) for a NUMA node with enough free huge pages
- will pick the NUMA node with more free pages (if no NUMA node is specified)
- configure the VM with CPU affinity to the selected NUMA node

(cherry picked from commit 1b3b88e)
feldsam added a commit to FELDSAM-INC/one that referenced this issue Nov 4, 2024
@feldsam
F OpenNebula#6185: Support huge pages without CPU pinning
ab1e151 
Ensure that OpenNebula will: configure the VM with CPU affinity to the auto-selected NUMA node

Signed-off-by: Kristian Feldsam <feldsam@gmail.com>
rsmontero pushed a commit that referenced this issue Dec 11, 2024
@feldsam @rsmontero
F #6185: Support huge pages without CPU pinning
a68e0a7 
Ensure that OpenNebula will: configure the VM with CPU affinity to the auto-selected NUMA node

Signed-off-by: Kristian Feldsam <feldsam@gmail.com>
(cherry picked from commit c6adde2)
(cherry picked from commit a456c817d018281b4678480066add9baba3af16c)
rsmontero added a commit to OpenNebula/docs that referenced this issue Dec 11, 2024
@rsmontero
F OpenNebula/one#6185: Added fix to resolved issues
6081444
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rsmontero rsmontero

Labels
Category: Core & System Priority: Normal Sponsored Status: Accepted Type: Feature
Projects
None yet
Milestone
Release 6.6.2
Development

No branches or pull requests
2 participants
@rsmontero @feldsam