Run Unbind/Bind VF in a serial manner. #30
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Missing testing on a real system |
adrianchiris
force-pushed
the
serialize-unbind-bind-op
branch
from
dad1d62 to
4bf4694
Compare
Unbind/Bind VF as well as moving RDMA device to namespace causes rdma resources to be re-created. CNI may be invoked in parallel and kernel may provide a VF RDMA resources under a different name. As the mapping of RDMA resources is done in Device plugin prior to CNI invocation, it must not change during CNI invocation. We serialize the CNI's Add,Del operations by Creating/Locking/Unlocking a file, causing kernel to allocate RDMA resources the same name for a given VF. In the future, Systems should use udev PCI based RDMA device names, ensuring consistent RDMA resources names.
adrianchiris
force-pushed
the
serialize-unbind-bind-op
branch
from
4bf4694 to
7de7717
Compare
adrianchiris
commented
May 11, 2020
| @@ -82,6 +108,14 @@ func cmdAdd(args *skel.CmdArgs) error { | |||
| } | |||
|
|
|||
| sm := sriov.NewSriovManager() | |||
|
|
|||
| // Lock CNI operation to serialize the operation | |||
| lock, err := lockCNIExecution() | |||
There was a problem hiding this comment.
it should be noted, that while further optimizations where possible both here and in CmdDel (unlock eralier than on first error or end of execution)
it would have introduced (imo) un-needed code complexity
moshe010
reviewed
May 11, 2020
Mmduh-483
reviewed
May 12, 2020
| @@ -230,6 +264,13 @@ func cmdDel(args *skel.CmdArgs) error { | |||
| } | |||
| defer netns.Close() | |||
|
|
|||
| // Lock CNI operation to serialize the operation | |||
| lock, err := lockCNIExecution() | |||
There was a problem hiding this comment.
It is already locked from line 113
There was a problem hiding this comment.
good catch, ill remove
There was a problem hiding this comment.
looking at this again, i originally thought that i mistakenly locked twice. but this is not the case.
this lock is for cmdDel and on line 113 its for cmdAdd
moshe010
changed the title
|
please fix lint errors |
This is required to release the lock in case of a signal sent to CNI. Note: While operating systems are supposed to free a locked file if the PID of the locking process does not exist, they may not do so promptly.
adrianchiris
force-pushed
the
serialize-unbind-bind-op
branch
from
205901f to
f842f5f
Compare
|
forgot to push :\ it should pass now |
moshe010
approved these changes
May 12, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Unbind/Bind VF as well as moving RDMA device to namespace
causes rdma resources to be re-created.
CNI may be invoked in parallel and kernel may provide a VF
RDMA resources under a different name.
As the mapping of RDMA resources is done in Device plugin
prior to CNI invocation, it must not change during CNI invocation.
We serialize the CNI's Add,Del operations by Creating/Locking/Unlocking
a file, causing kernel to allocate RDMA resources the same name
for a given VF.
In the future, Systems should use udev PCI based RDMA device
names, ensuring consistent RDMA resources names.