feat(runtime): add RDMA support for jindocache engine #5395
Conversation
- Introduce RDMA protocol support for JindoCache RPC communication - Auto add necessary container capabilities for RDMA-enabled deployments - Filter out RDMA-specific properties from cache client configurations to make dataload work Signed-off-by: TzZtzt <trafalgarz@outlook.com>
Summary of ChangesHello @TrafalgarZZZ, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request integrates RDMA support into the JindoCache runtime, aiming to significantly improve the performance of inter-component communication. It includes the necessary infrastructure changes to automatically configure Kubernetes container capabilities for RDMA-enabled components and refines configuration handling to prevent conflicts with existing dataload processes. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces RDMA support for the JindoCache engine, which is a significant feature enhancement. The changes are well-structured across Helm charts and the Go controller logic. The implementation correctly uses annotations to enable the feature and dynamically adds the required container capabilities. My review includes a few suggestions to improve code robustness and fix a minor issue in a new function. Overall, this is a solid contribution.
| func (e *JindoCacheEngine) RdmaEnabled(runtime *datav1alpha1.JindoRuntime) bool { | ||
| if len(runtime.Annotations) > 0 { | ||
| if rpcStr, exists := runtime.Annotations[common.LabelAnnotationRPCProtocol]; exists { | ||
| switch rpcStr { | ||
| case common.RPCProtocolRDMA: | ||
| return true | ||
| case common.RPCProtocolTCP: | ||
| return false | ||
| default: | ||
| e.Log.Info("WARNING: unrecognized rpc protocol defined in runtime's annotation, exepect to be either \"tcp\" or \"rdma\". Default to use \"tcp\" to continue", "key", common.LabelAnnotationRPCProtocol, "value", rpcStr) | ||
| return false | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // default to false | ||
| return false | ||
| } |
There was a problem hiding this comment.
This function can be slightly simplified by removing the redundant len(runtime.Annotations) > 0 check. Accessing a key from a nil or empty map in Go is safe and will correctly return that the key does not exist. Also, there's a typo exepect in the log message.
I've suggested a cleaner implementation.
func (e *JindoCacheEngine) RdmaEnabled(runtime *datav1alpha1.JindoRuntime) bool {
if rpcStr, exists := runtime.Annotations[common.LabelAnnotationRPCProtocol]; exists {
switch rpcStr {
case common.RPCProtocolRDMA:
return true
case common.RPCProtocolTCP:
return false
default:
e.Log.Info("WARNING: unrecognized rpc protocol defined in runtime's annotation, expect to be either \"tcp\" or \"rdma\". Default to use \"tcp\" to continue", "key", common.LabelAnnotationRPCProtocol, "value", rpcStr)
return false
}
}
// default to false
return false
}| if e.RdmaEnabled(runtime) { | ||
| properties["namespace.rpc.rdma.enable"] = "true" | ||
| if value.Master.ContainerCapabilities == nil { | ||
| value.Master.ContainerCapabilities = &corev1.Capabilities{} | ||
| } | ||
| value.Master.ContainerCapabilities.Add = append(value.Master.ContainerCapabilities.Add, corev1.Capability("SYS_RESOURCE")) | ||
| value.Master.ContainerCapabilities.Add = append(value.Master.ContainerCapabilities.Add, corev1.Capability("IPC_LOCK")) | ||
| } |
There was a problem hiding this comment.
The current implementation appends capabilities without checking if they already exist. This could lead to duplicate capabilities in ContainerCapabilities.Add if this function were called multiple times on the same value object, or if capabilities could be pre-configured in the future. To make the code more robust and idempotent, it's better to add capabilities only if they are not already present.
| if e.RdmaEnabled(runtime) { | |
| properties["namespace.rpc.rdma.enable"] = "true" | |
| if value.Master.ContainerCapabilities == nil { | |
| value.Master.ContainerCapabilities = &corev1.Capabilities{} | |
| } | |
| value.Master.ContainerCapabilities.Add = append(value.Master.ContainerCapabilities.Add, corev1.Capability("SYS_RESOURCE")) | |
| value.Master.ContainerCapabilities.Add = append(value.Master.ContainerCapabilities.Add, corev1.Capability("IPC_LOCK")) | |
| } | |
| if e.RdmaEnabled(runtime) { | |
| properties["namespace.rpc.rdma.enable"] = "true" | |
| if value.Master.ContainerCapabilities == nil { | |
| value.Master.ContainerCapabilities = &corev1.Capabilities{} | |
| } | |
| // A simple helper to add a capability if it doesn't exist | |
| addCap := func(caps []corev1.Capability, c corev1.Capability) []corev1.Capability { | |
| for _, ec := range caps { | |
| if ec == c { | |
| return caps | |
| } | |
| } | |
| return append(caps, c) | |
| } | |
| value.Master.ContainerCapabilities.Add = addCap(value.Master.ContainerCapabilities.Add, "SYS_RESOURCE") | |
| value.Master.ContainerCapabilities.Add = addCap(value.Master.ContainerCapabilities.Add, "IPC_LOCK") | |
| } |
| if e.RdmaEnabled(runtime) { | ||
| properties["storage.namespace.rpc.rdma.enable"] = "true" | ||
| properties["storage.rpc.rdma.enable"] = "true" | ||
| if value.Worker.ContainerCapabilities == nil { | ||
| value.Worker.ContainerCapabilities = &corev1.Capabilities{} | ||
| } | ||
| value.Worker.ContainerCapabilities.Add = append(value.Worker.ContainerCapabilities.Add, corev1.Capability("SYS_RESOURCE")) | ||
| value.Worker.ContainerCapabilities.Add = append(value.Worker.ContainerCapabilities.Add, corev1.Capability("IPC_LOCK")) | ||
|
|
||
| } |
There was a problem hiding this comment.
Similar to the master transformation, this code appends capabilities without checking for their existence. This could lead to duplicates. To improve robustness and idempotency, it's recommended to ensure a capability is added only once.
if e.RdmaEnabled(runtime) {
properties["storage.namespace.rpc.rdma.enable"] = "true"
properties["storage.rpc.rdma.enable"] = "true"
if value.Worker.ContainerCapabilities == nil {
value.Worker.ContainerCapabilities = &corev1.Capabilities{}
}
// A simple helper to add a capability if it doesn't exist
addCap := func(caps []corev1.Capability, c corev1.Capability) []corev1.Capability {
for _, ec := range caps {
if ec == c {
return caps
}
}
return append(caps, c)
}
value.Worker.ContainerCapabilities.Add = addCap(value.Worker.ContainerCapabilities.Add, "SYS_RESOURCE")
value.Worker.ContainerCapabilities.Add = addCap(value.Worker.ContainerCapabilities.Add, "IPC_LOCK")
}
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #5395 +/- ##
==========================================
+ Coverage 57.42% 57.52% +0.10%
==========================================
Files 443 444 +1
Lines 30796 30744 -52
==========================================
+ Hits 17686 17687 +1
+ Misses 11499 11443 -56
- Partials 1611 1614 +3 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Signed-off-by: TzZtzt <trafalgarz@outlook.com>
|
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: RongGu The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
2 participants
Ⅰ. Describe what this PR does
Ⅱ. Does this pull request fix one issue?
NONE
Ⅲ. List the added test cases (unit test/integration test) if any, please explain if no tests are needed.
Ⅳ. Describe how to verify it
Ⅴ. Special notes for reviews
This feature currently is only supported for JindoCache engine. But the code logic can be easily extend to other cache runtime engines.