test: add tests for kv_router::scheduler #1491
Conversation
Signed-off-by: Tianer Zhou <ezhoureal@gmail.com>
ezhoureal
requested review from
a team,
GuanLuo,
PeaBrane,
alec-flowers,
biswapanda,
grahamking,
jthomson04,
kkranen,
oandreeva-nv,
paulhendricks,
rmccorm4,
ryanolson and
tmonty12
as code owners
|
👋 Hi ezhoureal! Thank you for contributing to ai-dynamo/dynamo. Just a reminder: The 🚀 |
github-actions
bot
added
external-contribution
Signed-off-by: Tianer Zhou <ezhoureal@gmail.com>
WalkthroughA new test module was added to Changes
Sequence Diagram(s)sequenceDiagram
participant Test as Test Function
participant Selector as DefaultWorkerSelector
participant Endpoints as Mocked Endpoints
participant Request as SchedulingRequest
Test->>Endpoints: Create mock endpoints
Test->>Request: Create scheduling request
Test->>Selector: Call select_worker(endpoints, request)
alt Endpoints available
Selector->>Endpoints: Evaluate each endpoint
Selector->>Request: Use overlap scores and weights
Selector-->>Test: Return selected worker and scores
else No endpoints
Selector-->>Test: Return NoEndpoints error
end
Possibly related PRs
Poem
📜 Recent review detailsConfiguration used: .coderabbit.yaml 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms (4)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
lib/llm/src/kv_router/scheduler.rs (2)
466-478: Probabilistic tie-breaker ⇒ flaky test
test_tie_breaker_randomnessrelies on random choice and asserts that both
workers are chosen at least once over 10 runs.
With two workers the chance of picking the same worker ten times is 1 / 1024,
small but non-zero – CI flakes are painful.Consider either:
- Injecting a deterministic RNG via DI so the test can control the seed.
- Raising the iteration count and/or using
assert!(results.iter().any(|…|))
with statistical tolerance.
387-395: Unusedresp_txsender – small leak / warning
create_requestbuilds aoneshot::channel()and drops the receiver.
While harmless here, dropping the receiver causes the send in
SchedulingRequest::respondto returnErr, which pollutes trace logs for
any future use of this helper.If the response isn’t needed, create the struct with
tokio::sync::oneshot::channel().0
and explicitly ignore the error inside tests, or add a#[allow(dead_code)]
dummy receiver to keep the channel alive.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
lib/llm/src/kv_router/scheduler.rs(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (4)
- GitHub Check: Build and Test - vllm
- GitHub Check: pre-merge-rust (lib/runtime/examples)
- GitHub Check: pre-merge-rust (lib/bindings/python)
- GitHub Check: pre-merge-rust (.)
Signed-off-by: Tianer Zhou <ezhoureal@gmail.com>
Signed-off-by: Tianer Zhou <ezhoureal@gmail.com>
Signed-off-by: Tianer Zhou <ezhoureal@gmail.com>
Signed-off-by: Tianer Zhou <ezhoureal@gmail.com>
2 participants
Overview:
Adding unit tests for select_worker function.
Details:
Where should the reviewer start?
Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)
Summary by CodeRabbit