Skip to content

Commit 82f6694

Browse files
committed
only prefill-route if there is at least one prefill router
Signed-off-by: PeaBrane <yanrpei@gmail.com>
1 parent 9c368dc commit 82f6694

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

components/backends/vllm/src/dynamo/vllm/handlers.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -166,7 +166,10 @@ async def generate(self, request, context):
166166

167167
try:
168168
prefill_worker_id = None
169-
if self.prefill_router_client is not None:
169+
if (
170+
self.prefill_router_client is not None
171+
and self.prefill_router_client.instance_ids()
172+
):
170173
# Use the prefill router to get best worker ID
171174
best_worker_response = await anext(
172175
await self.prefill_router_client.generate(

0 commit comments

Comments
 (0)