Skip to content

Commit 48c40aa

Browse files
biswapandaZichengMa
authored andcommitted
fix: vllm router examples (#1942)
1 parent 2fab3ee commit 48c40aa

File tree

2 files changed

+7
-3
lines changed

2 files changed

+7
-3
lines changed

examples/vllm/deploy/agg_router.yaml

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@
1515
apiVersion: nvidia.com/v1alpha1
1616
kind: DynamoGraphDeployment
1717
metadata:
18-
name: vllm-v1-agg
18+
name: vllm-v1-agg-router
1919
spec:
2020
services:
2121
Frontend:
@@ -37,7 +37,7 @@ spec:
3737
periodSeconds: 60
3838
timeoutSeconds: 30
3939
failureThreshold: 10
40-
dynamoNamespace: vllm-v1-agg
40+
dynamoNamespace: vllm-v1-agg-router
4141
componentType: main
4242
replicas: 1
4343
resources:
@@ -58,6 +58,8 @@ spec:
5858
- out=dyn
5959
- --http-port
6060
- "8000"
61+
- --router-mode
62+
- kv
6163
VllmDecodeWorker:
6264
envFromSecret: hf-token-secret
6365
livenessProbe:
@@ -79,7 +81,7 @@ spec:
7981
periodSeconds: 60
8082
timeoutSeconds: 30
8183
failureThreshold: 10
82-
dynamoNamespace: vllm-v1-agg
84+
dynamoNamespace: vllm-v1-agg-router
8385
componentType: worker
8486
replicas: 2
8587
resources:

examples/vllm/deploy/disagg_router.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,8 @@ spec:
5858
- out=dyn
5959
- --http-port
6060
- "8000"
61+
- --router-mode
62+
- kv
6163
VllmDecodeWorker:
6264
dynamoNamespace: vllm-v1-disagg-router
6365
envFromSecret: hf-token-secret

0 commit comments

Comments
 (0)