You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This includes the specific commit [vllm-project/vllm#19790](https://github.com/vllm-project/vllm/pull/19790) which enables support for external control of the DP ranks.
@@ -74,31 +74,31 @@ Note: The above architecture illustrates all the components. The final component
74
74
75
75
```bash
76
76
# requires one gpu
77
-
cd examples/vllm_v1
77
+
cd examples/vllm
78
78
bash launch/agg.sh
79
79
```
80
80
81
81
#### Aggregated Serving with KV Routing
82
82
83
83
```bash
84
84
# requires two gpus
85
-
cd examples/vllm_v1
85
+
cd examples/vllm
86
86
bash launch/agg_router.sh
87
87
```
88
88
89
89
#### Disaggregated Serving
90
90
91
91
```bash
92
92
# requires two gpus
93
-
cd examples/vllm_v1
93
+
cd examples/vllm
94
94
bash launch/disagg.sh
95
95
```
96
96
97
97
#### Disaggregated Serving with KV Routing
98
98
99
99
```bash
100
100
# requires three gpus
101
-
cd examples/vllm_v1
101
+
cd examples/vllm
102
102
bash launch/disagg_router.sh
103
103
```
104
104
@@ -108,7 +108,7 @@ This example is not meant to be performant but showcases dynamo routing to data
0 commit comments