Skip to content

Commit 336db4d

Browse files
committed
docs: add more comments to explain how to write a router script for
whisper backend Signed-off-by: David Gao <davidgao313@outlook.com>
1 parent 18e3f5e commit 336db4d

File tree

1 file changed

+14
-10
lines changed

1 file changed

+14
-10
lines changed

tutorials/23-whisper-api-transcription.md

Lines changed: 14 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,8 @@ vllm serve \
2929

3030
Create and run a router connected to the Whisper backend:
3131

32+
run-router.sh:
33+
3234
```bash
3335
#!/bin/bash
3436
if [[ $# -ne 2 ]]; then
@@ -37,21 +39,23 @@ if [[ $# -ne 2 ]]; then
3739
fi
3840

3941
uv run python3 -m vllm_router.app \
40-
--host 0.0.0.0 --port "$1" \
41-
--service-discovery static \
42-
--static-backends "$2" \
43-
--static-models "openai/whisper-small" \
44-
--static-model-labels "transcription" \
45-
--routing-logic roundrobin \
46-
--log-stats \
47-
--engine-stats-interval 10 \
48-
--request-stats-window 10
42+
--host 0.0.0.0 --port "$1" \
43+
--service-discovery static \
44+
--static-backends "$2" \
45+
--static-models "openai/whisper-small" \
46+
--static-model-labels "transcription" \
47+
--routing-logic roundrobin \
48+
--log-stats \
49+
--log-level debug \ # log level: "debug", "info", "warning", "error", "critical"
50+
--engine-stats-interval 10 \
51+
--request-stats-window 10
52+
--static-backend-health-checks # Enable this flag to make vllm-router check periodically if the models work by sending dummy requests to their endpoints.
4953
```
5054

5155
Example usage:
5256

5357
```bash
54-
./run-router.sh 8000 http://localhost:8002
58+
./run-router.sh 8000 http://0.0.0.0:8002
5559
```
5660

5761
## 3. Sending a Transcription Request

0 commit comments

Comments
 (0)