Skip to content

Commit 5a1063f

Browse files
committed
docs: using the correct --static-model-types "transcription" param for
proper LM-Cache integration Signed-off-by: David Gao <davidgao313@outlook.com>
1 parent 336db4d commit 5a1063f

File tree

1 file changed

+7
-4
lines changed

1 file changed

+7
-4
lines changed

tutorials/23-whisper-api-transcription.md

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -43,15 +43,18 @@ uv run python3 -m vllm_router.app \
4343
--service-discovery static \
4444
--static-backends "$2" \
4545
--static-models "openai/whisper-small" \
46-
--static-model-labels "transcription" \
46+
--static-model-types "transcription" \
4747
--routing-logic roundrobin \
4848
--log-stats \
49-
--log-level debug \ # log level: "debug", "info", "warning", "error", "critical"
49+
--log-level debug \
5050
--engine-stats-interval 10 \
51-
--request-stats-window 10
52-
--static-backend-health-checks # Enable this flag to make vllm-router check periodically if the models work by sending dummy requests to their endpoints.
51+
--request-stats-window 10 \
52+
--static-backend-health-checks
5353
```
5454

55+
* `--log-level` options: "debug", "info", "warning", "error", "critical"
56+
* `--static-backend-health-checks`: Enable this flag to make vllm-router check periodically if the models work by sending dummy requests to their endpoints.
57+
5558
Example usage:
5659

5760
```bash

0 commit comments

Comments
 (0)