Skip to content

Commit 1e701b1

Browse files
docs: add podman setup instructions for official image
Podman can be used for running the official vLLM container image but the run args are slightly different Signed-off-by: Nathan Weinberg <nweinber@redhat.com>
1 parent 05fcd1b commit 1e701b1

File tree

1 file changed

+12
-0
lines changed

1 file changed

+12
-0
lines changed

docs/source/deployment/docker.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,18 @@ $ docker run --runtime nvidia --gpus all \
1919
--model mistralai/Mistral-7B-v0.1
2020
```
2121

22+
This image can also be used with other container engines such as [Podman](https://podman.io/).
23+
24+
```console
25+
$ podman run --gpus all \
26+
-v ~/.cache/huggingface:/root/.cache/huggingface \
27+
--env "HUGGING_FACE_HUB_TOKEN=$HF_TOKEN" \
28+
-p 8000:8000 \
29+
--ipc=host \
30+
vllm/vllm-openai:latest \
31+
--model mistralai/Mistral-7B-v0.1
32+
```
33+
2234
You can add any other <project:#engine-args> you need after the image tag (`vllm/vllm-openai:latest`).
2335

2436
:::{note}

0 commit comments

Comments
 (0)