Skip to content

Commit 78818dd

Browse files
authored
[Docs] Have a try to improve frameworks/streamlit.md (#24841)
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
1 parent 8e5cdcd commit 78818dd

File tree

1 file changed

+18
-20
lines changed

1 file changed

+18
-20
lines changed

docs/deployment/frameworks/streamlit.md

Lines changed: 18 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -6,35 +6,33 @@ It can be quickly integrated with vLLM as a backend API server, enabling powerfu
66

77
## Prerequisites
88

9-
- Setup vLLM environment
10-
11-
## Deploy
12-
13-
- Start the vLLM server with the supported chat completion model, e.g.
9+
Set up the vLLM environment by installing all required packages:
1410

1511
```bash
16-
vllm serve qwen/Qwen1.5-0.5B-Chat
12+
pip install vllm streamlit openai
1713
```
1814

19-
- Install streamlit and openai:
15+
## Deploy
2016

21-
```bash
22-
pip install streamlit openai
23-
```
17+
1. Start the vLLM server with a supported chat completion model, e.g.
2418

25-
- Use the script: <gh-file:examples/online_serving/streamlit_openai_chatbot_webserver.py>
19+
```bash
20+
vllm serve Qwen/Qwen1.5-0.5B-Chat
21+
```
2622

27-
- Start the streamlit web UI and start to chat:
23+
1. Use the script: <gh-file:examples/online_serving/streamlit_openai_chatbot_webserver.py>
2824

29-
```bash
30-
streamlit run streamlit_openai_chatbot_webserver.py
25+
1. Start the streamlit web UI and start to chat:
3126

32-
# or specify the VLLM_API_BASE or VLLM_API_KEY
33-
VLLM_API_BASE="http://vllm-server-host:vllm-server-port/v1" \
27+
```bash
3428
streamlit run streamlit_openai_chatbot_webserver.py
3529
36-
# start with debug mode to view more details
37-
streamlit run streamlit_openai_chatbot_webserver.py --logger.level=debug
38-
```
30+
# or specify the VLLM_API_BASE or VLLM_API_KEY
31+
VLLM_API_BASE="http://vllm-server-host:vllm-server-port/v1" \
32+
streamlit run streamlit_openai_chatbot_webserver.py
33+
34+
# start with debug mode to view more details
35+
streamlit run streamlit_openai_chatbot_webserver.py --logger.level=debug
36+
```
3937

40-
![](../../assets/deployment/streamlit-chat.png)
38+
![Chat with vLLM assistant in Streamlit](../../assets/deployment/streamlit-chat.png)

0 commit comments

Comments
 (0)