@@ -6,35 +6,33 @@ It can be quickly integrated with vLLM as a backend API server, enabling powerfu
66
77## Prerequisites
88
9- - Setup vLLM environment
10-
11- ## Deploy
12-
13- - Start the vLLM server with the supported chat completion model, e.g.
9+ Set up the vLLM environment by installing all required packages:
1410
1511``` bash
16- vllm serve qwen/Qwen1.5-0.5B-Chat
12+ pip install vllm streamlit openai
1713```
1814
19- - Install streamlit and openai:
15+ ## Deploy
2016
21- ``` bash
22- pip install streamlit openai
23- ```
17+ 1 . Start the vLLM server with a supported chat completion model, e.g.
2418
25- - Use the script: < gh-file:examples/online_serving/streamlit_openai_chatbot_webserver.py >
19+ ``` bash
20+ vllm serve Qwen/Qwen1.5-0.5B-Chat
21+ ```
2622
27- - Start the streamlit web UI and start to chat:
23+ 1. Use the script: < gh-file:examples/online_serving/streamlit_openai_chatbot_webserver.py >
2824
29- ``` bash
30- streamlit run streamlit_openai_chatbot_webserver.py
25+ 1. Start the streamlit web UI and start to chat:
3126
32- # or specify the VLLM_API_BASE or VLLM_API_KEY
33- VLLM_API_BASE=" http://vllm-server-host:vllm-server-port/v1" \
27+ ` ` ` bash
3428 streamlit run streamlit_openai_chatbot_webserver.py
3529
36- # start with debug mode to view more details
37- streamlit run streamlit_openai_chatbot_webserver.py --logger.level=debug
38- ```
30+ # or specify the VLLM_API_BASE or VLLM_API_KEY
31+ VLLM_API_BASE=" http://vllm-server-host:vllm-server-port/v1" \
32+ streamlit run streamlit_openai_chatbot_webserver.py
33+
34+ # start with debug mode to view more details
35+ streamlit run streamlit_openai_chatbot_webserver.py --logger.level=debug
36+ ` ` `
3937
40- ![ ] ( ../../assets/deployment/streamlit-chat.png )
38+ ! [Chat with vLLM assistant in Streamlit ](../../assets/deployment/streamlit-chat.png)
0 commit comments