You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* fix multiple H1 headings
* remove unnecessary use of HTML
* fix missing indents on ordered list content
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
Co-authored-by: Abolfazl Shahbazi <abolfazl.shahbazi@intel.com>
This ChatQnA use case performs RAG using LangChain, Redis VectorDB and Text Generation Inference on Intel Gaudi2 or Intel XEON Scalable Processors. The Intel Gaudi2 accelerator supports both training and inference for deep learning models in particular for LLMs. Please visit[Habana AI products](https://habana.ai/products) for more details.
15
+
This ChatQnA use case performs RAG using LangChain, Redis VectorDB and Text Generation Inference on Intel Gaudi2 or Intel XEON Scalable Processors. The Intel Gaudi2 accelerator supports both training and inference for deep learning models in particular for LLMs. Visit[Habana AI products](https://habana.ai/products) for more details.
16
16
17
-
# Deploy ChatQnA Service
17
+
##Deploy ChatQnA Service
18
18
19
19
The ChatQnA service can be effortlessly deployed on either Intel Gaudi2 or Intel XEON Scalable Processors.
20
20
21
21
Currently we support two ways of deploying ChatQnA services with docker compose:
22
22
23
23
1. Start services using the docker image on `docker hub`:
24
24
25
-
```bash
26
-
docker pull opea/chatqna:latest
27
-
```
25
+
```bash
26
+
docker pull opea/chatqna:latest
27
+
```
28
28
29
-
Two type of UI are supported now, choose one you like and pull the referred docker image.
29
+
Two type of UI are supported now, choose one you like and pull the referred docker image.
30
30
31
-
If you choose conversational UI, follow the [instruction](https://github.com/opea-project/GenAIExamples/tree/main/ChatQnA/docker/gaudi#-launch-the-conversational-ui-optional) and modify the [compose.yaml](./docker/xeon/compose.yaml).
31
+
If you choose conversational UI, follow the [instruction](https://github.com/opea-project/GenAIExamples/tree/main/ChatQnA/docker/gaudi#-launch-the-conversational-ui-optional) and modify the [compose.yaml](./docker/xeon/compose.yaml).
32
32
33
-
```bash
34
-
docker pull opea/chatqna-ui:latest
35
-
# or
36
-
docker pull opea/chatqna-conversation-ui:latest
37
-
```
33
+
```bash
34
+
docker pull opea/chatqna-ui:latest
35
+
# or
36
+
docker pull opea/chatqna-conversation-ui:latest
37
+
```
38
38
39
39
2. Start services using the docker images `built from source`: [Guide](./docker)
40
40
41
-
## Setup Environment Variable
41
+
###Setup Environment Variable
42
42
43
43
To set up environment variables for deploying ChatQnA services, follow these steps:
2. If you are in a proxy environment, also set the proxy-related environment variables:
56
56
57
-
```bash
58
-
export http_proxy="Your_HTTP_Proxy"
59
-
export https_proxy="Your_HTTPs_Proxy"
60
-
```
57
+
```bash
58
+
export http_proxy="Your_HTTP_Proxy"
59
+
export https_proxy="Your_HTTPs_Proxy"
60
+
```
61
61
62
62
3. Set up other environment variables:
63
63
64
-
> Notice that you can only choose <b>one</b> command below to set up envs according to your hardware. Other that the port numbers may be set incorrectly.
64
+
> Notice that you can only choose **one** command below to set up envs according to your hardware. Other that the port numbers may be set incorrectly.
Find the corresponding [compose.yaml](./docker/xeon/compose.yaml).
91
91
92
92
```bash
93
93
cd GenAIExamples/ChatQnA/docker/xeon/
@@ -96,7 +96,7 @@ docker compose up -d
96
96
97
97
Refer to the [Xeon Guide](./docker/xeon/README.md) for more instructions on building docker images from source.
98
98
99
-
## Deploy ChatQnA on NVIDIA GPU
99
+
###Deploy ChatQnA on NVIDIA GPU
100
100
101
101
```bash
102
102
cd GenAIExamples/ChatQnA/docker/gpu/
@@ -105,61 +105,61 @@ docker compose up -d
105
105
106
106
Refer to the [NVIDIA GPU Guide](./docker/gpu/README.md) for more instructions on building docker images from source.
107
107
108
-
## Deploy ChatQnA into Kubernetes on Xeon & Gaudi with GMC
108
+
###Deploy ChatQnA into Kubernetes on Xeon & Gaudi with GMC
109
109
110
110
Refer to the [Kubernetes Guide](./kubernetes/README.md) for instructions on deploying ChatQnA into Kubernetes on Xeon & Gaudi with GMC.
111
111
112
-
## Deploy ChatQnA into Kubernetes on Xeon & Gaudi without GMC
112
+
###Deploy ChatQnA into Kubernetes on Xeon & Gaudi without GMC
113
113
114
114
Refer to the [Kubernetes Guide](./kubernetes/manifests/README.md) for instructions on deploying ChatQnA into Kubernetes on Xeon & Gaudi without GMC.
115
115
116
-
## Deploy ChatQnA into Kubernetes using Helm Chart
116
+
###Deploy ChatQnA into Kubernetes using Helm Chart
117
117
118
-
Install Helm (version >= 3.15) first. Please refer to the [Helm Installation Guide](https://helm.sh/docs/intro/install/) for more information.
118
+
Install Helm (version >= 3.15) first. Refer to the [Helm Installation Guide](https://helm.sh/docs/intro/install/) for more information.
119
119
120
120
Refer to the [ChatQnA helm chart](https://github.com/opea-project/GenAIInfra/tree/main/helm-charts/chatqna) for instructions on deploying ChatQnA into Kubernetes on Xeon & Gaudi.
121
121
122
-
## Deploy ChatQnA on AI PC
122
+
###Deploy ChatQnA on AI PC
123
123
124
124
Refer to the [AI PC Guide](./docker/aipc/README.md) for instructions on deploying ChatQnA on AI PC.
125
125
126
-
# Consume ChatQnA Service
126
+
##Consume ChatQnA Service
127
127
128
128
Two ways of consuming ChatQnA Service:
129
129
130
130
1. Use cURL command on terminal
131
131
132
-
```bash
133
-
curl http://${host_ip}:8888/v1/chatqna \
134
-
-H "Content-Type: application/json" \
135
-
-d '{
136
-
"messages": "What is the revenue of Nike in 2023?"
137
-
}'
138
-
```
132
+
```bash
133
+
curl http://${host_ip}:8888/v1/chatqna \
134
+
-H "Content-Type: application/json" \
135
+
-d '{
136
+
"messages": "What is the revenue of Nike in 2023?"
137
+
}'
138
+
```
139
139
140
140
2. Access via frontend
141
141
142
-
To access the frontend, open the following URL in your browser: `http://{host_ip}:5173`
142
+
To access the frontend, open the following URL in your browser: `http://{host_ip}:5173`
143
143
144
-
By default, the UI runs on port 5173 internally.
144
+
By default, the UI runs on port 5173 internally.
145
145
146
-
If you choose conversational UI, use this URL: `http://{host_ip}:5174`
146
+
If you choose conversational UI, use this URL: `http://{host_ip}:5174`
147
147
148
148
# Troubleshooting
149
149
150
-
1. If you get errors like "Access Denied", please [validate micro service](https://github.com/opea-project/GenAIExamples/tree/main/ChatQnA/docker/xeon#validate-microservices) first. A simple example:
150
+
1. If you get errors like "Access Denied", [validate micro service](https://github.com/opea-project/GenAIExamples/tree/main/ChatQnA/docker/xeon#validate-microservices) first. A simple example:
151
151
152
-
```bash
153
-
http_proxy="" curl ${host_ip}:6006/embed -X POST -d '{"inputs":"What is Deep Learning?"}' -H 'Content-Type: application/json'
154
-
```
152
+
```bash
153
+
http_proxy="" curl ${host_ip}:6006/embed -X POST -d '{"inputs":"What is Deep Learning?"}' -H 'Content-Type: application/json'
154
+
```
155
155
156
-
2. (Docker only) If all microservices work well, please check the port ${host_ip}:8888, the port may be allocated by other users, you can modify the `compose.yaml`.
156
+
2. (Docker only) If all microservices work well, check the port ${host_ip}:8888, the port may be allocated by other users, you can modify the `compose.yaml`.
157
157
158
-
3. (Docker only) If you get errors like "The container name is in use", please change container name in `compose.yaml`.
158
+
3. (Docker only) If you get errors like "The container name is in use", change container name in `compose.yaml`.
159
159
160
-
# Monitoring OPEA Service with Prometheus and Grafana dashboard
160
+
##Monitoring OPEA Service with Prometheus and Grafana dashboard
161
161
162
-
OPEA microservice deployment can easily be monitored through Grafana dashboards in conjunction with Prometheus data collection. Please follow the [README](https://github.com/opea-project/GenAIEval/blob/main/evals/benchmark/grafana/README.md) to setup Prometheus and Grafana servers and import dashboards to monitor the OPEA service.
162
+
OPEA microservice deployment can easily be monitored through Grafana dashboards in conjunction with Prometheus data collection. Follow the [README](https://github.com/opea-project/GenAIEval/blob/main/evals/benchmark/grafana/README.md) to setup Prometheus and Grafana servers and import dashboards to monitor the OPEA service.
0 commit comments