Skip to content

Commit

Permalink
updated README
Browse files Browse the repository at this point in the history
  • Loading branch information
Zhenzhong1 committed Sep 26, 2024
1 parent a21aab1 commit 2447f12
Show file tree
Hide file tree
Showing 2 changed files with 38 additions and 41 deletions.
32 changes: 15 additions & 17 deletions ChatQnA/benchmark/performance/helm_charts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,29 +10,27 @@ This document guides you through deploying ChatQnA pipelines using Helm charts.
# on k8s-master node
cd GenAIExamples/ChatQnA/benchmark/performance/helm_charts

# Replace <your token> with your actual Hugging Face token and run the following command:
HUGGINGFACE_TOKEN=<your token>
find . -name '*.yaml' -type f -exec sed -i "s#\${HF_TOKEN}#${HUGGINGFACE_TOKEN}#g" {} \;

# Replace the following placeholders with the desired model IDs:
LLM_MODEL_ID=Intel/neural-chat-7b-v3-3
EMBEDDING_MODEL_ID=BAAI/bge-base-en-v1.5
RERANK_MODEL_ID=BAAI/bge-reranker-base
find . -name '*.yaml' -type f -exec sed -i "s#\$(LLM_MODEL_ID)#${LLM_MODEL_ID}#g" {} \;
find . -name '*.yaml' -type f -exec sed -i "s#\$(EMBEDDING_MODEL_ID)#${EMBEDDING_MODEL_ID}#g" {} \;
find . -name '*.yaml' -type f -exec sed -i "s#\$(RERANK_MODEL_ID)#${RERANK_MODEL_ID}#g" {} \;

# Replace the key of HUGGINGFACEHUB_API_TOKEN with your actual Hugging Face token:
# vim customize.yaml
HUGGINGFACEHUB_API_TOKEN: hf_xxxxx
```

### ChatQnA Installation
### Deploy your ChatQnA

```bash
# Deploy a ChatQnA pipeline using the specified YAML configuration.
# To deploy with different configurations, simply provide a different YAML file.
helm install chatqna helm_charts/ -f helm_charts/oob_single_node.yaml

# Tips: To display rendered manifests according to the given yaml.
helm template chatqna helm_charts/ -f helm_charts/oob_single_node.yaml
helm install chatqna helm_charts/ -f customize.yaml
```

Notes: The provided [BKC manifests](https://github.com/opea-project/GenAIExamples/tree/main/ChatQnA/benchmark) for single, two, and four node Kubernetes clusters are generated using this tool.

## Customize your own ChatQnA pipelines. (Optional)

There are two yaml configs you can specify.

* customize.yaml
This file can specify image names, the number of replicas and CPU cores to mange your pods.

* values.yaml
This file contains the default microservice configurations for ChatQnA. Please review and understand each parameter before making any changes.
47 changes: 23 additions & 24 deletions ChatQnA/benchmark/performance/helm_charts/customize.yaml
Original file line number Diff line number Diff line change
@@ -1,15 +1,14 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0


HUGGINGFACEHUB_API_TOKEN: ${HF_TOKEN}

podSpecs:
- name: chatqna-backend-server-deploy
spec:
image_name: opea/chatqna-no-wrapper
image_tag: latest
replicas: 1
replicas: 2
resources:
limits:
cpu: "8"
Expand All @@ -18,28 +17,6 @@ podSpecs:
cpu: "8"
memory: "8000Mi"

- name: dataprep-deploy
spec:
image_name: opea/dataprep-redis
image_tag: latest
replicas: 1

- name: vector-db
spec:
image_name: redis/redis-stack
image_tag: 7.2.0-v9
replicas: 1

- name: retriever-deploy
spec:
image_name: opea/retriever-redis
image_tag: latest
replicas: 2
resources:
requests:
cpu: "4"
memory: "4000Mi"

- name: embedding-dependency-deploy
spec:
image_name: ghcr.io/huggingface/text-embeddings-inference
Expand Down Expand Up @@ -70,3 +47,25 @@ podSpecs:
resources:
limits:
habana.ai/gaudi: 1

- name: dataprep-deploy
spec:
image_name: opea/dataprep-redis
image_tag: latest
replicas: 1

- name: vector-db
spec:
image_name: redis/redis-stack
image_tag: 7.2.0-v9
replicas: 1

- name: retriever-deploy
spec:
image_name: opea/retriever-redis
image_tag: latest
replicas: 2
resources:
requests:
cpu: "4"
memory: "4000Mi"

0 comments on commit 2447f12

Please sign in to comment.