Enable vLLM Gaudi support for LLM service based on officially habana vllm release #137

tianyil1 · 2024-06-06T06:13:49Z

Description

This PR enabled the vLLM Gaudi support for LLM service, which leveraged the habana/vllm-fork the newly officially released version Intel® Gaudi® 2 support with SynapseAI 1.16.0.

Issues

n/a.

Type of change

List the type of change like below. Please delete options that are not relevant.

New feature (non-breaking change which adds new functionality)

Dependencies

No newly introduced 3rd party dependency.

Tests

This PR is tested in the Gaudi2 server with:
2 sockets Intel(R) Xeon(R) Platinum 8368 CPU @ 2.40GHz
8 Gaudi nodes, HL-SMI Version: hl-1.16.0-fw-50.1.2.0, Driver version: 1.16.0-94aac46

vLLM Gaudi Service
Client Curl Test

tianyil1 · 2024-06-06T06:19:10Z

Have updated the PR based on habana team's officially released 1.16 version from the previous PR #126, which was tested well in the 8 Gaudi nodes with 1.16 hw and sw version. Please have a check and review @Jian-Zhang @carsonwang @ftian1 @lvliang-intel.

tianyil1 · 2024-06-11T07:27:24Z

This PR is ready to merge. Please help review and merge. @ftian1 @lvliang-intel

Signed-off-by: tianyil1 <tianyi.liu@intel.com>

for more information, see https://pre-commit.ci

Signed-off-by: tianyil1 <tianyi.liu@intel.com>

* fix stream=false doesn't work issue Signed-off-by: letonghan <letong.han@intel.com> * support embedding comp with llama_index Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add More Contents to the Table of MicroService (#141) * Add More Contents to the Table MicroService Signed-off-by: zehao-intel <zehao.huang@intel.com> * reorder Signed-off-by: zehao-intel <zehao.huang@intel.com> * Update README.md * refine structure Signed-off-by: zehao-intel <zehao.huang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix model Signed-off-by: zehao-intel <zehao.huang@intel.com> * refine table Signed-off-by: zehao-intel <zehao.huang@intel.com> * put llm to the ground Signed-off-by: zehao-intel <zehao.huang@intel.com> --------- Signed-off-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Use common security content for OPEA projects (#151) * add python coverage Signed-off-by: chensuyue <suyue.chen@intel.com> * docs update Signed-off-by: chensuyue <suyue.chen@intel.com> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Enable vLLM Gaudi support for LLM service based on officially habana vllm release (#137) Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * support embedding comp with llama_index Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add test script for embedding llama_inde Signed-off-by: letonghan <letong.han@intel.com> * remove conflict requirements Signed-off-by: letonghan <letong.han@intel.com> * update test script Signed-off-by: letonghan <letong.han@intel.com> * udpate Signed-off-by: letonghan <letong.han@intel.com> * update Signed-off-by: letonghan <letong.han@intel.com> * update Signed-off-by: letonghan <letong.han@intel.com> * fix ut issue Signed-off-by: letonghan <letong.han@intel.com> --------- Signed-off-by: letonghan <letong.han@intel.com> Signed-off-by: zehao-intel <zehao.huang@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> Co-authored-by: Tianyi Liu <tianyi.liu@intel.com>

…vllm release (opea-project#137) Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>

* fix stream=false doesn't work issue Signed-off-by: letonghan <letong.han@intel.com> * support embedding comp with llama_index Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add More Contents to the Table of MicroService (opea-project#141) * Add More Contents to the Table MicroService Signed-off-by: zehao-intel <zehao.huang@intel.com> * reorder Signed-off-by: zehao-intel <zehao.huang@intel.com> * Update README.md * refine structure Signed-off-by: zehao-intel <zehao.huang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix model Signed-off-by: zehao-intel <zehao.huang@intel.com> * refine table Signed-off-by: zehao-intel <zehao.huang@intel.com> * put llm to the ground Signed-off-by: zehao-intel <zehao.huang@intel.com> --------- Signed-off-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <suyue.chen@intel.com> * docs update Signed-off-by: chensuyue <suyue.chen@intel.com> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Enable vLLM Gaudi support for LLM service based on officially habana vllm release (opea-project#137) Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * support embedding comp with llama_index Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add test script for embedding llama_inde Signed-off-by: letonghan <letong.han@intel.com> * remove conflict requirements Signed-off-by: letonghan <letong.han@intel.com> * update test script Signed-off-by: letonghan <letong.han@intel.com> * udpate Signed-off-by: letonghan <letong.han@intel.com> * update Signed-off-by: letonghan <letong.han@intel.com> * update Signed-off-by: letonghan <letong.han@intel.com> * fix ut issue Signed-off-by: letonghan <letong.han@intel.com> --------- Signed-off-by: letonghan <letong.han@intel.com> Signed-off-by: zehao-intel <zehao.huang@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> Co-authored-by: Tianyi Liu <tianyi.liu@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>

…vllm release (opea-project#137) Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: gadmarkovits <gad.markovits@intel.com>

* fix stream=false doesn't work issue Signed-off-by: letonghan <letong.han@intel.com> * support embedding comp with llama_index Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add More Contents to the Table of MicroService (opea-project#141) * Add More Contents to the Table MicroService Signed-off-by: zehao-intel <zehao.huang@intel.com> * reorder Signed-off-by: zehao-intel <zehao.huang@intel.com> * Update README.md * refine structure Signed-off-by: zehao-intel <zehao.huang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix model Signed-off-by: zehao-intel <zehao.huang@intel.com> * refine table Signed-off-by: zehao-intel <zehao.huang@intel.com> * put llm to the ground Signed-off-by: zehao-intel <zehao.huang@intel.com> --------- Signed-off-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <suyue.chen@intel.com> * docs update Signed-off-by: chensuyue <suyue.chen@intel.com> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Enable vLLM Gaudi support for LLM service based on officially habana vllm release (opea-project#137) Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * support embedding comp with llama_index Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add test script for embedding llama_inde Signed-off-by: letonghan <letong.han@intel.com> * remove conflict requirements Signed-off-by: letonghan <letong.han@intel.com> * update test script Signed-off-by: letonghan <letong.han@intel.com> * udpate Signed-off-by: letonghan <letong.han@intel.com> * update Signed-off-by: letonghan <letong.han@intel.com> * update Signed-off-by: letonghan <letong.han@intel.com> * fix ut issue Signed-off-by: letonghan <letong.han@intel.com> --------- Signed-off-by: letonghan <letong.han@intel.com> Signed-off-by: zehao-intel <zehao.huang@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> Co-authored-by: Tianyi Liu <tianyi.liu@intel.com> Signed-off-by: gadmarkovits <gad.markovits@intel.com>

* enable ragas (#129) Signed-off-by: XuhuiRen <xuhui.ren@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Fix RAG performance issues (#132) * Fix RAG performance issues Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * add microservice level perf statistics (#135) * add statistics * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Add More Contents to the Table of MicroService (#141) * Add More Contents to the Table MicroService Signed-off-by: zehao-intel <zehao.huang@intel.com> * reorder Signed-off-by: zehao-intel <zehao.huang@intel.com> * Update README.md * refine structure Signed-off-by: zehao-intel <zehao.huang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix model Signed-off-by: zehao-intel <zehao.huang@intel.com> * refine table Signed-off-by: zehao-intel <zehao.huang@intel.com> * put llm to the ground Signed-off-by: zehao-intel <zehao.huang@intel.com> --------- Signed-off-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Use common security content for OPEA projects (#151) * add python coverage Signed-off-by: chensuyue <suyue.chen@intel.com> * docs update Signed-off-by: chensuyue <suyue.chen@intel.com> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Enable vLLM Gaudi support for LLM service based on officially habana vllm release (#137) Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * add knowledge graph Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * knowledge graph microservice update Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Support Dataprep Microservice with Llama Index (#154) * move file to langchain folder Signed-off-by: letonghan <letong.han@intel.com> * support dataprep with llama_index Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add e2e test script Signed-off-by: letonghan <letong.han@intel.com> * update test script name Signed-off-by: letonghan <letong.han@intel.com> --------- Signed-off-by: letonghan <letong.han@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Support Embedding Microservice with Llama Index (#150) * fix stream=false doesn't work issue Signed-off-by: letonghan <letong.han@intel.com> * support embedding comp with llama_index Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add More Contents to the Table of MicroService (#141) * Add More Contents to the Table MicroService Signed-off-by: zehao-intel <zehao.huang@intel.com> * reorder Signed-off-by: zehao-intel <zehao.huang@intel.com> * Update README.md * refine structure Signed-off-by: zehao-intel <zehao.huang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix model Signed-off-by: zehao-intel <zehao.huang@intel.com> * refine table Signed-off-by: zehao-intel <zehao.huang@intel.com> * put llm to the ground Signed-off-by: zehao-intel <zehao.huang@intel.com> --------- Signed-off-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Use common security content for OPEA projects (#151) * add python coverage Signed-off-by: chensuyue <suyue.chen@intel.com> * docs update Signed-off-by: chensuyue <suyue.chen@intel.com> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Enable vLLM Gaudi support for LLM service based on officially habana vllm release (#137) Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * support embedding comp with llama_index Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add test script for embedding llama_inde Signed-off-by: letonghan <letong.han@intel.com> * remove conflict requirements Signed-off-by: letonghan <letong.han@intel.com> * update test script Signed-off-by: letonghan <letong.han@intel.com> * udpate Signed-off-by: letonghan <letong.han@intel.com> * update Signed-off-by: letonghan <letong.han@intel.com> * update Signed-off-by: letonghan <letong.han@intel.com> * fix ut issue Signed-off-by: letonghan <letong.han@intel.com> --------- Signed-off-by: letonghan <letong.han@intel.com> Signed-off-by: zehao-intel <zehao.huang@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> Co-authored-by: Tianyi Liu <tianyi.liu@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Support Ollama microservice (#142) * Add Ollama Support Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Fix dataprep microservice path issue (#163) Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * update CI to support dataprep_redis path level change (#155) Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: letonghan <letong.han@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Add Gateway for Translation (#169) * add translation gateway Signed-off-by: zehao-intel <zehao.huang@intel.com> * fix import Signed-off-by: zehao-intel <zehao.huang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Update LLM readme (#172) * Update LLM readme Signed-off-by: lvliang-intel <liang1.lv@intel.com> * update readme Signed-off-by: lvliang-intel <liang1.lv@intel.com> * update tgi readme Signed-off-by: lvliang-intel <liang1.lv@intel.com> * rollback requirements.txt Signed-off-by: lvliang-intel <liang1.lv@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: lvliang-intel <liang1.lv@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * add milvus microservice (#158) * Use common security content for OPEA projects (#151) * add python coverage Signed-off-by: chensuyue <suyue.chen@intel.com> * docs update Signed-off-by: chensuyue <suyue.chen@intel.com> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: jinjunzh <jasper.zhu@intel.com> * add milvus microservice Signed-off-by: jinjunzh <jasper.zhu@intel.com> * fix the typo Signed-off-by: jinjunzh <jasper.zhu@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: jinjunzh <jasper.zhu@intel.com> --------- Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: jinjunzh <jasper.zhu@intel.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * enable python coverage (#149) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Add Ray version for multi file process (#119) * add ray version document to redis Signed-off-by: Chendi Xue <chendi.xue@intel.com> * update test Signed-off-by: Chendi Xue <chendi.xue@intel.com> * Add test Signed-off-by: Chendi Xue <chendi.xue@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add TIMEOUT in container environment and return status Signed-off-by: Chendi Xue <chendi.xue@intel.com> * rebase on new folder layout Signed-off-by: Chendi Xue <chendi.xue@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chendi Xue <chendi.xue@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Add codecov (#178) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Rename lm-eval folder to utils/lm-eval (#179) Signed-off-by: changwangss <chang1.wang@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Support rerank and retrieval of RAG OPT (#164) * supported bce model for rerank. Signed-off-by: Xinyu Ye <xinyu.ye@intel.com> * change folder Signed-off-by: Xinyu Ye <xinyu.ye@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * change path in test file. Signed-off-by: Xinyu Ye <xinyu.ye@intel.com> --------- Signed-off-by: Xinyu Ye <xinyu.ye@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Support vLLM XFT LLM microservice (#174) * Support vLLM XFT serving Signed-off-by: lvliang-intel <liang1.lv@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix access vllm issue Signed-off-by: lvliang-intel <liang1.lv@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add permission for run.sh Signed-off-by: lvliang-intel <liang1.lv@intel.com> * add readme Signed-off-by: lvliang-intel <liang1.lv@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix proxy issue Signed-off-by: lvliang-intel <liang1.lv@intel.com> --------- Signed-off-by: lvliang-intel <liang1.lv@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Update Dataprep Microservice README (#173) * update dataprep readme Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: letonghan <letong.han@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * fixed milvus port conflict issues during deployment (#183) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixed milvus port conflict issues during deployment * align port for unified retrieval microservice --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * remove Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * remove hard address Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update readme Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * add example data and ingestion Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * fix typ Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix dataprep timeout issue (#203) Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Add a new embedding MosecEmbedding (#182) * Add a new embedding MosecEmbedding. Signed-off-by: Jincheng Miao <jincheng.miao@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jincheng Miao <jincheng.miao@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * expand timeout for microservice test (#208) Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * fix typo Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * fix requirement Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: XuhuiRen <xuhui.ren@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: zehao-intel <zehao.huang@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: tianyil1 <tianyi.liu@intel.com> Signed-off-by: letonghan <letong.han@intel.com> Signed-off-by: jinjunzh <jasper.zhu@intel.com> Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> Signed-off-by: Chendi Xue <chendi.xue@intel.com> Signed-off-by: changwangss <chang1.wang@intel.com> Signed-off-by: Xinyu Ye <xinyu.ye@intel.com> Signed-off-by: Jincheng Miao <jincheng.miao@intel.com> Co-authored-by: XuhuiRen <44249229+XuhuiRen@users.noreply.github.com> Co-authored-by: lvliang-intel <liang1.lv@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> Co-authored-by: Tianyi Liu <tianyi.liu@intel.com> Co-authored-by: Letong Han <106566639+letonghan@users.noreply.github.com> Co-authored-by: jasperzhu <jasper.zhu@intel.com> Co-authored-by: Chendi.Xue <chendi.xue@intel.com> Co-authored-by: Sun, Xuehao <xuehao.sun@intel.com> Co-authored-by: Wang, Chang <491521017@qq.com> Co-authored-by: XinyuYe-Intel <xinyu.ye@intel.com> Co-authored-by: Jincheng Miao <jincheng.miao@intel.com>

…vllm release (opea-project#137) Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com>

* fix stream=false doesn't work issue Signed-off-by: letonghan <letong.han@intel.com> * support embedding comp with llama_index Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add More Contents to the Table of MicroService (opea-project#141) * Add More Contents to the Table MicroService Signed-off-by: zehao-intel <zehao.huang@intel.com> * reorder Signed-off-by: zehao-intel <zehao.huang@intel.com> * Update README.md * refine structure Signed-off-by: zehao-intel <zehao.huang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix model Signed-off-by: zehao-intel <zehao.huang@intel.com> * refine table Signed-off-by: zehao-intel <zehao.huang@intel.com> * put llm to the ground Signed-off-by: zehao-intel <zehao.huang@intel.com> --------- Signed-off-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <suyue.chen@intel.com> * docs update Signed-off-by: chensuyue <suyue.chen@intel.com> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Enable vLLM Gaudi support for LLM service based on officially habana vllm release (opea-project#137) Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * support embedding comp with llama_index Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add test script for embedding llama_inde Signed-off-by: letonghan <letong.han@intel.com> * remove conflict requirements Signed-off-by: letonghan <letong.han@intel.com> * update test script Signed-off-by: letonghan <letong.han@intel.com> * udpate Signed-off-by: letonghan <letong.han@intel.com> * update Signed-off-by: letonghan <letong.han@intel.com> * update Signed-off-by: letonghan <letong.han@intel.com> * fix ut issue Signed-off-by: letonghan <letong.han@intel.com> --------- Signed-off-by: letonghan <letong.han@intel.com> Signed-off-by: zehao-intel <zehao.huang@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> Co-authored-by: Tianyi Liu <tianyi.liu@intel.com> Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com>

* enable ragas (opea-project#129) Signed-off-by: XuhuiRen <xuhui.ren@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Fix RAG performance issues (opea-project#132) * Fix RAG performance issues Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * add microservice level perf statistics (opea-project#135) * add statistics * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Add More Contents to the Table of MicroService (opea-project#141) * Add More Contents to the Table MicroService Signed-off-by: zehao-intel <zehao.huang@intel.com> * reorder Signed-off-by: zehao-intel <zehao.huang@intel.com> * Update README.md * refine structure Signed-off-by: zehao-intel <zehao.huang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix model Signed-off-by: zehao-intel <zehao.huang@intel.com> * refine table Signed-off-by: zehao-intel <zehao.huang@intel.com> * put llm to the ground Signed-off-by: zehao-intel <zehao.huang@intel.com> --------- Signed-off-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <suyue.chen@intel.com> * docs update Signed-off-by: chensuyue <suyue.chen@intel.com> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Enable vLLM Gaudi support for LLM service based on officially habana vllm release (opea-project#137) Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * add knowledge graph Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * knowledge graph microservice update Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Support Dataprep Microservice with Llama Index (opea-project#154) * move file to langchain folder Signed-off-by: letonghan <letong.han@intel.com> * support dataprep with llama_index Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add e2e test script Signed-off-by: letonghan <letong.han@intel.com> * update test script name Signed-off-by: letonghan <letong.han@intel.com> --------- Signed-off-by: letonghan <letong.han@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Support Embedding Microservice with Llama Index (opea-project#150) * fix stream=false doesn't work issue Signed-off-by: letonghan <letong.han@intel.com> * support embedding comp with llama_index Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add More Contents to the Table of MicroService (opea-project#141) * Add More Contents to the Table MicroService Signed-off-by: zehao-intel <zehao.huang@intel.com> * reorder Signed-off-by: zehao-intel <zehao.huang@intel.com> * Update README.md * refine structure Signed-off-by: zehao-intel <zehao.huang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix model Signed-off-by: zehao-intel <zehao.huang@intel.com> * refine table Signed-off-by: zehao-intel <zehao.huang@intel.com> * put llm to the ground Signed-off-by: zehao-intel <zehao.huang@intel.com> --------- Signed-off-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <suyue.chen@intel.com> * docs update Signed-off-by: chensuyue <suyue.chen@intel.com> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Enable vLLM Gaudi support for LLM service based on officially habana vllm release (opea-project#137) Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * support embedding comp with llama_index Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add test script for embedding llama_inde Signed-off-by: letonghan <letong.han@intel.com> * remove conflict requirements Signed-off-by: letonghan <letong.han@intel.com> * update test script Signed-off-by: letonghan <letong.han@intel.com> * udpate Signed-off-by: letonghan <letong.han@intel.com> * update Signed-off-by: letonghan <letong.han@intel.com> * update Signed-off-by: letonghan <letong.han@intel.com> * fix ut issue Signed-off-by: letonghan <letong.han@intel.com> --------- Signed-off-by: letonghan <letong.han@intel.com> Signed-off-by: zehao-intel <zehao.huang@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> Co-authored-by: Tianyi Liu <tianyi.liu@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Support Ollama microservice (opea-project#142) * Add Ollama Support Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Fix dataprep microservice path issue (opea-project#163) Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * update CI to support dataprep_redis path level change (opea-project#155) Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: letonghan <letong.han@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Add Gateway for Translation (opea-project#169) * add translation gateway Signed-off-by: zehao-intel <zehao.huang@intel.com> * fix import Signed-off-by: zehao-intel <zehao.huang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Update LLM readme (opea-project#172) * Update LLM readme Signed-off-by: lvliang-intel <liang1.lv@intel.com> * update readme Signed-off-by: lvliang-intel <liang1.lv@intel.com> * update tgi readme Signed-off-by: lvliang-intel <liang1.lv@intel.com> * rollback requirements.txt Signed-off-by: lvliang-intel <liang1.lv@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: lvliang-intel <liang1.lv@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * add milvus microservice (opea-project#158) * Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <suyue.chen@intel.com> * docs update Signed-off-by: chensuyue <suyue.chen@intel.com> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: jinjunzh <jasper.zhu@intel.com> * add milvus microservice Signed-off-by: jinjunzh <jasper.zhu@intel.com> * fix the typo Signed-off-by: jinjunzh <jasper.zhu@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: jinjunzh <jasper.zhu@intel.com> --------- Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: jinjunzh <jasper.zhu@intel.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * enable python coverage (opea-project#149) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Add Ray version for multi file process (opea-project#119) * add ray version document to redis Signed-off-by: Chendi Xue <chendi.xue@intel.com> * update test Signed-off-by: Chendi Xue <chendi.xue@intel.com> * Add test Signed-off-by: Chendi Xue <chendi.xue@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add TIMEOUT in container environment and return status Signed-off-by: Chendi Xue <chendi.xue@intel.com> * rebase on new folder layout Signed-off-by: Chendi Xue <chendi.xue@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chendi Xue <chendi.xue@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Add codecov (opea-project#178) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Rename lm-eval folder to utils/lm-eval (opea-project#179) Signed-off-by: changwangss <chang1.wang@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Support rerank and retrieval of RAG OPT (opea-project#164) * supported bce model for rerank. Signed-off-by: Xinyu Ye <xinyu.ye@intel.com> * change folder Signed-off-by: Xinyu Ye <xinyu.ye@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * change path in test file. Signed-off-by: Xinyu Ye <xinyu.ye@intel.com> --------- Signed-off-by: Xinyu Ye <xinyu.ye@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Support vLLM XFT LLM microservice (opea-project#174) * Support vLLM XFT serving Signed-off-by: lvliang-intel <liang1.lv@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix access vllm issue Signed-off-by: lvliang-intel <liang1.lv@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add permission for run.sh Signed-off-by: lvliang-intel <liang1.lv@intel.com> * add readme Signed-off-by: lvliang-intel <liang1.lv@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix proxy issue Signed-off-by: lvliang-intel <liang1.lv@intel.com> --------- Signed-off-by: lvliang-intel <liang1.lv@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Update Dataprep Microservice README (opea-project#173) * update dataprep readme Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: letonghan <letong.han@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * fixed milvus port conflict issues during deployment (opea-project#183) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixed milvus port conflict issues during deployment * align port for unified retrieval microservice --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * remove Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * remove hard address Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update readme Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * add example data and ingestion Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * fix typ Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix dataprep timeout issue (opea-project#203) Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Add a new embedding MosecEmbedding (opea-project#182) * Add a new embedding MosecEmbedding. Signed-off-by: Jincheng Miao <jincheng.miao@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jincheng Miao <jincheng.miao@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * expand timeout for microservice test (opea-project#208) Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * fix typo Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * fix requirement Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: XuhuiRen <xuhui.ren@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: zehao-intel <zehao.huang@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: tianyil1 <tianyi.liu@intel.com> Signed-off-by: letonghan <letong.han@intel.com> Signed-off-by: jinjunzh <jasper.zhu@intel.com> Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> Signed-off-by: Chendi Xue <chendi.xue@intel.com> Signed-off-by: changwangss <chang1.wang@intel.com> Signed-off-by: Xinyu Ye <xinyu.ye@intel.com> Signed-off-by: Jincheng Miao <jincheng.miao@intel.com> Co-authored-by: XuhuiRen <44249229+XuhuiRen@users.noreply.github.com> Co-authored-by: lvliang-intel <liang1.lv@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> Co-authored-by: Tianyi Liu <tianyi.liu@intel.com> Co-authored-by: Letong Han <106566639+letonghan@users.noreply.github.com> Co-authored-by: jasperzhu <jasper.zhu@intel.com> Co-authored-by: Chendi.Xue <chendi.xue@intel.com> Co-authored-by: Sun, Xuehao <xuehao.sun@intel.com> Co-authored-by: Wang, Chang <491521017@qq.com> Co-authored-by: XinyuYe-Intel <xinyu.ye@intel.com> Co-authored-by: Jincheng Miao <jincheng.miao@intel.com> Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com>

* enable ragas (opea-project#129) Signed-off-by: XuhuiRen <xuhui.ren@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Fix RAG performance issues (opea-project#132) * Fix RAG performance issues Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * add microservice level perf statistics (opea-project#135) * add statistics * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Add More Contents to the Table of MicroService (opea-project#141) * Add More Contents to the Table MicroService Signed-off-by: zehao-intel <zehao.huang@intel.com> * reorder Signed-off-by: zehao-intel <zehao.huang@intel.com> * Update README.md * refine structure Signed-off-by: zehao-intel <zehao.huang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix model Signed-off-by: zehao-intel <zehao.huang@intel.com> * refine table Signed-off-by: zehao-intel <zehao.huang@intel.com> * put llm to the ground Signed-off-by: zehao-intel <zehao.huang@intel.com> --------- Signed-off-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <suyue.chen@intel.com> * docs update Signed-off-by: chensuyue <suyue.chen@intel.com> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Enable vLLM Gaudi support for LLM service based on officially habana vllm release (opea-project#137) Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * add knowledge graph Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * knowledge graph microservice update Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Support Dataprep Microservice with Llama Index (opea-project#154) * move file to langchain folder Signed-off-by: letonghan <letong.han@intel.com> * support dataprep with llama_index Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add e2e test script Signed-off-by: letonghan <letong.han@intel.com> * update test script name Signed-off-by: letonghan <letong.han@intel.com> --------- Signed-off-by: letonghan <letong.han@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Support Embedding Microservice with Llama Index (opea-project#150) * fix stream=false doesn't work issue Signed-off-by: letonghan <letong.han@intel.com> * support embedding comp with llama_index Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add More Contents to the Table of MicroService (opea-project#141) * Add More Contents to the Table MicroService Signed-off-by: zehao-intel <zehao.huang@intel.com> * reorder Signed-off-by: zehao-intel <zehao.huang@intel.com> * Update README.md * refine structure Signed-off-by: zehao-intel <zehao.huang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix model Signed-off-by: zehao-intel <zehao.huang@intel.com> * refine table Signed-off-by: zehao-intel <zehao.huang@intel.com> * put llm to the ground Signed-off-by: zehao-intel <zehao.huang@intel.com> --------- Signed-off-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <suyue.chen@intel.com> * docs update Signed-off-by: chensuyue <suyue.chen@intel.com> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Enable vLLM Gaudi support for LLM service based on officially habana vllm release (opea-project#137) Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * support embedding comp with llama_index Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add test script for embedding llama_inde Signed-off-by: letonghan <letong.han@intel.com> * remove conflict requirements Signed-off-by: letonghan <letong.han@intel.com> * update test script Signed-off-by: letonghan <letong.han@intel.com> * udpate Signed-off-by: letonghan <letong.han@intel.com> * update Signed-off-by: letonghan <letong.han@intel.com> * update Signed-off-by: letonghan <letong.han@intel.com> * fix ut issue Signed-off-by: letonghan <letong.han@intel.com> --------- Signed-off-by: letonghan <letong.han@intel.com> Signed-off-by: zehao-intel <zehao.huang@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> Co-authored-by: Tianyi Liu <tianyi.liu@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Support Ollama microservice (opea-project#142) * Add Ollama Support Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Fix dataprep microservice path issue (opea-project#163) Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * update CI to support dataprep_redis path level change (opea-project#155) Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: letonghan <letong.han@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Add Gateway for Translation (opea-project#169) * add translation gateway Signed-off-by: zehao-intel <zehao.huang@intel.com> * fix import Signed-off-by: zehao-intel <zehao.huang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Update LLM readme (opea-project#172) * Update LLM readme Signed-off-by: lvliang-intel <liang1.lv@intel.com> * update readme Signed-off-by: lvliang-intel <liang1.lv@intel.com> * update tgi readme Signed-off-by: lvliang-intel <liang1.lv@intel.com> * rollback requirements.txt Signed-off-by: lvliang-intel <liang1.lv@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: lvliang-intel <liang1.lv@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * add milvus microservice (opea-project#158) * Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <suyue.chen@intel.com> * docs update Signed-off-by: chensuyue <suyue.chen@intel.com> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: jinjunzh <jasper.zhu@intel.com> * add milvus microservice Signed-off-by: jinjunzh <jasper.zhu@intel.com> * fix the typo Signed-off-by: jinjunzh <jasper.zhu@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: jinjunzh <jasper.zhu@intel.com> --------- Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: jinjunzh <jasper.zhu@intel.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * enable python coverage (opea-project#149) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Add Ray version for multi file process (opea-project#119) * add ray version document to redis Signed-off-by: Chendi Xue <chendi.xue@intel.com> * update test Signed-off-by: Chendi Xue <chendi.xue@intel.com> * Add test Signed-off-by: Chendi Xue <chendi.xue@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add TIMEOUT in container environment and return status Signed-off-by: Chendi Xue <chendi.xue@intel.com> * rebase on new folder layout Signed-off-by: Chendi Xue <chendi.xue@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chendi Xue <chendi.xue@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Add codecov (opea-project#178) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Rename lm-eval folder to utils/lm-eval (opea-project#179) Signed-off-by: changwangss <chang1.wang@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Support rerank and retrieval of RAG OPT (opea-project#164) * supported bce model for rerank. Signed-off-by: Xinyu Ye <xinyu.ye@intel.com> * change folder Signed-off-by: Xinyu Ye <xinyu.ye@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * change path in test file. Signed-off-by: Xinyu Ye <xinyu.ye@intel.com> --------- Signed-off-by: Xinyu Ye <xinyu.ye@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Support vLLM XFT LLM microservice (opea-project#174) * Support vLLM XFT serving Signed-off-by: lvliang-intel <liang1.lv@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix access vllm issue Signed-off-by: lvliang-intel <liang1.lv@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add permission for run.sh Signed-off-by: lvliang-intel <liang1.lv@intel.com> * add readme Signed-off-by: lvliang-intel <liang1.lv@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix proxy issue Signed-off-by: lvliang-intel <liang1.lv@intel.com> --------- Signed-off-by: lvliang-intel <liang1.lv@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Update Dataprep Microservice README (opea-project#173) * update dataprep readme Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: letonghan <letong.han@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * fixed milvus port conflict issues during deployment (opea-project#183) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixed milvus port conflict issues during deployment * align port for unified retrieval microservice --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * remove Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * remove hard address Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update readme Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * add example data and ingestion Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * fix typ Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix dataprep timeout issue (opea-project#203) Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Add a new embedding MosecEmbedding (opea-project#182) * Add a new embedding MosecEmbedding. Signed-off-by: Jincheng Miao <jincheng.miao@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jincheng Miao <jincheng.miao@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * expand timeout for microservice test (opea-project#208) Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * fix typo Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * fix requirement Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: XuhuiRen <xuhui.ren@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: zehao-intel <zehao.huang@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: tianyil1 <tianyi.liu@intel.com> Signed-off-by: letonghan <letong.han@intel.com> Signed-off-by: jinjunzh <jasper.zhu@intel.com> Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> Signed-off-by: Chendi Xue <chendi.xue@intel.com> Signed-off-by: changwangss <chang1.wang@intel.com> Signed-off-by: Xinyu Ye <xinyu.ye@intel.com> Signed-off-by: Jincheng Miao <jincheng.miao@intel.com> Co-authored-by: XuhuiRen <44249229+XuhuiRen@users.noreply.github.com> Co-authored-by: lvliang-intel <liang1.lv@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> Co-authored-by: Tianyi Liu <tianyi.liu@intel.com> Co-authored-by: Letong Han <106566639+letonghan@users.noreply.github.com> Co-authored-by: jasperzhu <jasper.zhu@intel.com> Co-authored-by: Chendi.Xue <chendi.xue@intel.com> Co-authored-by: Sun, Xuehao <xuehao.sun@intel.com> Co-authored-by: Wang, Chang <491521017@qq.com> Co-authored-by: XinyuYe-Intel <xinyu.ye@intel.com> Co-authored-by: Jincheng Miao <jincheng.miao@intel.com> Signed-off-by: Yogesh Pandey <yogesh.pandey@intel.com>

* enable ragas (opea-project#129) Signed-off-by: XuhuiRen <xuhui.ren@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Fix RAG performance issues (opea-project#132) * Fix RAG performance issues Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * add microservice level perf statistics (opea-project#135) * add statistics * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Add More Contents to the Table of MicroService (opea-project#141) * Add More Contents to the Table MicroService Signed-off-by: zehao-intel <zehao.huang@intel.com> * reorder Signed-off-by: zehao-intel <zehao.huang@intel.com> * Update README.md * refine structure Signed-off-by: zehao-intel <zehao.huang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix model Signed-off-by: zehao-intel <zehao.huang@intel.com> * refine table Signed-off-by: zehao-intel <zehao.huang@intel.com> * put llm to the ground Signed-off-by: zehao-intel <zehao.huang@intel.com> --------- Signed-off-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <suyue.chen@intel.com> * docs update Signed-off-by: chensuyue <suyue.chen@intel.com> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Enable vLLM Gaudi support for LLM service based on officially habana vllm release (opea-project#137) Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * add knowledge graph Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * knowledge graph microservice update Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Support Dataprep Microservice with Llama Index (opea-project#154) * move file to langchain folder Signed-off-by: letonghan <letong.han@intel.com> * support dataprep with llama_index Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add e2e test script Signed-off-by: letonghan <letong.han@intel.com> * update test script name Signed-off-by: letonghan <letong.han@intel.com> --------- Signed-off-by: letonghan <letong.han@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Support Embedding Microservice with Llama Index (opea-project#150) * fix stream=false doesn't work issue Signed-off-by: letonghan <letong.han@intel.com> * support embedding comp with llama_index Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add More Contents to the Table of MicroService (opea-project#141) * Add More Contents to the Table MicroService Signed-off-by: zehao-intel <zehao.huang@intel.com> * reorder Signed-off-by: zehao-intel <zehao.huang@intel.com> * Update README.md * refine structure Signed-off-by: zehao-intel <zehao.huang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix model Signed-off-by: zehao-intel <zehao.huang@intel.com> * refine table Signed-off-by: zehao-intel <zehao.huang@intel.com> * put llm to the ground Signed-off-by: zehao-intel <zehao.huang@intel.com> --------- Signed-off-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <suyue.chen@intel.com> * docs update Signed-off-by: chensuyue <suyue.chen@intel.com> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Enable vLLM Gaudi support for LLM service based on officially habana vllm release (opea-project#137) Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * support embedding comp with llama_index Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add test script for embedding llama_inde Signed-off-by: letonghan <letong.han@intel.com> * remove conflict requirements Signed-off-by: letonghan <letong.han@intel.com> * update test script Signed-off-by: letonghan <letong.han@intel.com> * udpate Signed-off-by: letonghan <letong.han@intel.com> * update Signed-off-by: letonghan <letong.han@intel.com> * update Signed-off-by: letonghan <letong.han@intel.com> * fix ut issue Signed-off-by: letonghan <letong.han@intel.com> --------- Signed-off-by: letonghan <letong.han@intel.com> Signed-off-by: zehao-intel <zehao.huang@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: tianyil1 <tianyi.liu@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> Co-authored-by: Tianyi Liu <tianyi.liu@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Support Ollama microservice (opea-project#142) * Add Ollama Support Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Fix dataprep microservice path issue (opea-project#163) Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * update CI to support dataprep_redis path level change (opea-project#155) Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: letonghan <letong.han@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Add Gateway for Translation (opea-project#169) * add translation gateway Signed-off-by: zehao-intel <zehao.huang@intel.com> * fix import Signed-off-by: zehao-intel <zehao.huang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Update LLM readme (opea-project#172) * Update LLM readme Signed-off-by: lvliang-intel <liang1.lv@intel.com> * update readme Signed-off-by: lvliang-intel <liang1.lv@intel.com> * update tgi readme Signed-off-by: lvliang-intel <liang1.lv@intel.com> * rollback requirements.txt Signed-off-by: lvliang-intel <liang1.lv@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: lvliang-intel <liang1.lv@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * add milvus microservice (opea-project#158) * Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <suyue.chen@intel.com> * docs update Signed-off-by: chensuyue <suyue.chen@intel.com> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: jinjunzh <jasper.zhu@intel.com> * add milvus microservice Signed-off-by: jinjunzh <jasper.zhu@intel.com> * fix the typo Signed-off-by: jinjunzh <jasper.zhu@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: jinjunzh <jasper.zhu@intel.com> --------- Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: jinjunzh <jasper.zhu@intel.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * enable python coverage (opea-project#149) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Add Ray version for multi file process (opea-project#119) * add ray version document to redis Signed-off-by: Chendi Xue <chendi.xue@intel.com> * update test Signed-off-by: Chendi Xue <chendi.xue@intel.com> * Add test Signed-off-by: Chendi Xue <chendi.xue@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add TIMEOUT in container environment and return status Signed-off-by: Chendi Xue <chendi.xue@intel.com> * rebase on new folder layout Signed-off-by: Chendi Xue <chendi.xue@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chendi Xue <chendi.xue@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Add codecov (opea-project#178) Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Rename lm-eval folder to utils/lm-eval (opea-project#179) Signed-off-by: changwangss <chang1.wang@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Support rerank and retrieval of RAG OPT (opea-project#164) * supported bce model for rerank. Signed-off-by: Xinyu Ye <xinyu.ye@intel.com> * change folder Signed-off-by: Xinyu Ye <xinyu.ye@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * change path in test file. Signed-off-by: Xinyu Ye <xinyu.ye@intel.com> --------- Signed-off-by: Xinyu Ye <xinyu.ye@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Support vLLM XFT LLM microservice (opea-project#174) * Support vLLM XFT serving Signed-off-by: lvliang-intel <liang1.lv@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix access vllm issue Signed-off-by: lvliang-intel <liang1.lv@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add permission for run.sh Signed-off-by: lvliang-intel <liang1.lv@intel.com> * add readme Signed-off-by: lvliang-intel <liang1.lv@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix proxy issue Signed-off-by: lvliang-intel <liang1.lv@intel.com> --------- Signed-off-by: lvliang-intel <liang1.lv@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Update Dataprep Microservice README (opea-project#173) * update dataprep readme Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: letonghan <letong.han@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * fixed milvus port conflict issues during deployment (opea-project#183) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixed milvus port conflict issues during deployment * align port for unified retrieval microservice --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * remove Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * remove hard address Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update readme Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * add example data and ingestion Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * fix typ Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix dataprep timeout issue (opea-project#203) Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * Add a new embedding MosecEmbedding (opea-project#182) * Add a new embedding MosecEmbedding. Signed-off-by: Jincheng Miao <jincheng.miao@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jincheng Miao <jincheng.miao@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * expand timeout for microservice test (opea-project#208) Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * fix typo Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * fix requirement Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: XuhuiRen <xuhui.ren@intel.com> Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> Signed-off-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: zehao-intel <zehao.huang@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: tianyil1 <tianyi.liu@intel.com> Signed-off-by: letonghan <letong.han@intel.com> Signed-off-by: jinjunzh <jasper.zhu@intel.com> Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com> Signed-off-by: Chendi Xue <chendi.xue@intel.com> Signed-off-by: changwangss <chang1.wang@intel.com> Signed-off-by: Xinyu Ye <xinyu.ye@intel.com> Signed-off-by: Jincheng Miao <jincheng.miao@intel.com> Co-authored-by: XuhuiRen <44249229+XuhuiRen@users.noreply.github.com> Co-authored-by: lvliang-intel <liang1.lv@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: zehao-intel <zehao.huang@intel.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> Co-authored-by: Tianyi Liu <tianyi.liu@intel.com> Co-authored-by: Letong Han <106566639+letonghan@users.noreply.github.com> Co-authored-by: jasperzhu <jasper.zhu@intel.com> Co-authored-by: Chendi.Xue <chendi.xue@intel.com> Co-authored-by: Sun, Xuehao <xuehao.sun@intel.com> Co-authored-by: Wang, Chang <491521017@qq.com> Co-authored-by: XinyuYe-Intel <xinyu.ye@intel.com> Co-authored-by: Jincheng Miao <jincheng.miao@intel.com> Signed-off-by: Daniel Whitenack <whitenack.daniel@gmail.com>

tianyil1 mentioned this pull request Jun 6, 2024

Enable vLLM Gaudi Support for LLM Service #126

Closed

1 task

tianyil1 force-pushed the vllm-gaudi branch 3 times, most recently from 8ce8c25 to d6b3cf6 Compare June 6, 2024 08:12

kevinintel requested a review from lvliang-intel June 7, 2024 08:03

tianyil1 force-pushed the vllm-gaudi branch 2 times, most recently from 1453729 to 3ebdf73 Compare June 11, 2024 06:38

tianyil1 force-pushed the vllm-gaudi branch from fea2bb1 to 1a6ff29 Compare June 11, 2024 11:45

lvliang-intel approved these changes Jun 11, 2024

View reviewed changes

tianyil1 and others added 8 commits June 12, 2024 11:11

added the vllm gaudi build docker files

ae3b274

Signed-off-by: tianyil1 <tianyi.liu@intel.com>

refine the docker and add the launch vllm script

23f28fe

Signed-off-by: tianyil1 <tianyi.liu@intel.com>

refine the vllm readme for gaudi support

5b3202b

Signed-off-by: tianyil1 <tianyi.liu@intel.com>

updated the gaudi version with the official version

2a7c729

Signed-off-by: tianyil1 <tianyi.liu@intel.com>

add the host ip to the launch script

a331e4e

Signed-off-by: tianyil1 <tianyi.liu@intel.com>

add the vllm gaudi into the main readme

f08c917

Signed-off-by: tianyil1 <tianyi.liu@intel.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

ef3eb8f

for more information, see https://pre-commit.ci

refine the readme with the default model 'Intel/neural-chat-7b-v3-3'

6df3912

Signed-off-by: tianyil1 <tianyi.liu@intel.com>

tianyil1 force-pushed the vllm-gaudi branch from 1a6ff29 to 6df3912 Compare June 12, 2024 03:14

ftian1 approved these changes Jun 12, 2024

View reviewed changes

ftian1 merged commit 0dedc28 into opea-project:main Jun 12, 2024
6 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable vLLM Gaudi support for LLM service based on officially habana vllm release #137

Enable vLLM Gaudi support for LLM service based on officially habana vllm release #137

tianyil1 commented Jun 6, 2024

tianyil1 commented Jun 6, 2024

tianyil1 commented Jun 11, 2024

Enable vLLM Gaudi support for LLM service based on officially habana vllm release #137

Enable vLLM Gaudi support for LLM service based on officially habana vllm release #137

Conversation

tianyil1 commented Jun 6, 2024

Description

Issues

Type of change

Dependencies

Tests

tianyil1 commented Jun 6, 2024

tianyil1 commented Jun 11, 2024