Add vLLM on Ray microservice #245
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
for more information, see https://pre-commit.ci
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
|
Please add the e2e test like this one, https://github.com/opea-project/GenAIComps/blob/main/tests/test_llm_summarization_tgi.sh |
KepingYan
reviewed
Jun 27, 2024
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
for more information, see https://pre-commit.ci
Signed-off-by: Abolfazl Shahbazi <abolfazl.shahbazi@intel.com>
* refine vllm instruction --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* remove ingest in retriever and support empty vector db Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update readme Signed-off-by: letonghan <letong.han@intel.com> * refine test script Signed-off-by: letonghan <letong.han@intel.com> * update test script Signed-off-by: letonghan <letong.han@intel.com> --------- Signed-off-by: letonghan <letong.han@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * DataPrep extract info from table in the docs (opea-project#146) * Add microservice for table extraction Signed-off-by: Liangyx2 <yuxiang.liang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix Signed-off-by: Liangyx2 <yuxiang.liang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update license copyright Signed-off-by: Liangyx2 <yuxiang.liang@intel.com> * DataPrep extract info from table in the docs Signed-off-by: Liangyx2 <yuxiang.liang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * refine Signed-off-by: Liangyx2 <yuxiang.liang@intel.com> * refine Signed-off-by: Liangyx2 <yuxiang.liang@intel.com> * Update prepare_doc_redis.py * Update prepare_doc_qdrant.py * Update prepare_doc_milvus.py --------- Signed-off-by: Liangyx2 <yuxiang.liang@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> Co-authored-by: XuhuiRen <44249229+XuhuiRen@users.noreply.github.com> Signed-off-by: jinjunzh <jasper.zhu@intel.com> * Remove sensitive info logs (opea-project#251) Signed-off-by: Chendi Xue <chendi.xue@intel.com> Signed-off-by: jinjunzh <jasper.zhu@intel.com> * Added support for extracting info from image in the docs (opea-project#120) Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com> Signed-off-by: jinjunzh <jasper.zhu@intel.com> * enhance statistics ut coverage (opea-project#252) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: jinjunzh <jasper.zhu@intel.com> * bump version (opea-project#253) Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: jinjunzh <jasper.zhu@intel.com> * support file upload feature for milvus service Signed-off-by: jinjunzh <jasper.zhu@intel.com> * update embedding with MOSEC_EMBEDDING_ENDPOINT Signed-off-by: jinjunzh <jasper.zhu@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: jinjunzh <jasper.zhu@intel.com> * support file upload feature for milvus service Signed-off-by: jinjunzh <jasper.zhu@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: jinjunzh <jasper.zhu@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: jinjunzh <jasper.zhu@intel.com> * fix duplicate ci test (opea-project#256) * fix duplicate test Signed-off-by: chensuyue <suyue.chen@intel.com> * for test only Signed-off-by: chensuyue <suyue.chen@intel.com> * Revert "for test only" This reverts commit a7718aa. --------- Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: jinjunzh <jasper.zhu@intel.com> * DataPrep extract info from table in the docs (opea-project#146) * Add microservice for table extraction Signed-off-by: Liangyx2 <yuxiang.liang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix Signed-off-by: Liangyx2 <yuxiang.liang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update license copyright Signed-off-by: Liangyx2 <yuxiang.liang@intel.com> * DataPrep extract info from table in the docs Signed-off-by: Liangyx2 <yuxiang.liang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * refine Signed-off-by: Liangyx2 <yuxiang.liang@intel.com> * refine Signed-off-by: Liangyx2 <yuxiang.liang@intel.com> * Update prepare_doc_redis.py * Update prepare_doc_qdrant.py * Update prepare_doc_milvus.py --------- Signed-off-by: Liangyx2 <yuxiang.liang@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> Co-authored-by: XuhuiRen <44249229+XuhuiRen@users.noreply.github.com> --------- Signed-off-by: Liangyx2 <yuxiang.liang@intel.com> Signed-off-by: jinjunzh <jasper.zhu@intel.com> Signed-off-by: Chendi Xue <chendi.xue@intel.com> Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com> Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Liangyx2 <yuxiang.liang@intel.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> Co-authored-by: XuhuiRen <44249229+XuhuiRen@users.noreply.github.com> Co-authored-by: Chendi.Xue <chendi.xue@intel.com> Co-authored-by: XinyuYe-Intel <xinyu.ye@intel.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com>
* support get_file for dataprep Signed-off-by: letonghan <letong.han@intel.com> * add e2e test for get_file api Signed-off-by: letonghan <letong.han@intel.com> * support get_file in ray & llama_index Signed-off-by: letonghan <letong.han@intel.com> * udpate README Signed-off-by: letonghan <letong.han@intel.com> * update e2e tests for ray & llama_index Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * modify port for langchain e2e test Signed-off-by: letonghan <letong.han@intel.com> * modify ports of ray test Signed-off-by: letonghan <letong.han@intel.com> * mofidy container name and ports Signed-off-by: letonghan <letong.han@intel.com> * update Signed-off-by: letonghan <letong.han@intel.com> * fix depencendy error of ray & unify container names Signed-off-by: letonghan <letong.han@intel.com> * modify port of 8001 Signed-off-by: letonghan <letong.han@intel.com> * modify conflict port Signed-off-by: letonghan <letong.han@intel.com> * Add container log print Signed-off-by: Chendi Xue <chendi.xue@intel.com> * Add LOG_PATH define Signed-off-by: Chendi Xue <chendi.xue@intel.com> * support get link & delete file/link Signed-off-by: letonghan <letong.han@intel.com> * update readme and test scripts Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix test issue Signed-off-by: letonghan <letong.han@intel.com> * update Signed-off-by: letonghan <letong.han@intel.com> * update Signed-off-by: letonghan <letong.han@intel.com> --------- Signed-off-by: letonghan <letong.han@intel.com> Signed-off-by: Chendi Xue <chendi.xue@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> Co-authored-by: Chendi Xue <chendi.xue@intel.com>
Signed-off-by: Abolfazl Shahbazi <abolfazl.shahbazi@intel.com>
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: Abolfazl Shahbazi <abolfazl.shahbazi@intel.com>
…-project#269) * Added support for new changes in utils.py Signed-off-by: Yogesh <yogeshpandey@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * trigger GitHub actions Signed-off-by: Yogesh <yogeshpandey@intel.com> --------- Signed-off-by: Yogesh <yogeshpandey@intel.com> Co-authored-by: Yogesh <yogeshpandey@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: lvliang-intel <liang1.lv@intel.com>
* PGvector service (opea-project#86) * Support PGvector service Signed-off-by: V, Ganesan <ganesan.v@intel.com> Signed-off-by: gadmarkovits <gad.markovits@intel.com>
…pendency for llm and rerank microservice (opea-project#152) * remove langchain dependency for llm and rerank Signed-off-by: lvliang-intel <liang1.lv@intel.com> * add llamaindex support for retrieval Signed-off-by: lvliang-intel <liang1.lv@intel.com> * fix schema issue Signed-off-by: lvliang-intel <liang1.lv@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix dockerfile Signed-off-by: lvliang-intel <liang1.lv@intel.com> * update readme Signed-off-by: lvliang-intel <liang1.lv@intel.com> * update reamde Signed-off-by: lvliang-intel <liang1.lv@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix entrypoint Signed-off-by: lvliang-intel <liang1.lv@intel.com> * add dataprep process in test script Signed-off-by: lvliang-intel <liang1.lv@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix redis url for dataprep Signed-off-by: lvliang-intel <liang1.lv@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update readme Signed-off-by: lvliang-intel <liang1.lv@intel.com> * update code Signed-off-by: lvliang-intel <liang1.lv@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: lvliang-intel <liang1.lv@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: root <root@idc708053.jf.intel.com>
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
for more information, see https://pre-commit.ci
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
for more information, see https://pre-commit.ci
lkk12014402
pushed a commit
that referenced
this pull request
Aug 8, 2024
Signed-off-by: Yue, Wenjiao <wenjiao.yue@intel.com> Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Add vLLM on Ray microservice
Issues
#194
Type of change
Dependencies
Tests