Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new DocIndexRetriever example #405

Merged
merged 12 commits into from
Aug 21, 2024

Conversation

xuechendi
Copy link
Collaborator

@xuechendi xuechendi commented Jul 16, 2024

Description

Add a new example for only add data to vector store and retrieve from vector store

related AIComps change:
opea-project/GenAIComps#314

Issues

n/a.

Type of change

List the type of change like below. Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds new functionality)
  • Breaking change (fix or feature that would break existing design and interface)

Dependencies

No

Tests

No

@xuechendi
Copy link
Collaborator Author

@minmin-intel @ftian1 @lvliang-intel , please take a look

Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
@xuechendi xuechendi force-pushed the DocIndexRetriever branch from ee907e6 to efa8275 Compare July 16, 2024 20:02
pre-commit-ci bot and others added 2 commits July 16, 2024 20:03
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
@xuechendi xuechendi force-pushed the DocIndexRetriever branch 5 times, most recently from eb91097 to 9f73f29 Compare July 16, 2024 21:30
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
@xuechendi xuechendi force-pushed the DocIndexRetriever branch from 9f73f29 to 484a033 Compare July 16, 2024 21:36
Copy link
Collaborator

@minmin-intel minmin-intel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

Copy link
Contributor

@hshen14 hshen14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xuechendi It seems the new GenAI example could be part of ChatQnA (RAG). Does it make sense to improve ChatQnA to add this example?

@xuechendi
Copy link
Collaborator Author

@xuechendi It seems the new GenAI example could be part of ChatQnA (RAG). Does it make sense to improve ChatQnA to add this example?

@hshen14 , thanks for the review.
The reason we extract RAG from ChatQnA is that we will not need LLM inference part. And we are planning to call this RAG megaservice by Agent.

If we use ChatQnA directly, we will need to provide different Spec from ChatQnA with different chatQnA megaflow to only enable RAG part.

Either way is working for me, but I felt adding a new example is clearer. Also, we can support more than one optional vectorstore backend in RAG megaservice for different scenarios.

@xuechendi xuechendi mentioned this pull request Jul 17, 2024
3 tasks
@hshen14
Copy link
Contributor

hshen14 commented Jul 25, 2024

@xuechendi It seems the new GenAI example could be part of ChatQnA (RAG). Does it make sense to improve ChatQnA to add this example?

@hshen14 , thanks for the review. The reason we extract RAG from ChatQnA is that we will not need LLM inference part. And we are planning to call this RAG megaservice by Agent.

If we use ChatQnA directly, we will need to provide different Spec from ChatQnA with different chatQnA megaflow to only enable RAG part.

Either way is working for me, but I felt adding a new example is clearer. Also, we can support more than one optional vectorstore backend in RAG megaservice for different scenarios.

We are now having Agent example. Since it's designed to be used by Agent, how about moving under Agent?

@chensuyue chensuyue modified the milestones: v0.8, v0.9 Jul 25, 2024
@xuechendi xuechendi linked an issue Aug 15, 2024 that may be closed by this pull request
xuechendi and others added 5 commits August 19, 2024 18:37
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
@xuechendi
Copy link
Collaborator Author

@kevinintel , can you help to '@' the right person who should review/approve this PR.
So we can provide it to RagAgent and InsightAgent who will treat DocRetriever as Tool.

@kevinintel kevinintel merged commit 566cf93 into opea-project:main Aug 21, 2024
10 checks passed
dmsuehir pushed a commit to dmsuehir/GenAIExamples that referenced this pull request Sep 11, 2024
* Add DocIndexRetriever example

Signed-off-by: Chendi.Xue <chendi.xue@intel.com>


---------

Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: XuhuiRen <44249229+XuhuiRen@users.noreply.github.com>
wangkl2 pushed a commit to wangkl2/GenAIExamples that referenced this pull request Dec 11, 2024
…format (opea-project#405)

Co-authored-by: sys-lpot-val <sys_lpot_val@intel.com>
Co-authored-by: lvliang-intel <liang1.lv@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

IndexRetriever Example
6 participants