Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: ✨ Added filtering option to FAISS vectorstore #5966

Merged
merged 3 commits into from
Jun 11, 2023

Conversation

vempaliakhil96
Copy link
Contributor

@vempaliakhil96 vempaliakhil96 commented Jun 10, 2023

Inspired by the filtering capability available in ChromaDB, added the same functionality to the FAISS vectorestore as well. Since FAISS does not have an inbuilt method of filtering used the approach suggested in this thread
Langchain Issue inspiration: #4572

  • Added filtering capability to semantic similarly and MMR
  • Added test cases for filtering in tests/integration_tests/vectorstores/test_faiss.py

Who can review?

Tag maintainers/contributors who might be interested:

VectorStores / Retrievers / Memory

Inspired by the filtering capability available in ChromaDB, added the same functionality to FAISS vectorestore aswell. Since FAISS does not have an inbuilt method of filtering used the approach suggested in this thread facebookresearch/faiss#1079

fix: 🧪 Removed FAISS vectorestore testcase from unit tests as the git actions were failing
refactor: 🚨 fixed line error in comments

chore: 🚨 lint errors
Copy link
Contributor

@hwchase17 hwchase17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is fantastic! can we add an example to the FAISS notebook?

langchain/vectorstores/faiss.py Outdated Show resolved Hide resolved
refactor: 🚨 Changed Typing Anotator for filters
Copy link
Contributor

@hwchase17 hwchase17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm - thanks!

@hwchase17 hwchase17 merged commit d7d6299 into langchain-ai:master Jun 11, 2023
Undertone0809 pushed a commit to Undertone0809/langchain that referenced this pull request Jun 19, 2023
Inspired by the filtering capability available in ChromaDB, added the
same functionality to the FAISS vectorestore as well. Since FAISS does
not have an inbuilt method of filtering used the approach suggested in
this [thread](facebookresearch/faiss#1079)
Langchain Issue inspiration:
langchain-ai#4572

- [x] Added filtering capability to semantic similarly and MMR
- [x] Added test cases for filtering in
`tests/integration_tests/vectorstores/test_faiss.py`

#### Who can review?

Tag maintainers/contributors who might be interested:

  VectorStores / Retrievers / Memory
  - @dev2049
  - @hwchase17
This was referenced Jun 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants