Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Cohere reranker & add base class for rerankers #5979

Open
brian-ogrady opened this issue Jan 28, 2025 · 0 comments
Open

Fix Cohere reranker & add base class for rerankers #5979

brian-ogrady opened this issue Jan 28, 2025 · 0 comments
Labels
enhancement New feature or request

Comments

@brian-ogrady
Copy link
Contributor

Feature Request

New Features

This PR adds a new component called the LCCompressorComponent (standing for LangChain Compressor Component) which is a base class for reranker components. Common attributes of the rerankers are abstracted away into this class, leaving the build_compressor abstract method to be implemented by the child class. This requires sourcing BaseDocumentCompressor from langchain_core.documents.compressors but otherwise is a fairly lightweight class and is even an improvement because the previous implementation of rerankers were inaccurately subclassed from LCVectorStoreComponent.

New Rerankers

Adding new reranker components will be easier. There are desires for an integration with Voyage AI because their rerank-2 is both cost-effective and has high relevance.

Motivation

The Cohere & NVIDIA rerankers were marked as legacy because we wanted to deprecate retrievers and both components had relied on the ContextualCompressionRetriever abstraction from LangChain. This particular abstraction is redundant in LangFlow because simply performs similarity search on a vector store and then reranks the resulting documents using the provided reranker (referred to as a compressor in LC documentation). Because we have the search results coming directly from VectorStore components we can instead pass the results directly into the reranker without this abstracted class.

Your Contribution

This PR adds a new component called the LCCompressorComponent (standing for LangChain Compressor Component) which is a base class for reranker components. Common attributes of the rerankers are abstracted away into this class, leaving the build_compressor abstract method to be implemented by the child class. This requires sourcing BaseDocumentCompressor from langchain_core.documents.compressors but otherwise is a fairly lightweight class and is even an improvement because the previous implementation of rerankers were inaccurately subclassed from LCVectorStoreComponent.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant