Denser Retriever

An enterprise-grade AI retriever designed to streamline AI integration into your applications, ensuring cutting-edge accuracy.

📝 Description

Denser Retriever combines multiple search technologies into a single platform. It utilizes gradient boosting ( xgboost) machine learning technique to combine:

• Keyword-based searches that focus on fetching precisely what the query mentions.
• Vector databases that are great for finding a wide range of potentially relevant answers.
• Machine Learning rerankers that fine-tune the results to ensure the most relevant answers top the list.

Our experiments on MTEB datasets show that the combination of keyword search, vector search and a reranker via an xgboost model (denoted as ES+VS+RR_n) can significantly improve the vector search (VS) baseline.

🚀 Features

The initial release of Denser Retriever provides the following features.

• Supporting heterogeneous retrievers such as keyword search, vector search, and ML model reranking
• Leveraging xgboost ML technique to effectively combine heterogeneous retrievers
• State-of-the-art accuracy on MTEB Retrieval benchmarking
• Demonstrating how to use Denser retriever to power an end-to-end applications such as chatbot and semantic search

📦 Installation

We highly recommend using poetry to install Denser Retriever. If you don't have poetry installed, you can install it with the following command.

pip install poetry

Then, you can install Denser Retriever with the following command.

poetry add git+https://github.com/denser-org/denser-retriever.git#main

📃 Documentation

The official documentation is hosted on retriever.denser.ai. Click here to get started.

👨🏼‍💻 Development

You can start developing Denser Retriever on your local machine.

See DEVELOPMENT.md for more details.

🛡 License

This project is licensed under the terms of the MIT license. See LICENSE for more details.

📃 Citation

@misc{denser-retriever,
  author = {denser-org},
  title = {An enterprise-grade AI retriever designed to streamline AI integration into your applications, ensuring cutting-edge accuracy.},
  year = {2024},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/denser-org/denser-retriever}}
}