WikiChat v2.0!

• Multilingual Support: By default, retrieves information from 10 different Wikipedias: 🇺🇸 English, 🇨🇳 Chinese, 🇪🇸 Spanish, 🇵🇹 Portuguese, 🇷🇺 Russian, 🇩🇪 German, 🇮🇷 Farsi, 🇯🇵 Japanese, 🇫🇷 French, and 🇮🇹 Italian.

• Improved Information Retrieval
  - Now supports retrieval from structured data such as tables, infoboxes, and lists, in addition to text.
  - Has the highest quality public Wikipedia preprocessing scripts
  - Uses the state-of-the-art multilingual retrieval model BGE-M3.
  - Uses Qdrant for scalable vector search.
  - Uses RankGPT to rerank search results.

• Free Multilingual Wikipedia Search API: We offer a high-quality, free (but rate-limited) search API for access to 10 Wikipedias, encompassing over 180M vector embeddings. See its API documentation.

• Recipe for adapting WikiChat to your own documents (instead of Wikipedia).

• Expanded LLM Compatibility: Supports 100+ LLMs through a unified interface, thanks to LiteLLM.

• Optimized Pipeline: Option for a faster and more cost-effective pipeline by merging the "generate" and "extract claim" stages of WikiChat.

• LangChain Compatibility: Fully compatible with LangChain 🦜️🔗.

• And Much More!

Full Changelog: v1.0...v2.0

v1.0

This release marks the code for our Findings of EMNLP 2023 paper.