Multi-Modal Chatbot Powered by LangChain Agents, OpenAI's Function Calling, and Streamlit

Solution

The multi-modal chatbot I crafted is backed by an agent that uses three tools:

• REST Countries API Chain: enables retrieving information on countries, invoking Rest countries API
• DALL·E 3 Image Generator: Generates an image of countries based on the country name
• Google Search Tool: Useful for fetching information from the web

Getting Started

Clone the repository, set up the virtual environment, and install the required packages

1. git clone git@github.com:nirbar1985/country-compass-ai.git

2. ( In case you have python version 3.11.4 installed in pyenv)

   pyenv local 3.11.4

3. Install dependencies

   poetry install

4. Enter virtual env by:

   poetry shell

Store your API keys

• Create .env file
• Place your OPENAI_API_KEY into .env file
• Place your SERPAPI_API_KEY into .env file

Running the Multi-Modal ChatBot

Kick of the chatbot by running:

streamlit run chatbot_ui.py

Pose queries for instance -

• Create images of countries

  

• Retrieve information on countries

  

• Get current data

License

Distributed under the MIT License. See LICENSE.txt for more information.