Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add Google AI Gemini Pro support #1209

Merged
merged 13 commits into from
Apr 11, 2024
Merged

feat: add Google AI Gemini Pro support #1209

merged 13 commits into from
Apr 11, 2024

Conversation

cpacker
Copy link
Collaborator

@cpacker cpacker commented Apr 2, 2024

Example Gemini API + MemGPT usage

Setting up Gemini with MemGPT configure:

$ memgpt configure
Loading config from /Users/loaner/.memgpt/config
? Select LLM inference provider: google_ai
? Enter your Google AI (Gemini) API key (see https://aistudio.google.com/app/apikey): *********
? Enter your Google AI (Gemini) service endpoint (see https://ai.google.dev/api/rest): generativelanguage
? Select default model: gemini-pro
Got context window 30720 for model gemini-pro (from Google API)
? Select your model's context window (see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/model-versioning#gemini-model-versions): 30720
? Select embedding provider: openai
? Select default preset: memgpt_chat
? Select default persona: sam_pov
? Select default human: basic
? Select storage backend for archival data: chroma
? Select chroma backend: persistent
? Select storage backend for recall data: sqlite
📖 Saving config to /Users/loaner/.memgpt/config

Testing an agent using the Gemini API:

$ memgpt run
Loading config from /Users/loaner/.memgpt/config

? Would you like to select an existing agent? No

🧬 Creating new agent...
->  🤖 Using persona profile: 'sam_pov'
->  🧑 Using human profile: 'basic'
Loading config from /Users/loaner/.memgpt/config
Loading config from /Users/loaner/.memgpt/config
🎉 Created new agent 'CapriciousDaffodil' (id=602e6e68-be69-480e-a744-374d329ccb9f)

Hit enter to begin (will request first MemGPT message)

💭 This is more like it! Hello there! This is indeed my first login. Let's see if I can access my 
memory core.
🤖 Hey there! Excited to be here.

Please describe the purpose of this pull request.

  • Adds support for Google AI Gemini
  • Refactors the llm_api_tools.py file into a nested directory for all code related to calling LLM API services (via REST)
    • "LLM API services code" = code where we call a server that supports ChatCompletion format with tool/function calling
  • Refactors the HTTP request code to:
    • Use ChatCompletionRequest as the payload/data arg type (to match ChatCompletionResponse in the return)
    • Pass around List[Message] instead of List[dict] in the calls to the APIs (get_ai_reply)
      • This makes it easier to convert to arbitrary payload/request types (such as Gemini) that are slightly different from OAI format

How to test
How can we test your PR during review? What commands should we run? What outcomes should we expect?

Have you tested this PR?

  • Test OpenAI
  • Test Azure
  • Test local
  • Test Gemini
  • Test summarize method

@cpacker cpacker mentioned this pull request Apr 2, 2024
32 tasks
Copy link
Collaborator

@sarahwooders sarahwooders left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@sarahwooders sarahwooders merged commit bdf7aeb into main Apr 11, 2024
4 checks passed
@cpacker cpacker deleted the gemini branch April 15, 2024 19:26
@quantumalchemy
Copy link

quantumalchemy commented May 6, 2024

Request upgrade to gemini 1.5 pro api
--> gemini-1.5-pro-latest
Add allow option / switch --> safety_settings=None

mattzh72 pushed a commit that referenced this pull request Oct 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants