Skip to content

Improve OpenRouter Support #144

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Improve OpenRouter Support #144

wants to merge 3 commits into from

Conversation

sirwolfgang
Copy link

@sirwolfgang sirwolfgang commented Apr 29, 2025

This fixes issues with OpenRouter support by splitting the media handling off of OpenAI's implementation, which is a little different. As part of that it also improves the way Media is loaded by Chat to ensure each provider is running it's own version, allowing for each extension by others.

Currently this works with either, making this default a URI option.

assistant.ask "Summarize this document", with: { pdf: pdf_path }

assistant.ask "Summarize this document", with: { pdf: pdf_url }

This interface does limit us from being able to pass in contents itself. This interface should likely be expanded or made more explicit to support raw files or already base64 encoded files.

Additionally this PR exposes the ability to set OpenRouter's Header settings and Provider Settings:

  # --- Open Router Configuration ---
  config.openrouter_referer                     = 'https://rubyllm.com'
  config.openrouter_title                       = 'RubyLLM'
  config.openrouter_provider_order              = nil
  config.openrouter_provider_allow_fallbacks    = true
  config.openrouter_provider_require_parameters = false
  config.openrouter_provider_data_collection    = 'allow'
  config.openrouter_provider_ignore             = nil
  config.openrouter_provider_quantizations      = nil
  config.openrouter_provider_sort               = nil

@sirwolfgang sirwolfgang marked this pull request as ready for review April 29, 2025 23:40
@sirwolfgang sirwolfgang force-pushed the main branch 4 times, most recently from 696e779 to 186998a Compare April 30, 2025 16:38
@sirwolfgang sirwolfgang changed the title Improve OpenRouter Media Support Improve OpenRouter Support Apr 30, 2025
@sirwolfgang
Copy link
Author

@crmne What would you like to see to get this merged in? It improves OpenRouter support a lot, and I want to prevent someone else from duplicating work

Copy link
Owner

@crmne crmne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this PR changes anthropic, bedrock, deepseek, gemini, ollama, and openai too?

@sirwolfgang
Copy link
Author

sirwolfgang commented May 12, 2025

@crmne Because the current setup doesn't totally allow code reuse through extension, this tweaks it to allow that to work as needed while cleaning up some of the usage that was patched in.

We can remove these tweaks. My expectations is that you'd like to continue to be able to share code across providers, and continue to use this module_function style. At the intersection of these, when Provider A makes use of Provider B module it moves into B's namespace; Which means when it needs to call back to Provider A's version of something, it's unavailable and will call Provider B's version. This is likely going to lead to some unexpected bugs.

These changes copy Provider's B implantation into Provider A namespace, so that calls can be directed back to Provider A implantation when it has been defined.

@crmne
Copy link
Owner

crmne commented May 14, 2025

@sirwolfgang could you please move these improvements in another PR? I'd like to merge the improved OpenRouter support but don't want to change half of the library for it.

@sirwolfgang
Copy link
Author

@crmne I can move some of them, but not all of them. I made this changes because I was simply unable to extend the OpenAI code and have it work within the OpenRouter namespace. So I can remove the self:: and the deepseek stub, but that would leave part of the code fixed and part of the code base buggy.

Do you still wish for me to remove the improvements from the other files?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants