Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(OpenAIClient/PluginsClient): allow non-v1 reverse proxy, handle "v1/completions" reverse proxy #1029

Merged
merged 5 commits into from
Oct 8, 2023

Conversation

danny-avila
Copy link
Owner

@danny-avila danny-avila commented Oct 8, 2023

fixes #1026
fixes #1027

Aside from handling the niche case of a reverse proxy pointing to v1/completions instead of v1/chat/completions, you can also force a prompt payload instead of messages (useful for some localai models)

# (Advanced) Sometimes when using Local LLM APIs, you may need to force the API
# to be called with a `prompt` payload instead of a `messages` payload; to mimic the
# a `/v1/completions` request instead of `/v1/chat/completions`
# This may be the case for LocalAI with some models. To do so, uncomment the following:
# OPENAI_FORCE_PROMPT=true

Change Type

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Documentation update

Testing

ci(OpenAIClient): add tests for OPENROUTER_API_KEY, FORCE_PROMPT, and reverseProxyUrl handling in setOptions

Checklist

  • My code adheres to this project's style guidelines
  • I have performed a self-review of my own code
  • I have commented in any complex areas of my code
  • I have made pertinent documentation changes
  • My changes do not introduce new warnings
  • I have written tests demonstrating that my changes are effective or that my feature works
  • Local unit tests pass with my changes
  • Any changes dependent on mine have been merged and published in downstream modules.

@danny-avila danny-avila merged commit 2dd545e into main Oct 8, 2023
3 checks passed
@danny-avila danny-avila deleted the reverse-proxy branch October 8, 2023 20:57
cnkang pushed a commit to cnkang/LibreChat that referenced this pull request Feb 6, 2024
…v1/completions" reverse proxy (danny-avila#1029)

* fix(OpenAIClient): handle completions request in reverse proxy, also force prompt by env var

* fix(reverseProxyUrl): allow url without /v1/ but add server warning as it will not be compatible with plugins

* fix(ModelService): handle reverse proxy without v1

* refactor: make changes cleaner

* ci(OpenAIClient): add tests for OPENROUTER_API_KEY, FORCE_PROMPT, and reverseProxyUrl handling in setOptions
jinzishuai pushed a commit to aitok-ai/LibreChat that referenced this pull request May 20, 2024
…v1/completions" reverse proxy (danny-avila#1029)

* fix(OpenAIClient): handle completions request in reverse proxy, also force prompt by env var

* fix(reverseProxyUrl): allow url without /v1/ but add server warning as it will not be compatible with plugins

* fix(ModelService): handle reverse proxy without v1

* refactor: make changes cleaner

* ci(OpenAIClient): add tests for OPENROUTER_API_KEY, FORCE_PROMPT, and reverseProxyUrl handling in setOptions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant