Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Request] Allow custom openai endpoint #1424

Closed
lenaxia opened this issue Dec 19, 2023 · 3 comments · Fixed by #1413
Closed

[Request] Allow custom openai endpoint #1424

lenaxia opened this issue Dec 19, 2023 · 3 comments · Fixed by #1413
Labels

Comments

@lenaxia
Copy link

lenaxia commented Dec 19, 2023

Due to changes in PrivateGPT, openai replacements no longer work as we cannot define custom openai endpoints.

LocalAI is an example: https://github.com/mudler/LocalAI/tree/master/examples/privateGPT

It is a drop in replacement for ChatGPT Openai with compatible REST APIs. Being able to define custom openai endpoints in the settings files would enable us to use localai.

@creasydude
Copy link

any updates?

@mchill
Copy link
Contributor

mchill commented Dec 21, 2023

This is pretty easy to implement with OpenAILike. I tested it briefly at first with LocalAI, and then with vLLM. I also need this feature, so I'll get around to opening a PR if no one else picks it up.

mchill added a commit to mchill/privateGPT that referenced this issue Dec 22, 2023
This mode behaves the same as the openai mode, except that it allows setting custom models not
supported by OpenAI. It can be used with any tool that serves models from an OpenAI compatible API.

Implements zylon-ai#1424
imartinez pushed a commit that referenced this issue Dec 26, 2023
This mode behaves the same as the openai mode, except that it allows setting custom models not
supported by OpenAI. It can be used with any tool that serves models from an OpenAI compatible API.

Implements #1424
Copy link
Contributor

github-actions bot commented Jan 6, 2024

Stale issue

@github-actions github-actions bot added the stale label Jan 6, 2024
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Feb 1, 2024
simonbermudez pushed a commit to simonbermudez/saimon that referenced this issue Feb 24, 2024
This mode behaves the same as the openai mode, except that it allows setting custom models not
supported by OpenAI. It can be used with any tool that serves models from an OpenAI compatible API.

Implements zylon-ai#1424
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants