Local LLM Support

[vyrti]

Feature Description

Hi, is it possible to add local LLM support?
Ollama, llama.cpp, lmstudio?

Use Case

Additional Context

No response

[walidshaari]

+1 or local or general OpenAI Compatible API endpoint.
I have a use case for regulated customers where they need to use it onprem with a local model

[a-canary]

+1 for local or general OpenAI Compatible API endpoint.

[maluki2001]

That will be a good option to not be limited just to Claude!
Please add the option to add external LLM's

[xdien]

+1 OpenAI Compatible endpoint

[kazini]

I dream of this, but I doubt they'll allow it anytime soon, if ever. I hope I'm wrong, this would be incredible.

[alphaleadership]

+1