Handle custom model API

[Etelis]

Hey,

I've been looking at the documentation and I can't find an easy approach to create a custom model, like one based on an API.

I came across FunctionModel, but according to the docs, it seems to be more of a testing pipeline rather than a generic interface.

Could you please provide guidance on how to create such a custom model?

[samuelcolvin]

You just need to implement the Model abstract base class.

To do that, you'll likely also need to implement the other abstract types in that class AgentModel, StreamTextResponse and StreamStructuredResponse.

For examples you can look at:

• Ollama support #162 - this doesn't have to implement AgentModel, StreamTextResponse and StreamStructuredResponse as it reuses the OpenAI model
• add Groq client support #84 - this implements all the abstract types

We should document how to create a custom model.

[sydney-runkle]

This should probably wait until we move the AgentModel refactor across the line.

Happy to take this on when that's done.

[izzyacademy]

I wonder if this is still necessary given the change in direction to just leverage the OpenAI client to OpenAI-compatible endpoints.