How can I use my custom Ollama model in this project?

[vatche-t]

Hello!
I'm interested in integrating my custom Ollama model into this project. Could you please guide me on:

How to configure the project to work with a locally hosted model like Ollama?
Any specific requirements or dependencies I should be aware of?
Example setups or configurations for similar integrations would be very helpful.

Thank you in advance!

[cediorio]

I was able to use ollama easily. Here's the relevant code:

import instructor
from openai import OpenAI

# ...

client = instructor.from_openai(
    OpenAI(
        base_url="http://localhost:11434/v1",
        api_key="ollama", # not actually used...
    ),
    mode=instructor.Mode.JSON
)