Connect to Ollama

[razvanab]

I would like to use a llm model through ollama when using --nomodel with Koboltcpp.

[LostRuins]

You can click connect to external endpoint, and then connect it as a custom OpenAI API.
Then run ollama in openai api mode.

But koboldcpp should provide superior results. Why do you need ollama?

[LostRuins]

Edit: Ollama API is now supported!

[razvanab]

Ollama now supports downloading models directly from hugging faces. There is no need to download them separately in GGUF format for other apps.

[razvanab]

Yes, it did work changing the AI provider to OpenAI Compatible API.
Thank you.

Or not. It's asking for the API key to work, but Ollama doesn't use an API key.

[LostRuins]

Hi @razvanab you can just use any random string as the api key

[razvanab]

I tried. It didn't work.

[razvanab]

It is working now in the latest release. Great work, guys!
Hopefully you will manage to fix the streaming too in the next release.

👍

[LostRuins]

Glad it works well for you