-
Notifications
You must be signed in to change notification settings - Fork 1
Closed
Labels
Description
Scope
Implement vision support in the Ollama provider, including a separate configurable model for image processing.
Changes
-
crates/zeph-core/src/config/types.rs- Add
vision_model: Option<String>field to Ollama config section
- Add
-
config/default.toml- Add commented
# vision_model = "llava"example
- Add commented
-
crates/zeph-llm/src/ollama.rs- Override
supports_vision() -> truewhenvision_modelis configured - In
convert_message(): extractMessagePart::Imagedata, pass viaChatMessage::with_images() - When vision_model is set and message contains images: temporarily switch to vision model for that request, then switch back
- Override
Notes
ollama-rsChatMessagealready has.with_images()— just needs wiring- Vision models (llava, bakllava, moondream) are pulled separately in Ollama
- Test with mock: verify images are passed through to the API request
Reactions are currently unavailable