Some thoughts on keeping the space of AI models open and accessible for everyone (LocalAI)

[dragetd]

Hey everyone,

I do not really have a feature request or idea, as I checked the code and the OpenAI API BaseURL is already a property that can be overwritten. So I guess this is already possible. But I still thought I'd bring attention to a neat project: https://github.com/mudler/LocalAI

The idea is, that it recreates the OpenAI API to some extend (not only chat, but also image and voice) but allows running local models of different kinds. Basically any tool build for OpenAI the exposes the API URL can thus be used locally, without any privacy concerns.

I do not think this relates that much to spring-ai as the BaseURL is configurable. Maybe worth a mention in the docs.

Very cool project tho. I have built my own abstractions around LLM APIs it Kotlin a while ago, but nothing so neatly integrated into spring like this project seems to be. I really hope we can push it towards GA :D

[markpollack]

Hi. Thanks for the input. Yes, this is worthwhile to mention that many other projects have adopted the open ai API.

I've added the following section to the docs.

== OpenAI-Compatible Models

A variety of models compatible with the OpenAI API are available, including those that can be operated locally, such as LocalAI. The standard configuration for connecting to the OpenAI API is through the spring.ai.openai.baseUrl property, which defaults to https://api.openai.com.

To link the OpenAI client to a compatible model that utilizes the OpenAI API, you should adjust the spring.ai.openai.baseUrl property to the corresponding URL of the model you wish to connect to.