Open access LLMs

[Tenfleques]

In the spirit of research, would it be idea if the assistant can connect to any other LLM even local?

[lifan0127]

Behind the scene, this plugin uses LangChain JS to interact with a chat model. So far the only choices in this family are OpenAI and Anthropic, but I imagine more open-access LLMs will become available in the future.

https://js.langchain.com/docs/modules/models/chat/integrations

[jmgoguikian]

How did you decide to go with OpenAI fam vs. Anthropic for Zoro RA?

[lifan0127]

Hi @jmgoguikian, right now we use the default OpenAI chat model. I personally do not have deep experience with the Anthropic model/api.

[juniorbonner]

It would be good to be able to choose to use gpt-4

[man00ka]

Local LLMs seem to be possible now, as, for example, ChatOllama is listed by now: https://js.langchain.com/docs/integrations/chat/

Would be really great if this worked in the near future within ARIA : )

[lifan0127]

Hi @adkiem, ARIA heavily uses GPT's function calling feature to interact with Zotero APIs. I see that open-source LLMs such as Ollama has started to support similar features but we will need to measure their performance in comparison to GPT-4. To that end, I am curating a dataset to support such functional testing so we can more easily explore/validate different model configurations.

[BradKML]

LiteLLM, OpenRouter, Ollama. The trinity of local use + semi-local services.