Completion on available Ollama models support for `quarkus.langchain4j.ollama.chat-model.model-id` property value #985

angelozerr · 2024-08-20T14:24:57Z

feat: Available Ollama models support for quarkus.langchain4j.ollama.chat-model.model-id property value

Those models are retrieved by consuming the Ollama API ${base-url}/v1/models where base-url equals to:

the value coming from declared quarkus.langchain4j.ollama.base-url property in the application.properties.
http://localhost:11434 otherwise

Here the result in vscode with completion:

Here the result with validtaion:

As the error message is generic, it is not very good, LSP4MP must be improved to contribute with custom error validation message.

The Ollama models are cached, so if you add a new model in Ollama, you need to close the file and reopen it to refresh the cache (not tested but I think it should work).

...uarkus.ls/src/main/java/com/redhat/quarkus/extensions/ollama/OllamaItemMetadataProvider.java

datho7561 · 2024-09-10T14:56:26Z

If ollama is not running, then the list of models cannot be collected. There is an error logged to the lsp4mp output. I think it's okay to not list the models, but I think we should prevent the error from being logged:

SEVERE: Error while collecting Ollama Models with 'http://localhost:11434/v1/models'.
java.net.ConnectException
	at java.net.http/jdk.internal.net.http.HttpClientImpl.send(HttpClientImpl.java:956)
	at java.net.http/jdk.internal.net.http.HttpClientFacade.send(HttpClientFacade.java:133)
	at com.redhat.quarkus.extensions.ollama.OllamaItemMetadataProvider.collectOllamaModels(OllamaItemMetadataProvider.java:129)
	at com.redhat.quarkus.extensions.ollama.OllamaItemMetadataProvider.update(OllamaItemMetadataProvider.java:105)
	at org.eclipse.lsp4mp.extensions.ExtendedMicroProfileProjectInfo.synchUpdateCustomProperties(ExtendedMicroProfileProjectInfo.java:210)
	at org.eclipse.lsp4mp.extensions.ExtendedMicroProfileProjectInfo.updateCustomProperties(ExtendedMicroProfileProjectInfo.java:195)
	at org.eclipse.lsp4mp.services.properties.PropertiesFileLanguageService.updateProperties(PropertiesFileLanguageService.java:271)
	at org.eclipse.lsp4mp.services.properties.PropertiesFileLanguageService.doDiagnostics(PropertiesFileLanguageService.java:227)
	at org.eclipse.lsp4mp.ls.properties.PropertiesFileTextDocumentService.triggerValidationFor(PropertiesFileTextDocumentService.java:347)
	at org.eclipse.lsp4mp.ls.properties.PropertiesFileTextDocumentService.lambda$triggerValidationFor$17(PropertiesFileTextDocumentService.java:332)
	at java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1194)
	at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:527)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:507)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1458)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:2034)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:189)
Caused by: java.net.ConnectException
	at java.net.http/jdk.internal.net.http.common.Utils.toConnectException(Utils.java:1069)
	at java.net.http/jdk.internal.net.http.PlainHttpConnection.connectAsync(PlainHttpConnection.java:227)
	at java.net.http/jdk.internal.net.http.PlainHttpConnection.checkRetryConnect(PlainHttpConnection.java:280)
	at java.net.http/jdk.internal.net.http.PlainHttpConnection.lambda$connectAsync$2(PlainHttpConnection.java:238)
	at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:978)
	at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:955)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1817)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1575)
Caused by: java.nio.channels.ClosedChannelException
	at java.base/sun.nio.ch.SocketChannelImpl.ensureOpen(SocketChannelImpl.java:204)
	at java.base/sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:958)
	at java.net.http/jdk.internal.net.http.PlainHttpConnection.lambda$connectAsync$1(PlainHttpConnection.java:210)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:571)
	at java.net.http/jdk.internal.net.http.PlainHttpConnection.connectAsync(PlainHttpConnection.java:212)
	... 9 more

datho7561 · 2024-09-10T15:01:28Z

Gotta update my ollama; I'm on 0.1.20 and it looks like the /v1/models endpoint doesn't exist in that version.

angelozerr · 2024-09-10T15:06:22Z

If ollama is not running, then the list of models cannot be collected. There is an error logged to the lsp4mp output. I think it's okay to not list the models, but I think we should prevent the error from being logged:

SEVERE: Error while collecting Ollama Models with 'http://localhost:11434/v1/models'.
java.net.ConnectException
	at java.net.http/jdk.internal.net.http.HttpClientImpl.send(HttpClientImpl.java:956)
	at java.net.http/jdk.internal.net.http.HttpClientFacade.send(HttpClientFacade.java:133)
	at com.redhat.quarkus.extensions.ollama.OllamaItemMetadataProvider.collectOllamaModels(OllamaItemMetadataProvider.java:129)
	at com.redhat.quarkus.extensions.ollama.OllamaItemMetadataProvider.update(OllamaItemMetadataProvider.java:105)
	at org.eclipse.lsp4mp.extensions.ExtendedMicroProfileProjectInfo.synchUpdateCustomProperties(ExtendedMicroProfileProjectInfo.java:210)
	at org.eclipse.lsp4mp.extensions.ExtendedMicroProfileProjectInfo.updateCustomProperties(ExtendedMicroProfileProjectInfo.java:195)
	at org.eclipse.lsp4mp.services.properties.PropertiesFileLanguageService.updateProperties(PropertiesFileLanguageService.java:271)
	at org.eclipse.lsp4mp.services.properties.PropertiesFileLanguageService.doDiagnostics(PropertiesFileLanguageService.java:227)
	at org.eclipse.lsp4mp.ls.properties.PropertiesFileTextDocumentService.triggerValidationFor(PropertiesFileTextDocumentService.java:347)
	at org.eclipse.lsp4mp.ls.properties.PropertiesFileTextDocumentService.lambda$triggerValidationFor$17(PropertiesFileTextDocumentService.java:332)
	at java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1194)
	at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:527)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:507)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1458)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:2034)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:189)
Caused by: java.net.ConnectException
	at java.net.http/jdk.internal.net.http.common.Utils.toConnectException(Utils.java:1069)
	at java.net.http/jdk.internal.net.http.PlainHttpConnection.connectAsync(PlainHttpConnection.java:227)
	at java.net.http/jdk.internal.net.http.PlainHttpConnection.checkRetryConnect(PlainHttpConnection.java:280)
	at java.net.http/jdk.internal.net.http.PlainHttpConnection.lambda$connectAsync$2(PlainHttpConnection.java:238)
	at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:978)
	at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:955)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1817)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1575)
Caused by: java.nio.channels.ClosedChannelException
	at java.base/sun.nio.ch.SocketChannelImpl.ensureOpen(SocketChannelImpl.java:204)
	at java.base/sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:958)
	at java.net.http/jdk.internal.net.http.PlainHttpConnection.lambda$connectAsync$1(PlainHttpConnection.java:210)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:571)
	at java.net.http/jdk.internal.net.http.PlainHttpConnection.connectAsync(PlainHttpConnection.java:212)
	... 9 more

I did that to understand some trouble with ollama. Perhaps a better thing to do is to report this error as LSP diagnostic to notify to the user that it cannot connect to Ollama? @fbricon @datho7561 what do you think about this idea?

angelozerr · 2024-09-10T15:07:29Z

Gotta update my ollama; I'm on 0.1.20 and it looks like the /v1/models endpoint doesn't exist in that version.

That's too bad -( Perhaps we should use api instead https://github.com/ollama/ollama/blob/main/docs/api.md ? @fbricon what do you think about that?

fbricon · 2024-09-10T15:22:56Z

Gotta update my ollama; I'm on 0.1.20 and it looks like the /v1/models endpoint doesn't exist in that version.

That's too bad -( Perhaps we should use api instead https://github.com/ollama/ollama/blob/main/docs/api.md ? @fbricon what do you think about that?

Yeah /api/tags will cover more ollama versions.
/v1/models is a recent addition for Open AI compat'

fbricon · 2024-09-10T15:25:18Z

I did that to understand some trouble with ollama. Perhaps a better thing to do is to report this error as LSP diagnostic to notify to the user that it cannot connect to Ollama? @fbricon @datho7561 what do you think about this idea?

I'd rather avoid adding annoying diags. I'd say we ignore this problem for the time being. We might revisit this decision over time, depending on user feedback.

datho7561 · 2024-09-10T18:59:53Z

I updated to the latest ollama. The PR works very well! The default ollama installation method on Linux sets ollama up as a background service. As a result, the user is unlikely to run into the error I ran into.

(oops meant to sent this a while ago)

angelozerr · 2024-09-10T19:27:09Z

Gotta update my ollama; I'm on 0.1.20 and it looks like the /v1/models endpoint doesn't exist in that version.

That's too bad -( Perhaps we should use api instead https://github.com/ollama/ollama/blob/main/docs/api.md ? @fbricon what do you think about that?

Yeah /api/tags will cover more ollama versions. /v1/models is a recent addition for Open AI compat'

It means that I need to switch to /api/tags, right?

angelozerr · 2024-09-10T19:28:20Z

I did that to understand some trouble with ollama. Perhaps a better thing to do is to report this error as LSP diagnostic to notify to the user that it cannot connect to Ollama? @fbricon @datho7561 what do you think about this idea?

I'd rather avoid adding annoying diags. I'd say we ignore this problem for the time being. We might revisit this decision over time, depending on user feedback.

Ok let's give up diags, but should hide the error from stacktrace like @datho7561 reported?

angelozerr · 2024-09-10T19:30:15Z

I updated to the latest ollama. The PR works very well!

Thanks!

The default ollama installation method on Linux sets ollama up as a background service. As a result, the user is unlikely to run into the error I ran into.

You mean using v1 instead of api/tags?

(oops meant to sent this a while ago)

datho7561 · 2024-09-10T19:38:45Z

You mean using v1 instead of api/tags?

No, I just needed to reinstall ollama.

In order to install ollama, I ran the script curl -fsSL https://ollama.com/install.sh | sh. This sets ollama up so that the background job that ollama needs is automatically started up when your computer starts.

The first time I installed ollama I didn't follow their instructions, I just downloaded the binary.

angelozerr · 2024-09-11T09:27:44Z

@datho7561 I updated my PR:

now when there is an error, I display in the log the error with the stacktrace to have this kind of error:
SEVERE: Could not connect to ollama 'http://localhost:8080/api/tags': null

Here null, is the e.getMessage()

I'm using /api/tags so it should work with any ollama version

...uarkus.ls/src/main/java/com/redhat/quarkus/extensions/ollama/OllamaItemMetadataProvider.java

fbricon · 2024-09-11T10:11:53Z

please add missing copyright headers

angelozerr · 2024-09-11T12:50:35Z

@datho7561 I did several refactoring, please retry it.

angelozerr · 2024-09-11T12:50:55Z

please add missing copyright headers

fixed

quarkus.ls.ext/com.redhat.quarkus.ls/mvnw

...uarkus.ls/src/main/java/com/redhat/quarkus/extensions/ollama/OllamaItemMetadataProvider.java

...om.redhat.quarkus.ls/src/main/java/com/redhat/quarkus/extensions/ollama/OllamaConstants.java

quarkus.langchain4j.ollama.chat-model.model-id property value Signed-off-by: azerr <azerr@redhat.com>

datho7561

Works well for me, and the code looks good. Thanks, Angelo!

datho7561 · 2024-09-12T12:55:56Z

test failures seem to be because lsp4mp hasn't built a snapshot yet

angelozerr · 2024-09-13T13:38:41Z

Thanks so much @fbricon @datho7561 for your great review!

openshift-ci bot added the do-not-merge/work-in-progress label Aug 20, 2024

angelozerr force-pushed the ollama branch 10 times, most recently from 07c85d8 to fa24e22 Compare August 21, 2024 08:07

angelozerr marked this pull request as ready for review August 21, 2024 08:07

openshift-ci bot removed the do-not-merge/work-in-progress label Aug 21, 2024

fbricon reviewed Aug 21, 2024

View reviewed changes

...uarkus.ls/src/main/java/com/redhat/quarkus/extensions/ollama/OllamaItemMetadataProvider.java Outdated Show resolved Hide resolved

fbricon reviewed Aug 21, 2024

View reviewed changes

...uarkus.ls/src/main/java/com/redhat/quarkus/extensions/ollama/OllamaItemMetadataProvider.java Outdated Show resolved Hide resolved

angelozerr force-pushed the ollama branch from fa24e22 to 35177e4 Compare August 21, 2024 08:40

angelozerr self-assigned this Aug 28, 2024

angelozerr added the enhancement New feature or request label Aug 28, 2024

angelozerr modified the milestones: 0.20.0, 0.19.1 Aug 28, 2024

angelozerr force-pushed the ollama branch from 35177e4 to 554034e Compare August 29, 2024 11:15

angelozerr force-pushed the ollama branch 4 times, most recently from ea8466c to 4b5acce Compare September 10, 2024 06:54

angelozerr force-pushed the ollama branch from 4b5acce to 688e164 Compare September 11, 2024 08:59

angelozerr force-pushed the ollama branch from 688e164 to 362b477 Compare September 11, 2024 09:28

fbricon reviewed Sep 11, 2024

View reviewed changes

...uarkus.ls/src/main/java/com/redhat/quarkus/extensions/ollama/OllamaItemMetadataProvider.java Outdated Show resolved Hide resolved

fbricon reviewed Sep 11, 2024

View reviewed changes

...uarkus.ls/src/main/java/com/redhat/quarkus/extensions/ollama/OllamaItemMetadataProvider.java Outdated Show resolved Hide resolved

angelozerr force-pushed the ollama branch from 362b477 to 11347ea Compare September 11, 2024 12:50

angelozerr force-pushed the ollama branch from 11347ea to 1144014 Compare September 11, 2024 12:54

datho7561 reviewed Sep 11, 2024

View reviewed changes

quarkus.ls.ext/com.redhat.quarkus.ls/mvnw Outdated Show resolved Hide resolved

fbricon reviewed Sep 11, 2024

View reviewed changes

...uarkus.ls/src/main/java/com/redhat/quarkus/extensions/ollama/OllamaItemMetadataProvider.java Outdated Show resolved Hide resolved

datho7561 reviewed Sep 11, 2024

View reviewed changes

...om.redhat.quarkus.ls/src/main/java/com/redhat/quarkus/extensions/ollama/OllamaConstants.java Show resolved Hide resolved

angelozerr force-pushed the ollama branch 3 times, most recently from fe7d331 to c412727 Compare September 11, 2024 13:57

feat: Available Ollama models support for

3aa4c13

quarkus.langchain4j.ollama.chat-model.model-id property value Signed-off-by: azerr <azerr@redhat.com>

angelozerr force-pushed the ollama branch from c412727 to 3aa4c13 Compare September 12, 2024 06:53

datho7561 approved these changes Sep 12, 2024

View reviewed changes

angelozerr merged commit bd268fe into redhat-developer:master Sep 13, 2024
1 check passed

angelozerr changed the title ~~feat: Available Ollama models support for quarkus.langchain4j.ollama.chat-model.model-id property value~~ Completion on available Ollama models support for quarkus.langchain4j.ollama.chat-model.model-id property value Oct 22, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Completion on available Ollama models support for `quarkus.langchain4j.ollama.chat-model.model-id` property value #985

Completion on available Ollama models support for `quarkus.langchain4j.ollama.chat-model.model-id` property value #985

angelozerr commented Aug 20, 2024 •

edited

Loading

datho7561 commented Sep 10, 2024

datho7561 commented Sep 10, 2024

angelozerr commented Sep 10, 2024

angelozerr commented Sep 10, 2024

fbricon commented Sep 10, 2024

fbricon commented Sep 10, 2024 •

edited

Loading

datho7561 commented Sep 10, 2024

angelozerr commented Sep 10, 2024 •

edited

Loading

angelozerr commented Sep 10, 2024

angelozerr commented Sep 10, 2024

datho7561 commented Sep 10, 2024

angelozerr commented Sep 11, 2024

fbricon commented Sep 11, 2024

angelozerr commented Sep 11, 2024

angelozerr commented Sep 11, 2024

datho7561 left a comment

datho7561 commented Sep 12, 2024

angelozerr commented Sep 13, 2024

Completion on available Ollama models support for quarkus.langchain4j.ollama.chat-model.model-id property value #985

Completion on available Ollama models support for quarkus.langchain4j.ollama.chat-model.model-id property value #985

Conversation

angelozerr commented Aug 20, 2024 • edited Loading

datho7561 commented Sep 10, 2024

datho7561 commented Sep 10, 2024

angelozerr commented Sep 10, 2024

angelozerr commented Sep 10, 2024

fbricon commented Sep 10, 2024

fbricon commented Sep 10, 2024 • edited Loading

datho7561 commented Sep 10, 2024

angelozerr commented Sep 10, 2024 • edited Loading

angelozerr commented Sep 10, 2024

angelozerr commented Sep 10, 2024

datho7561 commented Sep 10, 2024

angelozerr commented Sep 11, 2024

fbricon commented Sep 11, 2024

angelozerr commented Sep 11, 2024

angelozerr commented Sep 11, 2024

datho7561 left a comment

Choose a reason for hiding this comment

datho7561 commented Sep 12, 2024

angelozerr commented Sep 13, 2024

Completion on available Ollama models support for `quarkus.langchain4j.ollama.chat-model.model-id` property value #985

Completion on available Ollama models support for `quarkus.langchain4j.ollama.chat-model.model-id` property value #985

angelozerr commented Aug 20, 2024 •

edited

Loading

fbricon commented Sep 10, 2024 •

edited

Loading

angelozerr commented Sep 10, 2024 •

edited

Loading