Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prompting local LLMs throws exception #217

Closed
mydeveloperplanet opened this issue Aug 17, 2024 · 0 comments · Fixed by #218
Closed

Prompting local LLMs throws exception #217

mydeveloperplanet opened this issue Aug 17, 2024 · 0 comments · Fixed by #218

Comments

@mydeveloperplanet
Copy link
Collaborator

Version: 0.2.13
Using local LLMs is broken. When the response is returned a nullpointexception occurs:

java.util.concurrent.CompletionException: java.lang.NullPointerException: Cannot invoke "dev.langchain4j.model.output.TokenUsage.inputTokenCount()" because "tokenUsage" is null
	at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)
	at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320)
	at java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:722)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1773)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.lang.NullPointerException: Cannot invoke "dev.langchain4j.model.output.TokenUsage.inputTokenCount()" because "tokenUsage" is null
	at com.devoxx.genie.model.request.ChatMessageContext.setTokenUsageAndCost(ChatMessageContext.java:49)
	at com.devoxx.genie.service.NonStreamingPromptExecutor.lambda$execute$0(NonStreamingPromptExecutor.java:44)
	at java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:718)
	... 5 more

Fix is on its way...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant