Gemini : Streaming example not working on Android

[ImadSaddik]

Hello, I was following the quick-start guide that shows how to use the Gemini API in Android apps. So far I have been able to replicate most of the examples, but I was facing some problems because I have been getting errors while trying to run the code samples.

In this issue that I opened, I will focus on the streaming example. The getResponse method is a method that I created to get the model's response with streaming support.

public void getResponse(String query, ResponseCallback callback) {
        GenerativeModelFutures model = getModel();

        Content.Builder contentBuilder = new Content.Builder();
        contentBuilder.setRole("user");
        contentBuilder.addText(query);
        Content content = contentBuilder.build();

        Log.d("GeminiPro", "getResponse: " + query);
        Log.d("GeminiPro", "getResponse: " + content);

        final String[] fullResponse = {""};
        Publisher<GenerateContentResponse> streamingResponse = model.generateContentStream(content);
        Log.d("GeminiPro", "getResponse: " + streamingResponse);

        streamingResponse.subscribe(new Subscriber<GenerateContentResponse>() {
            @Override
            public void onSubscribe(Subscription s) {
            }

            @Override
            public void onNext(GenerateContentResponse generateContentResponse) {
                String chunk = generateContentResponse.getText();
                fullResponse[0] += chunk;
                Log.d("GeminiPro", "onNext: " + chunk);
            }

            @Override
            public void onComplete() {
                System.out.println(fullResponse[0]);
                Log.d("GeminiPro", "onComplete: " + fullResponse[0]);
                callback.onResponse(fullResponse[0]);
            }

            @Override
            public void onError(Throwable t) {
                t.printStackTrace();
                Log.d("GeminiPro", "onError: " + t.getMessage());
                callback.onError(t);
            }
        });
    }

First, in the code sample. The onSubscribe was not used, when I did the same nothing, I got nothing. The code kept running without giving a response. So, I added s.request(1) to request something from the publisher. This addition threw an exception that didn't make sense : contents is not specified why ?

FATAL EXCEPTION: DefaultDispatcher-worker-2
Process: com.example.streaminggeminiapp, PID: 2433
com.google.ai.client.generativeai.type.ServerException: * GenerateContentRequest.contents: contents is not specified
...

The content variable is already defined. I don't know if I am missing something or this is a bug that should be solved.

[MarkDaoust]

@rachelsaunders, @thatfiredev please take a look.

[davidmotson]

Hello, I notice this method is doing a lot of logging, would it be possible to see an example of one of these logs?

Also, I'm not sure what you mean by adding "s.request(1)". There are no variables named "s" in your example code. Please tell me more.

[ImadSaddik]

Hi @davidmotson, sorry about the confusion, I have added s.request(1) in the onSubscribe method like this.

@Override
            public void onSubscribe(Subscription s) {
                s.request(1);
            }

I did this because when I tried to Log the chunks inside the onNext method, I wasn't getting anything.

@Override
            public void onNext(GenerateContentResponse generateContentResponse) {
                String chunk = generateContentResponse.getText();
                fullResponse[0] += chunk;
                Log.d("GeminiPro", "onNext: " + chunk);
            }

[ImadSaddik]

You mentioned the logs, here they are :

This is what I get without adding s.request(1)

2024-02-09 08:31:55.960  4424-4424  GeminiPro               com.example.streaminggeminiapp       D  onCreate: Hello Gemini 
2024-02-09 08:31:55.997  4424-4424  GeminiPro               com.example.streaminggeminiapp       D  getResponse: Hello Gemini 
2024-02-09 08:31:55.998  4424-4424  GeminiPro               com.example.streaminggeminiapp       D  getResponse: com.google.ai.client.generativeai.type.Content@f02559b
2024-02-09 08:31:56.002  4424-4424  GeminiPro               com.example.streaminggeminiapp       D  getResponse: kotlinx.coroutines.reactive.FlowAsPublisher@af5dd38

This is what I get after adding s.request(1)

2024-02-09 08:37:17.445  5078-5078  GeminiPro               com.example.streaminggeminiapp       D  onCreate: Hello Gemini 
2024-02-09 08:37:17.482  5078-5078  GeminiPro               com.example.streaminggeminiapp       D  getResponse: Hello Gemini 
2024-02-09 08:37:17.482  5078-5078  GeminiPro               com.example.streaminggeminiapp       D  getResponse: com.google.ai.client.generativeai.type.Content@af5dd38
2024-02-09 08:37:17.487  5078-5078  GeminiPro               com.example.streaminggeminiapp       D  getResponse: kotlinx.coroutines.reactive.FlowAsPublisher@5667111
2024-02-09 08:37:18.579  5078-5158  GeminiPro               com.example.streaminggeminiapp       D  onError: * GenerateContentRequest.contents: contents is not specified
2024-02-09 08:37:18.589  5078-5158  AndroidRuntime          com.example.streaminggeminiapp       E  FATAL EXCEPTION: DefaultDispatcher-worker-2
                                                                                                    Process: com.example.streaminggeminiapp, PID: 5078
                                                                                                    com.google.ai.client.generativeai.type.ServerException: * GenerateContentRequest.contents: contents is not specified
                                                                                                    
                                                                                                    	at com.google.ai.client.generativeai.internal.api.APIControllerKt.validateResponse(APIController.kt:173)
                                                                                                    	at com.google.ai.client.generativeai.internal.api.APIControllerKt.access$validateResponse(APIController.kt:1)
                                                                                                    	at com.google.ai.client.generativeai.internal.api.APIController$generateContentStream$$inlined$postStream$1$1$1.invokeSuspend(APIController.kt:153)
                                                                                                    	at com.google.ai.client.generativeai.internal.api.APIController$generateContentStream$$inlined$postStream$1$1$1.invoke(Unknown Source:8)
                                                                                                    	at com.google.ai.client.generativeai.internal.api.APIController$generateContentStream$$inlined$postStream$1$1$1.invoke(Unknown Source:4)
                                                                                                    	at io.ktor.client.statement.HttpStatement.execute(HttpStatement.kt:50)
                                                                                                    	at io.ktor.client.statement.HttpStatement$execute$1.invokeSuspend(Unknown Source:15)
                                                                                                    	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
                                                                                                    	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
                                                                                                    	at kotlinx.coroutines.EventLoop.processUnconfinedEvent(EventLoop.common.kt:68)
                                                                                                    	at kotlinx.coroutines.DispatchedTaskKt.resumeUnconfined(DispatchedTask.kt:245)
                                                                                                    	at kotlinx.coroutines.DispatchedTaskKt.dispatch(DispatchedTask.kt:163)
                                                                                                    	at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:474)
                                                                                                    	at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl(CancellableContinuationImpl.kt:508)
                                                                                                    	at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl$default(CancellableContinuationImpl.kt:497)
                                                                                                    	at kotlinx.coroutines.CancellableContinuationImpl.resumeWith(CancellableContinuationImpl.kt:368)
                                                                                                    	at kotlinx.coroutines.ResumeAwaitOnCompletion.invoke(JobSupport.kt:1406)
                                                                                                    	at kotlinx.coroutines.JobSupport.notifyCompletion(JobSupport.kt:1497)
                                                                                                    	at kotlinx.coroutines.JobSupport.completeStateFinalization(JobSupport.kt:325)
                                                                                                    	at kotlinx.coroutines.JobSupport.finalizeFinishingState(JobSupport.kt:242)
                                                                                                    	at kotlinx.coroutines.JobSupport.tryMakeCompletingSlowPath(JobSupport.kt:910)
                                                                                                    	at kotlinx.coroutines.JobSupport.tryMakeCompleting(JobSupport.kt:867)
                                                                                                    	at kotlinx.coroutines.JobSupport.makeCompletingOnce$kotlinx_coroutines_core(JobSupport.kt:832)
                                                                                                    	at kotlinx.coroutines.AbstractCoroutine.resumeWith(AbstractCoroutine.kt:100)
                                                                                                    	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
                                                                                                    	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
                                                                                                    	at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
                                                                                                    	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103)
                                                                                                    	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
                                                                                                    	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
                                                                                                    	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
                                                                                                    	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
                                                                                                    	Suppressed: java.lang.NullPointerException: Can't toast on a thread that has not called Looper.prepare()
                                                                                                    		at com.android.internal.util.Preconditions.checkNotNull(Preconditions.java:167)
                                                                                                    		at android.widget.Toast.getLooper(Toast.java:212)
                                                                                                    		at android.widget.Toast.<init>(Toast.java:197)
                                                                                                    		at android.widget.Toast.makeText(Toast.java:583)
                                                                                                    		at android.widget.Toast.makeText(Toast.java:570)
                                                                                                    		at android.widget.Toast.makeText(Toast.java:544)
                                                                                                    		at com.example.streaminggeminiapp.MainActivity$1.onError(MainActivity.java:57)
                                                                                                    		at com.example.streaminggeminiapp.GeminiPro$1.onError(GeminiPro.java:66)
                                                                                                    		at kotlinx.coroutines.reactive.FlowSubscription.flowProcessing(ReactiveFlow.kt:215)
                                                                                                    		at kotlinx.coroutines.reactive.FlowSubscription.access$flowProcessing(ReactiveFlow.kt:187)
                                                                                                    		at kotlinx.coroutines.reactive.FlowSubscription$flowProcessing$1.invokeSuspend(Unknown Source:14)

[davidmotson]

Oh, I think you're right, I see the problem now, I'll get this fixed asap.

override fun generateContentStream(vararg prompt: Content): Publisher<GenerateContentResponse> =
      model.generateContentStream().asPublisher()

A slight oversight, but as you can see the prompt is not actually passed through to the generateContentStream method here, leading to this error.

While I'll fix it right away, the change may not be released immediately, if this is blocking for you, then my recommendation would be use kotlin's extension methods or maybe a static method to fix this within your own codebase:

     fun generateContentStream(model: GenerativeModel, vararg prompt: Content): Publisher<GenerateContentResponse> =
      model.generateContentStream(*prompt).asPublisher()

[ImadSaddik]

Thank you, @davidmotson, and I am glad that I help to raise this issue. As you saw, I was using Java to interact with the Gemini API and you showed me a solution using Kotlin. I would to try it, can you show me how to implement this ?

[davidmotson]

Ah, so this kotlin code is from the shim we have to allow interactions in java. If you make a new kotlin file in your project named, for example: "GenerativeModelFuturesExtension.kt" and throw this code in there, either as an extension function, or a static one, you should be able to call it from java, for example:

package com.google.ai.client.generativeai.java

import com.google.ai.client.generativeai.type.Content
import com.google.ai.client.generativeai.type.GenerateContentResponse
import kotlinx.coroutines.reactive.asPublisher
import org.reactivestreams.Publisher

class GenerativeModelFuturesExtension {
  companion object {
    fun generateContentStream(
      model: GenerativeModelFutures,
      vararg prompt: Content
    ): Publisher<GenerateContentResponse> =
      model.getGenerativeModel().generateContentStream(*prompt).asPublisher()
  }
}

can be called with:

GenerativeModelFuturesExtension.generateContentStream(model, new Content.Builder().addText("Some prompt goes here").build()).subscribe(new Subscriber<GenerateContentResponse>() {
  //implementation goes here
});

You'll likely need to change some stuff, like the package, to match where you've put this code, but it should be fairly similar to copying some java code into your codebase.

[ImadSaddik]

Thanks, I have tried yesterday to implement what you shared with me, but I had to add more dependencies and I still have some problems. I will try my best to see if I can make it work, and then I will post the solution here, maybe someone might find it helpful.

[ImadSaddik]

I am glad that I was able to implement what you suggested to me @davidmotson. Here is how I have done it.

First, if initially the project in Android Studio was a Java project then in the build.gradel.kts add the following dependencies

//    Dependencies for the Gemini API library
    implementation("com.google.ai.client.generativeai:generativeai:0.1.2")
    implementation("com.google.guava:guava:31.0.1-android")
    implementation("org.reactivestreams:reactive-streams:1.0.4")

//    Dependencies for the Kotlin Coroutines library (This is what you should add)
    implementation("androidx.core:core-ktx:+")
    implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.7.1")
    implementation("org.jetbrains.kotlinx:kotlinx-coroutines-reactive:1.7.3")

After that, create the GenerativeModelFuturesExtension Kotlin class and put the following code inside it

package <your-package-name>

import com.google.ai.client.generativeai.java.GenerativeModelFutures
import com.google.ai.client.generativeai.type.Content
import com.google.ai.client.generativeai.type.GenerateContentResponse
import kotlinx.coroutines.reactive.asPublisher
import org.reactivestreams.Publisher

class GenerativeModelFuturesExtension {
    companion object {
        fun generateContentStream(
                model: GenerativeModelFutures,
                vararg prompt: Content
        ): Publisher<GenerateContentResponse> =
                model.getGenerativeModel().generateContentStream(*prompt).asPublisher()
    }
}

At this point, if Android Studio is telling you that Kotlin is not configured, then click on the configure button as shown in the image below

Then click OK

Now, after the Kotlin configuration, make sure that the correct version of JVM is used. In my case, it was set to 17 then I changed it back to 1.8 because I am using JAVA 1.8

android {
    kotlinOptions {
        jvmTarget = "1.8"
    }
}

Now, you can use the Kotlin class you created like this

public void getResponse(String query, ResponseCallback callback) {
        GenerativeModelFutures model = getModel();

        Content.Builder contentBuilder = new Content.Builder();
        contentBuilder.setRole("user");
        contentBuilder.addText(query);
        Content content = contentBuilder.build();

        Log.d("GeminiPro", "getResponse: " + query);
        Log.d("GeminiPro", "getResponse: " + content);

        final String[] fullResponse = {""};

        Publisher<GenerateContentResponse> streamingResponse = GenerativeModelFuturesExtension.Companion.generateContentStream(model, content);

        streamingResponse.subscribe(new Subscriber<GenerateContentResponse>() {
            @Override
            public void onSubscribe(Subscription s) {
                s.request(Long.MAX_VALUE);
            }

            @Override
            public void onNext(GenerateContentResponse generateContentResponse) {
                String chunk = generateContentResponse.getText();
                fullResponse[0] += chunk;
                callback.onResponse(fullResponse[0]);
                Log.d("GeminiPro", "onNext: " + chunk);
            }

            @Override
            public void onError(Throwable t) {
                t.printStackTrace();
                callback.onError(t);
                Log.d("GeminiPro", "onError: " + t.getMessage());
            }

            @Override
            public void onComplete() {
                System.out.println(fullResponse[0]);
                callback.onResponse(fullResponse[0]);
                Log.d("GeminiPro", "onComplete: " + fullResponse[0]);
            }
        });
    }

In the getResponse method, I am passing a ResponseCallback interface that will help me send the response bakc to the MainActivity class. Here is how it looks like

public interface ResponseCallback {
    void onResponse(String response);
    void onError(Throwable throwable);
}

And here is how I handle the response in the MainActivity

model.getResponse(query, new ResponseCallback() {
                @Override
                public void onResponse(String response) {
                    runOnUiThread(() -> {
                        responseTextView.setText(response);
                        progressBar.setVisibility(View.GONE);
                    });
                }

                @Override
                public void onError(Throwable throwable) {
                    runOnUiThread(() -> {
                        responseTextView.setText("Error: " + throwable.getMessage());
                        progressBar.setVisibility(View.GONE);
                    });
                }
            });

One important note, to update the views, you should use the runOnUiThread(). If you don't, you will get an error because you will be trying to update the UI from a non-UI thread.

That's it, I hope this helps.

[ImadSaddik]

Hey @davidmotson,

Can you please look at issue #257 ? Regarding this issue, I will consider that it was solved and I will close it.