emptyMessagesLimit and Stream error

[vadim-ivlev]

Hi,

Thanks for your excellent work.

Yesterday, doing calls to CreateCompletionStream() function I started getting the following error:
Stream error: stream has sent too many empty messages

Requests were like this:

{
  "model": "text-davinci-003",
  "prompt": "В чем смысл жизни?",
  "max_tokens": 512,
  "temperature": 0.7,
  "stream": true
}

Seems like it's related to
emptyMessagesLimit = 100 in stream.go.

My code:

	stream, err := c.CreateCompletionStream(context.Background(), params)
	if err != nil {
		return
	}
	defer stream.Close()

	for {
		response, err := stream.Recv()
		if errors.Is(err, io.EOF) {
			return
		}

		if err != nil {
			fmt.Printf("Stream error: %v\n", err)
			return
		}

		choices := response.Choices
		if len(choices) > 0 {
			fmt.Printf("%s", choices[0].Text)
		}
	}

I copied it from your example. Before yesterday it worked fine. Today this error seems to persist.

Could you please check it or make the variable configurable?

Thanks again.

[sashabaranov]

@vadim-ivlev bumped this limit to 300 in #72, please re-try after doing go get -u github.com/sashabaranov/go-gpt3.

You may also want to treat ErrTooManyEmptyStreamMessages as the of the stream e.g.

if errors.Is(err, gogpt.ErrTooManyEmptyStreamMessages) {
	// ... handle stream end
}

[vadim-ivlev]

Thanks for a quick response. Will try it in a few hours.

…

On Tue, 14 Feb 2023, 17:30 sashabaranov, ***@***.***> wrote: @vadim-ivlev <https://github.com/vadim-ivlev> bumped this limit to 300 in #72 <#72>, please re-try after doing go get -u github.com/sashabaranov/go-gpt3. You may also want to treat ErrTooManyEmptyStreamMessages as the of the stream e.g. if errors.is(err, gogpt.ErrTooManyEmptyStreamMessages) { // ... handle stream end } — Reply to this email directly, view it on GitHub <#71 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXDFQAJAQSV6CBTKHSB5STWXO6KFANCNFSM6AAAAAAU32KQYE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[vadim-ivlev]

Hi Sasha,
Have tested your changes.
Seems to work.
Thanks again for your kind response and the quick fix.