Streamed completion and function calls

[sebrosse]

Hey,

I was wondering whether it is possible to combine streamed response and function calls ?

I did a few tests but couldn't get the function call to work as the streamed response format is quite different from the "standard" completion response.

Any idea how to do this ? is it even possible with the current API/Library ?

Thanks

[gehrisandro]

Hi @sebrosse

Yes that's possible.

You can get the function call information from the streamed response. Here's an excerpt of some code I am using:

foreach($response as $response) {
            $delta = $response->choices[0]->delta->functionCall->arguments ?? '';
}

Be aware, the arguments is returned as JSON and in a stream you receive the response part by part. So the JSON is only valid when the complete response is received.

But if you want to wait for the response to complete you could do the call without streaming. Therefore I am using a regex to parse parts from the response before the JSON is complete.

[sebrosse]

Hey @gehrisandro

Thanks a lot for getting back to me so quickly, you pointed me in the right direction !

I've been playing with the API and I noticed the first chunk received "always" contains the name of the function when ChatGPT wants to trigger a function call.

I believe I'm going to be checking the first chunk and based on whether it contains the function name, I'll execute the function or directly stream the response to the user. Do you see a risk with such approach ? Do you believe it's likely the order of the chunks sent might change frequently ? I don't believe that aspect is documented.

Cheers

[gehrisandro]

You can enforce the function calling by passing the name of the function here: https://platform.openai.com/docs/api-reference/chat/create#chat/create-function_call

[vesper8]

@sebrosse were you able to get this working? I believe that the answer from the function call should not be returned directly but rather it should be fed back to the LLM for interpretation. This is the part that we're all struggling to get working using this package.