Stream for SKFunction

[VladTolm]

Hi,

Why was it decided to make the complete_chat_stream_async function in OpenAIChatCompletion class and not in SKFunction? This approach breaks the whole concept of working with skills.

Maybe it makes sense to make the invoke_stream_async function in the SKFunction class?

#python

[shawncal]

Fear not, @VladTolm, this is coming soon!
Current plan: it won't be just another method on ISKFunction itself, but rather an implementation of the Observer pattern -- you can register callbacks for PreStep, PostStep, OnRunning, and OnError events (not their actual names). OnRunning will expose a hander to watch the results stream back, if the function supports it. This allows the caller to get the streamed results without getting in the way of the function chain (Plan) being run.

As part of this work, for native functions, we're looking to expose a Response StreamWriter to the function (via SKContext) so that it can opt to write results back incrementally and the observer can watch them (modeled after similar implementations in ASP, Azure Functions).

And, of course, all of this will be brought to Python and Java kernels too, using constructs that are similar in intent, but language-appropriate.

[OliverUlm]

Looking forward to this.

[MTCMarkFranco]

any update on this? currently trying to invoke_stream on a semantic function in SK in Python and although I get no errors, I also get no results over the AsyncGenerator object when using:

"async for message in answer:"

Thoughts?

[moonbox3]

@MTCMarkFranco what version of SK Python are you using? Are you running a pip package or running from latest code in main via a poetry venv? We've just merged a giant feature branch to main which close many large/breaking change work items to get us to a beta version within the next week. Streaming chat is working as of these latest changes (both from kernel examples and integration tests).

[evchaki]

https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/KernelSyntaxExamples/Example32_StreamingCompletion.cs