How to use the streaming_callback callback function of OpenAIChatGenerator to output sse or websocket to the frontend? #8160
Unanswered
springrain
asked this question in
Questions
Replies: 1 comment
-
|
This is hard to achieve in the single threaded request/chunk response execution model of OpenAIChatGenerator. You need to launch another thread for each request or wait for async implementation of OpenAIChatGenerator which should be coming soon. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
How to use the streaming_callback callback function of OpenAIChatGenerator to output sse or websocket to the frontend? Can you provide an example of a test? I'm having some problems with the implementation, thank you very much.
Beta Was this translation helpful? Give feedback.
All reactions