/llms/stream endpoint should be text/event-stream type

[borgmon]

The content type of /llms/stream should be text/event-stream. This align with the response from OpenAI endpoint as well.

Actually, this is already causing issue with cloudflared.

When deployed with cloudflared (cloudflare tunnel), the text will be buffered until the stream ends.
It seems like setting Content-Type to will fix the issue.

We can simply change this line to fix it.

[borgmon]

I already created a PR 👍

[enricoros]

Merged #148, thanks. Hopefully this won't cause problems with other deployment providers.

Technically the output of the /llms/stream API is a progressive text (e.g. token by token, or around 3 charts at a time) and not an event stream (a very special encoded text with new lines and all), but if this workaround works then the correctness doesn't matter much.

Thanks for providing this!

[enricoros]

Closing this ticket. Not sure this is the correct technical solution, as we stream plain text and not 'events' (from our backend (edge functions) to our UI). But so far there seem to be no issues.

NOTE: the vercel/ai SDK uses text/plain, here: https://github.com/vercel-labs/ai/blob/980004b512066dda3dfe49ff1432cc72b8c08450/packages/core/streams/streaming-text-response.ts#L28C38-L28C38 -- while we now use 'text/event-stream' -- again, hopefully this diverge is no issue.