feat: support server sent event

[kemingy]

close #273

reference:

• https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events
• https://html.spec.whatwg.org/multipage/server-sent-events.html#server-sent-events
• https://github.com/sysid/sse-starlette/blob/master/sse_starlette/sse.py

notes:

• event streams are always decoded as UTF-8
• text/event-stream MIME type
• Python context switch interval https://superfastpython.com/context-switch-interval-in-python/

[kemingy]

@lkevinzc It's ready to review.

[lkevinzc]

Since both normal server and sse server will fetch request from this protocol_recv, chances are that sse server will get data from /inference request when clients query the wrong endpoint. In this case, the server will process the data but the client will get nothing.

Similarly, if the client query a normal server with /sse_inference, weird thing will happen.

We'd better to handle this by sending error for now; for future we should not have both endpoints exposed by default, but use multi-route kind of feature to support the specified endpoint only when the server implements it.

[kemingy]

1. impl sse but called /inference => cannot find the stream event channel (info log), but return nothing to the client

2. impl inference but called /sse_inference => processed but return nothing

Shall we return an error if it cannot find the stream event channel? This might not be 100% accurate.

Not sure how to deal with the 2. under the current design.

[lkevinzc]

Is it possible to do a check at server side, knowing whether it is a normal sever or sse server, then send this info to rust side. In doing so, the rust side will know whether it should reject the requests from non-implemented route. Another benefit is that these rejected requests will never go to python side for processing.

A simple possible implementation is to let users set a class attribute to the sse worker? Only when it is set the send_stream_event can be called normally.

[kemingy]

This might need to implement an SSE worker class. But passing the type of worker might require a more complex config file than the current command line arguments.

[lkevinzc]

As discussed, we won't release until some refactoring is done and the issue mentioned in the comment is resolved. But we merge this first.