Bidi streaming use cases for grpc-web

[wenbozhu]

Options, use cases to enable bidi streaming support with grpc-web

[arthur-tacca]

As I understand it you're interested in possible use cases for streaming? Here's one (but suggestions for better alternative implementations are welcome).

Use case: Returning pages in a multipage view (e.g. list of threads in forum software)

In this use case the front-end JS opens a stream and sends a request (e.g. GetThreads(forum="General", page=0)) and gets a response containing only one page of results. On the server side, a cursor into the database query results (e.g. using PostgreSQL) is kept open, and on future requests in the stream (e.g. GetThreads(forum="General", page=1)) this cursor is used to get the results. This ensures a consist view of data (e.g. you don't see the same thread on page 2 as you just saw on page 1). When the stream is closed, the database cursor is closed. Without streams you would need to add logic about when to keep open or close database cursors, and under load balancing you need to find a way to make sure requests from the same client end up at the same server.

[johanbrandhorst]

With the grpc-gateway wrapped in a websocket proxy (https://github.com/tmc/grpc-websocket-proxy) it's currently possible to do "bidi streaming" from the browser. Is using websockets for bidirectional streaming completely out of the question for grpc-web? I realise it would mean that any grpc-server would need to monitor for websocket connections, but maybe it could be a stepping stone before full trailer support is in the browsers?

[wenbozhu]

@johanbrandhorst

Thanks for the point.

We are in the process of launching grpc-web as an alpha release, and will publish the plan on bidi support (road-map) after Q2.

[HTChang]

@wenbozhu Do the grpc-web now support bidi streaming in the example?
I saw the proto has defined bidi streaming API

  rpc FullDuplexEcho(stream EchoRequest) returns (stream EchoResponse);

but the doc of Known gaps / issues says

No client-streaming or bidi-streaming support

One of our use case would be in the search bar, customer will enter the keyword sequentially and the search request/response will be implemented via bidi streaming

[wenbozhu]

@HTChang no, the grpc-web repo version doesn't support bidi streaming. We probably should add a comment to the proto too to avoid confusion.

@stanley-cheung

[jonahbron]

@wenbozhu Have there been any developments on adding bidi support to the gRPC-Web spec?

[wenbozhu]

We don't yet have a future-proof transport story that allows us to provide cross-browser support and at same time serve both OSS and internal users. Will publish the road-map doc soon, and we are very close to making the repo public.

[jonahbron]

@wenbozhu I'm sure you've heard people scream "but websockets!" at you a million times. But I'm genuinely curious, can you explain or link to an article or something that explains why Websockets shouldn't be used here?

[johanbrandhorst]

grpc/grpc#8682

[jonahbron]

Thanks @johanbrandhorst. Based on the comments on that issue (and the presence of a vote item in the ROADMAP.md), it sounds like it's still a possibility. Guess we'll just need to wait until the roadmap is updated with a decision.

[johanbrandhorst]

Indeed :). I'll be watching as eagerly as you are!

[wenbozhu]

Thanks for reading the ROADMAP.

This slide deck talked about websockets v.s. pure HTTP, with some stats from chrome.

We understand the desire to design a single API that works for both micro-service and Internet clients, among a few other use cases that could truly benefit from full-duplex streaming.

[DieMyst]

Hello, I want to write about my use case of bidi streaming.
We are in the process of writing decentralized database.
So, all data is encrypted and client must follow the b-tree with encrypted keys step-by-step to decrypt each key and say in which direction should be next step.
We had good start in implementation this logic with grpc, but browser implementation without some wrappers (websocket, etc) of client is difficult now without bidi support.
Our goal is the ability to use a decentralized database via browser having only a private key available.

[johanbrandhorst]

Note that Imorobables grpc-web implementation has a beta bidi implementation over websockets. https://github.com/improbable-eng/grpc-web.

…

On Wed, Mar 28, 2018, 20:37 Dima ***@***.***> wrote: Hello, I want to write about my use case of bidi streaming. We are in the process of writing decentralized database. So, all data is encrypted and client must follow the b-tree with encrypted keys step-by-step to decrypt each key and say in which direction should be next step. We had good start in implementation this logic with grpc, but browser implementation without some wrappers (websocket, etc) of client is difficult now without bidi support. Our goal is the ability to use a decentralized database via browser having only a private key available. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#24 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGTFd-ucgkhhsClxjZPA0ogZ-AcX6fwNks5ti-ZhgaJpZM4K6Cd6> .

[shanshanzhu]

@johanbrandhorst the improbable beta bidi version is really premature. It is not documented and has no example code. All it does is just exporting a transport function with websocket, but doesn't really wire it up with grpc. If I miss something (i.e. branch) there, could you help point me to the right place of code? Thanks!

https://github.com/improbable-eng/grpc-web/blob/709b592779c35e17f581f9bf866f4cac4b0b93fb/ts/src/transports/Transport.ts#L64

[johanbrandhorst]

@shanshanzhu the websocket proxy implementation is good. It wraps the Go http.ResponseWriter and io.Reader interfaces with websocket read/writes and passes the request to the gRPC Go implementation, so the wire protocol is pure gRPC-over-http2-over-websockets. Take a look at the implementation, it's beautiful 😍 https://github.com/improbable-eng/grpc-web/blob/72eb701d6f320ca324b3347c7925a720b553eae5/go/grpcweb/wrapper.go#L133, https://github.com/improbable-eng/grpc-web/blob/709b592779c35e17f581f9bf866f4cac4b0b93fb/ts/src/transports/websocket.ts#L51.

Furthermore, as I already pointed out in improbable-eng/grpc-web#205, you can look at the tests for an example of how to use it: https://github.com/improbable-eng/grpc-web/blob/master/test/ts/src/client.websocket.spec.ts#L77.

The GopherJS gRPC-Web bindings have been using the websocket proxy on https://grpcweb.jbrandhorst.com since the start of the year.

Having said all of this, I think it is probably right to wait for Fetch API and Streams API to land in browsers before implementing this in this client.

[wenbozhu]

Re: websockets, we would welcome all the gateways that provide inter-operability with gRPC end-points.

WS over HTTP/2 is being finalized and will be available in Chrome. We will evaluate if it makes sense to make WS a transport for grpc-web (but the bar to deploy this for Google's web apps will be high) and there will be details such as ping-pong, half-close etc to spec out ...

[nhooyr]

@johanbrandhorst does this mean there is a client side grpc-http2 implementation thats reading everything from the web sockets?

[nhooyr]

@wenbozhu My use case for bidirectional streaming is for an online game where the client is sending message based on what the user is doing and the server is sending messages to clients to broadcast all events. I could just do this over plain web sockets with protobuf but it'd definitely be nice to see support for this. Can we implement something similar to what @johanbrandhorst noted in improbable's grpc-web repo? It seems like it'd be really simple given its just a wrapper around the protocol and we could document that bidirectional streaming is done over web sockets. Later on, with WS over HTTP/2, we could enable support for that and then later for the streams API. This approach would ensure broad browser compatibility.

[johanbrandhorst]

I don't think we should implement this in this repo until we have request streaming in Fetch. There is a third party, non-spec implementation that users can use if they need bidi streaming before then.

[nhooyr]

@johanbrandhorst the issue with that is Fetch will only work on bleeding edge browsers. I think its a pragmatic idea to include WebSockets over HTTP/1.1 as a fallback.

[johanbrandhorst]

I disagree. This will by necessity be bleeding edge tech. The spec talks about eventually speaking the HTTP2 protocol, which is not supported by todays browsers. I don't think the aim of this project is to provide backwards compatibility to old browsers.

[wenbozhu]

grpc-web does intend to support important browser use cases, i.e. features, versions, environments etc ... and the current spec is implementation details but important.

I don't necessarily agree tunneling grpc/http2 over websockets is a good or sane idea, and you are going to run into issues such as flow-control, half-close etc, to name a few.

One of the problems with websockets is that connectivity is not guaranteed (e.g. to support gmail) .. which means you always need a fallback. We don't yet want this level of complexity for grpc-web.

https://github.com/HTTPWorkshop/workshop2017/blob/master/talks/websockets.pdf

===

This bug is intended to collect use cases (title updated :) ... e.g. the case @nhooyr mentioned, could the API be actually better (e.g. resilient to failures) if client uses unary calls to post and uses server-stream to receive messages?

[johanbrandhorst]

@wenbozhu 403 on that document Im afraid

[wenbozhu]

fixed it.

[nhooyr]

I don't necessarily agree tunneling grpc/http2 over websockets is a good or sane idea, and you are going to run into issues such as flow-control, half-close etc, to name a few.

Couldn't flow control be implemented at the grpc-web level? Likewise with half close?

One of the problems with websockets is that connectivity is not guaranteed (e.g. to support gmail) .. which means you always need a fallback. We don't yet want this level of complexity for grpc-web.

In what way is the connectivity of WebSockets not guaranteed?

e.g. the case @nhooyr mentioned, could the API be actually better (e.g. resilient to failures) if client uses unary calls to post and uses server-stream to receive messages?

The issue with that would be the constant authentication checks on every single message (of which there would be a lot) when doing an RPC. A bidirectional stream completely avoids this by only authenticating the connection at the start. I guess I could give each client a token before they start a game and then they pass that token on each request and we just directly compare that token instead of hashing for performance reasons on each RPC but still its more complicated than just a bidi stream.

Furthermore, while unary RPC+server stream avoids depending on the client side streaming connection remaining open, its not really more resilient because it'd be very easy to just reopen the bidi stream and we still have to worry about the server stream closing anyway. I don't think its more resilient in my case.

[wenbozhu]

We have post a road-map doc on our 2020 plan to improve the streaming support in grpc-web. Feel free to post any questions here or file a separate issue to discuss.

@ydesgagn for websockets, you may want to contribute something under https://github.com/grpc-ecosystem first. I am happy to review any spec on grpc/websockets.

[kshcherban]

Use cases:

• stream request (audio, video, text) from client in browser (javascript) and receive back results (audio, video, text) from server (grpc-go), it can be any type of video, voice chat, speech to text or text to speech, face recognition, image detection, etc.

[chwzr]

the roadmap looks very good! For my understanding: we are waiting for https://www.chromestatus.com/feature/5274139738767360 this to be available in major browser, to see the bidi-streaming magic happening here?

I really like the idea of not using any websockets in the future, just pure grpc / http.
Please let us know if there are any updates on this :)

[chris-kruining]

upload streaming has landed in chrome 95. (https://web.dev/fetch-upload-streaming/). Any change this can be picked up, perhaps even soon?

[loeffel-io]

Webtransport support also shipped

[dunkymole]

An update on this this would be great.

[chris-kruining]

An update on this this would be great.

not as much an update as a note, I implemented a gRPC lib that uses both upload and download streaming (in my lib that is, somewhere in the stack it could still buffer, so no promises on actual duplexing, but I should be supporting it), you could take a look at that and see it if is usable for what you are doing.

A couple of notes / warnings:
This is a new project, just 2 weeks old, so expect bugs (Issues and PR's are very much welcome).
Whilst I am using this in the development for my app (and is working with what I use it for), I am the sole user as far as I know, ergo: no community.
This lib implements gRPC, not gRPC-web, beware of that!

So, use on your own risk!

gRPC lib
https://github.com/FYN-Software/grpc

protoc plugin to generate .ts files for your .proto files that is compatible with this lib
https://github.com/FYN-Software/protoc-plugin-ts

docker container running the plugin and protoc
https://hub.docker.com/r/chriskruining/protoc-all

p.s. motivation for telling all this is because I'd really love an extra pair of eyes looking at the code

[edaniels]

I did a simple proof of concept at https://github.com/jsmouret/grpc-over-webrtc More documentation to come but it is a start :)

There's https://github.com/viamrobotics/goutils/tree/main/rpc/examples/echo which has an example providing an RPC server over grpc, grpc-web, grpc-gateway and grpc-over-webrtc.

[loeffel-io]

any update?

[sampajano]

Thanks for the interest! No work is currently planned right now.. But will update if they are. Thanks! :)

[dyaskur]

So many people interested for this feature. Why is there no plan for it? May I know the reason?

[sampajano]

@dyaskur Hi thanks for the interest. I'll provide an update in the roadmap soon thanks!

[vladchuk]

Full support (including bi-di streaming) of gRPC in Flutter seems like a glaring omission indeed. And still no update on the road map - will there be such support at all?

[sampajano]

An update to the roadmap is provided in #1326. Thanks all for the patience and support! 😃

@vladchuk FYI :)

[bitnom]

need it before I start using in server apps

[pbower]

Hi, I'm very interested in this given Pyarrow's FlightRPC feature, as it opens up a number of high-performance data use cases.
https://arrow.apache.org/docs/format/Flight.html
Is the bidirectional streaming in the above roadmap (2023+) still being developed ?

Thanks!

[sampajano]

@pbower Thanks for checking! The bidirectional streaming support based on WebTransport is not currently being worked on yet but it's still planned as written. :)

We're making some changes soon to grpc-web including Typescript-ification so we'll probably get to that some time afterwards :)

[Lopamoko]

Hello! We are really looking forward to support for two-way data transfer in grpc from front-end to back-end via WebTransport. Please tell me, do you have any approximate understanding of when you will take on this? Now we cover this need using proxy: front-end -> WebSocket -> Proxy -> grpc -> BackEnd

[sampajano]

@Lopamoko Hi! Thanks for the interest and for checking! I cannot honestly give you a concrete date when this work will be started, since it's only planned but not really scheduled for this quarter or the next. Right now, i'm busy working on Typescript migration (first internal and external)which i'm hoping to finish later this year, and then I plan to improve on the set of fetch/stream performance / runtime related issues, before I have time to evaluate the bidi solution here. My optimistic hope is that I can start early next year. Hope this satisfies your question. :)

[popoolasubomi]

Hi guys, are there still any efforts to support bi-directional streaming? My use case involves a search bar with search suggestions based on text updates. I need the server to return a list of new query suggestions as the user changes text on the search bar

[sampajano]

@popoolasubomi Hi! Thanks for checking! My last comment is still mostly up-to-date status wrt bidi support. We weren't able to start any work on it yet because Typescript migration is still underway. I will update this thread when we begin to work on it. Thanks for your patience!

[popoolasubomi]

Do you have any recommendations we could use in the spare time? Till bidirectional streaming is complete? I heard about web transport with grpcs

[sampajano]

@popoolasubomi No sorry nothing from us (on web) unfortunately :) WebTransport is on our roadmap but it's not been implemented yet (it needs server (Envoy) support as well).

[oh-tarnished]

Hello all,

All our systems were written with full gRPC support. Recently, we started migrating our application to Svelte from Flutter. Midway through the process, we realized that the browser doesn't support all gRPC lifecycles. Here’s the workaround we implemented using tRPC.

Scenario:
You have a gRPC server written in any language (most of our workarounds and supported gateways are in Go). Then, there's a Node-based server (tRPC) which contains the logic for the gRPC client (Node) and connects to the tRPC client on the browser (Svelte).

Note: This isn't the best approach for scalability. Since our application is intended to run locally, it works for us. However, I’m open to working on a more scalable solution if needed.

Please feel free to try it out and see how it goes!

Demo:

GitHub Repository