implement non-blocking stream server handler

[naoh87]

What this PR do

• Eliminate blocking from stream server runtime.
• align unary server runtime I/F
• fix flaky ServerSuit

ClientStreaming Benchmark

method implement

request.compile.last.map(_.fold(HelloReply())(r => HelloReply(r.request)))

Request 100 messages with ghq --cpu=4 -z 60s --connections=5 ...

#503

actual cpu used: 1.0core
Summary:
  Count:	39264
  Total:	60.00 s
  Slowest:	385.42 ms
  Fastest:	2.80 ms
  Average:	74.86 ms
  Requests/sec:	654.39

main

actual cpu used: 1.0core
Summary:
  Count:	11280
  Total:	60.00 s
  Slowest:	1.09 s
  Fastest:	7.85 ms
  Average:	265.53 ms
  Requests/sec:	188.00

[ahjohannessen]

@naoh87 I have been busy with other stuff and have not forgotten about this PR. I'll look more closely into it when I find the time.

[ahjohannessen]

@naoh87 I have not forgotten this, however upcoming work with queues in cats-effect 3.4.x might give us better building blocks.

One question, does your work here take into account back pressure? For instance, a server can't keep up with a client streaming too much.

[flipp5b]

I suppose, call.request might be impure. Wouldn't it be better to delay the effect like this:

  import cats.syntax.functor._

  def requestOnPull[F[_]](implicit F: Sync[F]): Pipe[F, Request, Request] =
    _.chunks.flatMap(chunk =>
      Stream.evalUnChunk(F.delay(call.request(chunk.size)).as(chunk))
    )

[naoh87]

It looks good. thank you.

[naoh87]

@ahjohannessen back pressure is controlled from here of call.requsetOnPull .
Server requests next messages from the client each stream chunk is pulled.
Prefetch and buffer size is declarabled by 2 lines ago. _ <- call.request(1)

[naoh87]

Server to client message back pressure implementation is still missing as main branch.

[naoh87]

I think this server to client back pressure implementation may cause performance issue.

grpc-java says free to ignore this and main branch does.
https://github.com/grpc/grpc-java/blob/v1.46.0/api/src/main/java/io/grpc/ServerCall.java#L100

I think this feature should be opt-in.

[naoh87]

I think new cats-effect 3.4.x queue will now work as good as this PR, because new CE3 queue is implemented as MPMC, but we just need SPSC.
https://github.com/JCTools/JCTools#jctools

and also new implementation does not reduce Dispatcher[F] cost at onMessage.

[ahjohannessen]

I think new cats-effect 3.4.x queue will now work as good as this PR, because new CE3 queue is implemented as MPMC, >but we just need SPSC.
https://github.com/JCTools/JCTools#jctools

and also new implementation does not reduce Dispatcher[F] cost at onMessage.

Ok. I think we should try to use as much code from cats-effect/fs2 where possible. That reduces the amount of code we have to maintain - I am thinking mostly about OneShotChannel.