-
-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Streaming over http seems to be broken in v1.11 #624
Comments
Hey, thanks for reporting. This sounds like a real regression indeed. In 1.11.0, we changed the way rendering is done, and I may have introduced this regression. Can you try to replace the rendering pipe line with .through(fs2.data.text.render.pretty(0)(fs2.data.json.Token.compact)) And let me know if it fixes it? |
I am investigating the issue, and added a test (see #627). It looks like the Can I see how the resulting stream is used in tapir? Can you confirm that you still get several chunks right after the |
I wrote a little test program that tries to come as close as possible to the original report, but without tAPIr: import cats.effect.*
import fs2.*
import fs2.data.json.*
import io.circe.Json
import scala.concurrent.duration.*
object EndlessMain extends IOApp.Simple {
override def run: IO[Unit] =
countingJson
.meteredStartImmediately(0.001.seconds)
.through(ast.tokenize)
.through(wrap.asTopLevelArray)
.through(render.compact)
// .through(text.utf8.encode)
.evalMap(IO.println(_))
.compile
.drain
def countingJson: Stream[IO, Json] =
Stream.iterate(0)(_ + 1).map(Json.fromInt)
} When run, it outputs immediately, like this:
And as the |
Hi @thomasl8, did you have a chance to look into this issue? |
just following along, also ran into this with |
Hello,
We use this library (the circe version) to stream a large JSON payload over HTTP.
It has been working great with version 1.10, however when we upgrade to version 1.11 the server seems to want to collect the entire payload before returning any data to the UI (so basically not streaming). It could be that we are using the library incorrectly for our purposes. If it makes a difference, we are using
tapir
as our HTTP library.This is our code (some stuff renamed). Maybe we are doing something wrong? Any help will be much appreciated! Thanks.
myService .getBigFs2Stream .through(ast.tokenize) .through(wrap.asTopLevelArray) .through(render.compact) .through(text.utf8.encode) .pure[IO]
The text was updated successfully, but these errors were encountered: