Add support for streaming response body

[mspanc]

I am implementing a Phoenix app that has controller that effectively acts as a HTTP proxy. It fetches file over HTTP from the upstream server (in chunks) and sends it back to the client.

At the moment I am sending chunked response using Plug.Conn.send_chunked/2 and Plug.Conn.chunk/2.

The issue is that that sets Transfer-Encoding to chunked and according to the HTTP/1.1 RFC that causes Content-Length header to be obsolete. If it get further proxied through nginx, this header is just removed to match the RFC.

As I am serving audio/video to the web browsers that effectively causes them to disable seeking as total file length is unknown.

At the moment the only workaround is to download the whole from the upstream and send it to the client, but I want to do this on the fly as there's no reason to introduce this extra step.

It would be great if there would have been another interface allowing to build response body from parts but send it as regular (non-chunked) response.

[josevalim]

@mspanc just a quick question: does Plug.Conn.send_file has any use to you? Other than that, it seems we should allow a write mode without chunking but we also need to check if such is supported in at least Cowboy.

[ericmj]

Streaming is supported without chunking with cowboy_req:set_resp_body_fun/2.

[mspanc]

Here it is: https://github.com/ninenines/cowboy/blob/06e1e2be688e74e4644dbd25a29f863d81e4cbaa/examples/chunked_hello_world/src/toppage_handler.erl

[ericmj]

@mspanc That is for cowboy master, there is no stream_reply on stable.

[mspanc]

@josevalim Plug.Conn.send_file at the moment is useless for me, as that would have required to fetch whole file first to the temp storage (and file may have a few GB).

cowboy_req:set_resp_body_fun/2 is not very convenient, as you have to wrap the whole process of generating the body into one function call, and for example HTTPoison is sending messages. This does not fit very well.

[josevalim]

It seems your best option for now is to fallback to a custom cowboy handler and use cowboy_req:set_resp_body_fun/2, even if awkward, or deploy those routes in particular using a separate application using cowboy master. We will leave this open so we make this a first class experience though when Cowboy 2.0 is out.

[bsuh]

https://gist.github.com/bsuh/295759580d36fe42a533abb733cb912c

It seems like we have similar use cases @mspanc . I was able to streaming reverse proxy, but the performance is quite bad. Using it to reverse proxy a 1 GB file (for testing) results in ~500KB/s with one CPU core throttled. Even just fetching the file with HTTPoison into memory seems to result in high CPU. There's probably excessive data copying going on due to message passing. Also, the upstream request keeps happily consuming CPU resources even if the downstream client closed the socket. Maybe I need to be using Cowboy's loop handlers ninenines/cowboy#312?

Go's httputil.NewSingleHostReverseProxy happily reverse proxies at full speed while taking up 5% CPU.

Plug.Conn.send_file's performance seems to be on par with Go's http.ServeFile function.

I also might be missing something as I'm quite new to Elixir and Erlang. I was going to replace my Go app with a Phoenix app, starting with a reverse proxy and incrementally rewriting functionality into Elixir. Seems like I'll have to keep around a bit of Go :/

[bsuh]

wget         0.58 real         0.01 user         0.16 sys
httpc        18.25 real         1.93 user         1.54 sys
ibrowse       31.79 real         3.61 user         2.77 sys
hackney      15.36 real         0.78 user         0.51 sys
lhttpc        4.21 real         0.48 user         0.43 sys
fusco did not finish for my patience
shotgun       4.30 real         0.84 user         0.67 sys
httpoison       4.99 real         0.77 user         0.44 sys

:inets.start
:httpc.request('http://localhost:8080/test')

:ibrowse.start
:ibrowse.send_req('http://localhost:8080/test', [], :get); nil

:hackney.start
{:ok, status_code, headers, client} = :hackney.request(:get, "http://localhost:8080/test", [], "", []); nil
:hackney.body(client); nil

:lhttpc.start
:lhttpc.request('http://localhost:8080/test', :get, [], :infinity); nil

{:ok, client} = :fusco.start('http://localhost:8080', [])
{:ok, result} = :fusco.request(client, "/test", "GET", [], []); nil

:shotgun.start
{:ok, conn} = :shotgun.open('localhost', 8080)
{:ok, resp} = :shotgun.get(conn, '/test'); nil

HTTPoison.start
HTTPoison.get! "http://localhost:8080/test"; nil

Good news. I did some unscientific testing of various Erlang/Elixir HTTP clients downloading a 100 MB file and decided to try lhttpc based on the results, and it works very nicely!

Here's the link again to the gist with updated code. Note that it reads private variables from Plug.Conn to pass to cowboy functions and updates the Plug.Conn manually, so it might break with dependency updates. I'm using plug 1.2.2, cowboy 1.0.4, phoenix 1.2.1, lhttpc 1.5.0
https://gist.github.com/bsuh/295759580d36fe42a533abb733cb912c

[nhooyr]

@bsuh unrelated to this issue sorry, but what made you leave golang and pick up phoenix?

[bsuh]

@nhooyr Experimentation to try out the BEAM VM that I've heard a lot about and personal preference. Go is a bit low level and bare bones (the language features not the standard library which is fantastic) for me. My projects are more plain web apps gluing libraries together and Go seems to excel more at building infrastructure tooling and network protocolish stuff.

[OvermindDL1]

network protocolish stuff

Erlang/Elixir's bit-syntax makes building network protocols almost blissfully easy. ;-)

[josevalim]

Folks, Plug master supports Cowboy 2.0 which means we can make this feature available. PRs are welcome.

[ringods]

Can we extend this issue to also support chunked upload? I would like to bypass the Plug.Upload mechanism completely.

[ericmj]

@ringods It's already possible to send chunked responses, see https://hexdocs.pm/plug/Plug.Conn.html#chunk/2. This issue is about streaming responses without using chunked encoding.

[ringods]

@ericmj I might be confused, but are you talking about sending responses back to the client with that function?

I am after an alternative for uploading files from the client in the request handling. The file handling in Plug.Parsers gives you a Plug.Upload struct at the moment. My own route handler is only invoked after the Plug.Parsers have done their work. I would like to process the request body in a streaming way myself, specific to a few of my Phoenix routes.

According this gist:

Chunking is a 2 way street. The HTTP protocol allows the client to chunk HTTP requests. This allows the client to stream the HTTP request. Which is useful for uploading large files.

So I'm investigating how the HTTP messaging is done from client to server hoping to get an Elixir Plug implementation accepting chunked file uploading.

To avoid confusion: I'm not talking about mutipart uploads. ;-)

[josevalim]

@ringods you can already bypass Plug.Upload. You can plug your own parsers into Plug.Parsers (or not use Plug.Parsers at all and parse the request in a previous plug).

[josevalim]

You can also give :pass to plug parsers to specify that some mime types will be handled by your application.

[ringods]

Indeed all possible, but since I'm running in circles on how to implement this myself, I posted it on the forum. Some further input is welcome. ;-)

https://elixirforum.com/t/making-a-stream-out-of-the-http-upload-body-content-towards-genstage/13544

[ericmj]

After discussion with @essen on IRC it turns out this is not supported in cowboy2 on http1.1 connections, so I will look into contributing it to cowboy.

When investigating this I also noticed that it looks like the streaming is broken in the cowboy2 adapter because it never sends fin [1] so we may have to add new API for this. /cc @Gazler

[1] https://github.com/elixir-plug/plug/blob/master/lib/plug/adapters/cowboy2/conn.ex#L62

EDIT: Turn out I was wrong about fin, cowboy handles it for us.

[ericmj]

Cowboy 2.5.0 has added support for non-chunked streaming in http1.1. You need to set the content-length header before calling Plug.Conn.send_chunked/2`.

ninenines/cowboy@f08f461

[josevalim]

Oh, awesome. Let's reopen this so we update the docs and then we should be good to go.

[mspanc]

This change in cowboy is not resolving the original issue. The idea was to stream infinite file (such as MP3 stream) without sending content length and without using chunked encoding, similarly to how Icecast2 works but it can be used in other proxy scenarios.

Given that if there's an agreement that such feature should be present the issue IMO should be reopened.

[josevalim]

@mspanc thanks for the context. However, I don't think there is nothing to be done on Plug's side to support this feature, it all depends on Cowboy? So I think this need to be supported on Cowboy side and then we can resume the discussion if anything is necessary to support it on Plug.

[essen]

You can disable chunked on a per-response basis and if you don't send a content-length then Cowboy will just stream until the end of the body before closing the connection (or forever if there's no end), see https://github.com/ninenines/cowboy/blob/master/src/cowboy_http.erl#L1193-L1198