gzip filter: support chunked transfer encoding

[mjduijn]

Title: grpc-web: compress response for service methods with non-streaming response

Description:
I have a setup with a frontend making requests to our backend using grpc/grpc-web[^1] via the envoy grpc-web filter.
Currently a response to a simple request-response service method is never compressed, but this is accepted by the grpc-web client and thus should be possible.
I have enabled the envoy.gzip plugin and added application/grpc-web-text+proto and application/grpc-web-text in content_types.
The content being sent by the grpc-web protocol is Base64 encoded proto bytes[^2], meaning the reduction in response size should be significant.

The request headers:

Host: <host-name>
User-Agent: <user-agent>
Accept: application/grpc-web-text
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: <referer>
authorization: <auth-value>
Content-Type: application/grpc-web-text
X-User-Agent: grpc-web-javascript/0.1
X-Grpc-Web: 1
Content-Length: 12
DNT: 1
Connection: keep-alive
TE: Trailers

The response headers:

HTTP/2.0 200 OK
content-type: application/grpc-web-text+proto
grpc-encoding: identity
grpc-accept-encoding: gzip
x-envoy-upstream-service-time: 83
date: <date>
server: <server>
strict-transport-security: max-age=31536000
X-Firefox-Spdy: h2

The only missing requirement for the gzip filter to compress a response[^3] seems to be a missing content length. Unfortunately I can't see the response returned by the various filters (even with --log-level=trace) so I can't be sure this is what is missing.

Relevant Links

• [^1] https://github.com/grpc/grpc-web
• [^2] Unless the grpc services are compiled with mode=grpcweb but that completely disables all streaming calls (see README.md)
• [^3] https://www.envoyproxy.io/docs/envoy/latest/configuration/http_filters/gzip_filter

[htuch]

@fengli79 any thoughts on this?

[qiwzhang]

It seems that it is not easy to add content-length in grpc_web filter. encodeHeaders() doesn't have the response data. encodeData() could be called multiple times.

[htuch]

@qiwzhang this seems related to the recent #6600.

[stale]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or other activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted". Thank you for your contributions.

[fengli79]

@fengli79 any thoughts on this?

@wenbozhu
I feel this is more like getting gzip filter support chunked transfer encoding. grpc web filter may write back response multiple time, in case of response streaming. In that case, gzip filter should go ahead to compress the content without the content length, and later on, envoy may chunk them back.

[annismckenzie]

Can this be reopened?

[akan]

help wanted
same problem

[wenbozhu]

@stanley-cheung

[kim-nguyenkhn]

@mjduijn I'm facing the same problem, did you ever come to a resolution on this?

[mjduijn]

I did not, however I switched jobs and haven't used grpc-web in over a year

[anupkrish24]

@kim-nguyenkhn @wenbozhu @fengli79 @stanley-cheung Is there a way to resolve this? Would be great to have this so grpc-web responses can be compressed (or if there's another way to compress grpc-web responses, I would love to take a look). Thanks.

[sunjayBhatia]

Looks like Envoy 1.18 solved this? compression: extended the compression allow compressing when the content length header is not present. This behavior may be temporarily reverted by setting envoy.reloadable_features.enable_compression_without_content_length_header to false.: https://www.envoyproxy.io/docs/envoy/latest/version_history/v1.18.0#new-features