Unable to complete request when response transformer plugin is active

[fpahl]

When I add the response transfomer plugin to an existing api the response is cut off.
Example:

curl -X POST localhost:8001/apis/ --data 'name=users' --data 'upstream_url=http://jsonplaceholder.typicode.com/users' --data 'request_path=/users' --data 'strip_request_path=true'
curl -X POST localhost:8001/apis/users/plugins --data 'name=response-transformer'
curl  localhost:8000/users

When I call the API I get the following error:

HTTP/1.1 200 OK                                              
Date: Fri, 13 May 2016 12:25:47 GMT                          
Content-Type: application/json; charset=utf-8                
Content-Length: 5645                                         
Connection: keep-alive                                       
Server: Cowboy                                               
X-Powered-By: Express                                        
Vary: Origin, Accept-Encoding                                
Access-Control-Allow-Credentials: true                       
Cache-Control: no-cache                                      
Pragma: no-cache                                             
Expires: -1                                                  
X-Content-Type-Options: nosniff                              
Etag: W/"160d-MxiAGkI3ZBrjm0xiEDfwqw"                        
Via: kong/0.6.1                                              
X-Kong-Upstream-Latency: 2292                                
X-Kong-Proxy-Latency: 24                                     

curl: (18) transfer closed with 1551 bytes remaining to read 

The header is returned immediately. The error message appears after some time.

I have testeted this with kong 0.6.1, 0.7 and 0.8.1 and with different upstream urls.

[ErwannRobin]

Maybe try to deactivate the "Accept-Encoding: gzip" header in your query?
see: #1200

[fpahl]

Disabling the compression did not fix the issue, e.g.:

curl -v -H "accept-encoding: identity" localhost:8000/users/1

I guess it is a problem with the body parser. When I comment the function ResponseTransformerHandler:body_filter in the file kong/plugins/response-transformer/handler.lua the requests completes successful.

[fpahl]

Does anyone else experience the same problems?
This is a critical issue for us...

[Tieske]

Fixed in #1263. The problem occurred due to whitespace changes. The header filter checked whether the body would be adjusted, and only if so, it would alter the content length. The body filter would NOT check whether there was anything to do, it would just parse the json, and re-encode the result.

So in this case, the content length was unaltered, and the body just got re-jsonified with different whitespace settings. Causing a mismatch between the body and the set body length. Which caused the delay and timeout on the curl example which remained waiting for the missing whitespace bytes.

thanks for your help in reporting and resolving!

[Tieske]

@johnpaul2 as a side note;

curl: (18) transfer closed with 1551 bytes remaining to read

This means the upstream server "wasted" 1551 bytes in a single request on whitespace. You might want to reconfigure that.