Fix intermittent chunked response hang #746
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When streaming a chunked response, it is possible to cause a TCP receive hang under particular circumstances. If at one point the parser buffer doesn't have the whole chunk, at a later point the buffer ends up empty `<<>>`, and subsequently `hackney_response:stream_body/1` is called, `hackney_response:recv/2` will hang if the expected remaining size exceeds the remainder of the response. That expected size is actually stale, from the earlier point when the parser did not have the whole chunk. This issue slipped in with benoitc#710. This was identified when using https://github.com/benoitc/couchbeam and sending several chunked requests. If the last non-terminating chunk completed the response JSON object, `hackney_response:skip_body/1` is called to discard the remaining body, but is told to receive a number of bytes equal to the expected remaining size which will frequently exceed the small terminating chunk and trailers. As a result, the recv operation hangs waiting for bytes that will never arrive. Now, the transfer state (`BufSize`/`ExpectedSize`) are reset after each successful chunk. The speed benefit of benoitc#710 is retained (tested with the same approach as in that PR). - correct some related typespecs
|
@benoitc this has been working well for us lately. Curious if you have a chance to review? |
|
on it sorry for the delay. |
|
@benoitc hey there, anything needed from our end? I double-checked and I see the same EUnit failures on |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When streaming a chunked response, it is possible to cause a TCP receive hang under particular circumstances. If at one point the parser buffer doesn't have the whole chunk, at a later point the buffer ends up empty
<<>>, and subsequentlyhackney_response:stream_body/1is called,hackney_response:recv/2will hang if the expected remaining size exceeds the remainder of the response. That expected size is actually stale, from the earlier point when the parser did not have the whole chunk. This issue slipped in with #710.This was identified when using https://github.com/benoitc/couchbeam and sending several chunked requests. If the last non-terminating chunk completed the response JSON object,
hackney_response:skip_body/1is called to discard the remaining body, but is told to receive a number of bytes equal to the expected remaining size which will frequently exceed the small terminating chunk and trailers. As a result, the recv operation hangs waiting for bytes that will never arrive.Now, the transfer state (
BufSize/ExpectedSize) are reset after each successful chunk. The speed benefit of #710 is retained (tested with the same approach as in that PR).