TS-5092: ATS handling of too many concurrent streams too agressive #1262
Conversation
|
FreeBSD build successful! See https://ci.trafficserver.apache.org/job/Github-FreeBSD/1265/ for details. |
|
Linux build successful! See https://ci.trafficserver.apache.org/job/Github-Linux/1160/ for details. |
maskit
left a comment
maskit
left a comment
There was a problem hiding this comment.
The fix for the original issue seems good, but there are few unnecessary changes, and a typo.
proxy/http2/Http2ConnectionState.cc
Outdated
| } | ||
| if (stream->server_rwnd < payload_length) { | ||
| return Http2Error(HTTP2_ERROR_CLASS_STREAM, HTTP2_ERROR_FLOW_CONTROL_ERROR); | ||
| return Http2Error(HTTP2_ERROR_CLASS_STREAM, HTTP2_ERROR_FLOW_CONTROL_ERROR, "recvdata stream->server_rwnd < payload_length"); |
There was a problem hiding this comment.
"recv data" -- please add a space.
proxy/http2/Http2ConnectionState.cc
Outdated
|
|
||
| DebugHttp2Stream(cstate.ua_session, stream_id, "Received CONTINUATION frame"); | ||
|
|
||
| // Error("http2_cs [%" PRId64 "] Received RST_STREAM frame for %d", cs.connection_id(), stream_id); |
There was a problem hiding this comment.
This line should not be necessary.
proxy/http2/Http2ConnectionState.cc
Outdated
| return Http2Error(HTTP2_ERROR_CLASS_CONNECTION, HTTP2_ERROR_ENHANCE_YOUR_CALM, "continuation enhance your calm"); | ||
| } else { | ||
| return Http2Error(HTTP2_ERROR_CLASS_STREAM, HTTP2_ERROR_PROTOCOL_ERROR); | ||
| return Http2Error(HTTP2_ERROR_CLASS_CONNECTION, HTTP2_ERROR_PROTOCOL_ERROR, "continuation decode error"); |
There was a problem hiding this comment.
This case is actually not for a decode error but HTTP2 violation, such as using uppercase in header names. All decode errors should be handled as compression error. We don't have to abandon all following requests with a connection error, because the HPACK decoder instance keeps valid status. Please see 8.1.2.6.
I'd put a message like "continuation malformed request".
proxy/http2/Http2ConnectionState.cc
Outdated
| return Http2Error(HTTP2_ERROR_CLASS_CONNECTION, HTTP2_ERROR_ENHANCE_YOUR_CALM, "recv headers enhance your calm"); | ||
| } else { | ||
| return Http2Error(HTTP2_ERROR_CLASS_STREAM, HTTP2_ERROR_PROTOCOL_ERROR); | ||
| return Http2Error(HTTP2_ERROR_CLASS_STREAM, HTTP2_ERROR_PROTOCOL_ERROR, "recv headers decode error"); |
There was a problem hiding this comment.
This is not a decode error. Please see a comment for the same logic in rcv_continuation_frame.
|
Pushed new commit (squashed) to address maskit's comments. |
|
FreeBSD build successful! See https://ci.trafficserver.apache.org/job/Github-FreeBSD/1292/ for details. |
|
Linux build successful! See https://ci.trafficserver.apache.org/job/Github-Linux/1185/ for details. |
5 participants
This issue was identified while debugging new errors seen by an internal team after they enabled HTTP/2 in their client. On the backend, they saw an increase in the cases were ATS sends the origin the POST header but no POST body and then closes the connection.
With the addition of Error() messages we were able to see a case where the client is trying to open the 101'st stream on a session. This is beyond the 100 max concurrent stream limit, so ATS shuts down the session which kills the previous 100 streams.
A closer reading of section 5.1.2 of the spec (https://tools.ietf.org/html/rfc7540#section-5.1.2) indicates that this should be a stream error and not a connection error. Bryan Call, Masaori, and Maskit confirmed this interpretation. Maskit also noted that the other error case in the current createStream method must be treated as a connection error.
Presumably the client library is expecting the refused stream case so it can try again later.
The main change is in create_stream(). Added error messages through the H2Error() object to enable this debugging and future tracking down of error conditions.