Should have separate values for stream and session windows in HTTP/2

[shinrich]

Noticed this while debugging HTTP/2 to origin. Traffic Server provides proxy.config.http2.initial_window_size_in to change the initial window size from the default of 65536. However, this value is used for both the stream window and the session window. So any single stream can overrun the shared session window.

Looking at window values from our clients, it seems that many clients will have a higher session window than the stream window.

For my testing, I just multiplied the session window by 50. That plus greatly increasing the stream window size, got my test scenario working (one ATS level using a H2 connection to another ATS level to interleave streams from many client requests).

I propose that we add a second setting to control the session window. It could be a multiplier, e.g proxy.config.http2.session_window_multiplier_in, or just an explicit session window setting, e.g., proxy.config.http2.session initial_window_size_in. If the session version of the setting is not set, the stream version will be used as it is in the current code.

I would appreciate your thoughts on this.

[maskit]

Having higher session window size than stream window size means that you can't fully utilize the connection if you have just one stream. I think that's the reason why default session window size and default stream window size have the same size. When you have multiple streams on a session, the session window size should be shared equally (or fairly) by the streams. I think just increasing initial_window_size_in should work for any cases. But I may be missing something if many clients has higher session window size.

[shinrich]

I am definitely seeing cases where the stream will suck up the either session window. By separating the values, it seems easier to define values to smoothly share the bandwidth between streams.

I'm running into this issue in testing H2 to origin where the connection to origin is very busy with many streams from different clients.

[maskit]

I'm saying it may be due to a bug or poor implementation in window control if increasing initial_window_size_in doesn't work for your case. Having a setting for session window size and set it to a very big number sound like you are effectively removing window control for sessions.

It is unclear to me that the issue is caused by the sender side or the receiver side. In other words, the both giving insufficient window and using given window poorly can cause the issue.

Current implementation gives streams chances to send data equally (if stream priority is not specified). However, there's no guarantee that a stream can actually send data when the stream get a chance, because stream/session window may be 0. If there are many streams, the stream may have to wait for next chance for a bit long time. And it's possible that there are unfortunate streams that can't send any data for very long time.

[shinrich]

I am running into this with a heavily used Http2 sessions between to ATS boxes in the HTTP/2 to origin work. For both POST bodies and response bodies. Unlike the case of ATS communicating with a dedicated client (web browser), these are unrelated stream requests scheduled onto a common HTTP/2 session. If the stream and session are set to the same value, each stream will push through until it finishes or exhausts the stream (and session) window.

I agree that just opening up the current initial_window_size_in wide will address the issue by more or less eliminating the windowing logic. However, by having a larger session window, you could have multiple streams make progress (exhausting their stream windows in the process).

In my debugging where I added logic printing session/stream windows at the exhaustion point, the ATS implementation would always stop with session window at 0. Sometimes stream window was also at 0, but generally not. Looking at numbers from our non-ATS peer, it was not uncommon to see higher session window sizes than stream window sizes (i.e., the stream exhausts before the session window is exhausted).

Of course having a larger stream window than session window does not make sense for all cases, but I think it is a useful distinction for some cases. In the outbound case, we may in fact want to make these window size settings overridable, so we can tune them for specific origins..

[bryancall]

Taking a step back and please correct me if I am wrong. From my understanding we are talking about a setup like this:
client(s) -> ATS.1 -> ATS.2 -> origin
What is happening is when we have multiple clients connecting to an origin over the same HTTP/2 connection/session one client can be greedy when sending a POST and and fully use up the connection/session window by sending up to their stream window. Having this option will help us to better tune clients and make sure each stream is getting a share of the connection window and not starve out other streams.

By default the session and stream windows will be the same. This is a tunable setting for those that have a more complex setup and want to tune the windows size independently. I am totally in favor for this option.

[maskit]

If the stream and session are set to the same value, each stream will push through until it finishes or exhausts the stream (and session) window.

This makes perfect sense if there's only one stream on a connection, right? Needless to say, a connection should be fully utilized regardless of the number of streams on the session.

I think what we should do is updating stream window size when the number of streams changes. More specifically, we should adjust stream window sizes to connection-window-size / #-of-active-streams. Below is from RFC 7430:

In addition to changing the flow-control window for streams that are
not yet active, a SETTINGS frame can alter the initial flow-control
window size for streams with active flow-control windows (that is,
streams in the "open" or "half-closed (remote)" state). When the
value of SETTINGS_INITIAL_WINDOW_SIZE changes, a receiver MUST adjust
the size of all stream flow-control windows that it maintains by the
difference between the new value and the old value.

Sending a SETTING frame mentioned above automatically maintains the best window sizes for streams. You can simply set the max window size you want to allow for an H2 connection to the existing proxy.config.http2.initial_window_size_in.

What size are you going to recommend for initial window size for streams if we have the setting for connections that you suggested? You may be able to set session_window_multiplier_in based on an expected average number of streams on a connection, but that fully utilize a connection only if a connection has the expected number of streams. I don't think that is great where we can adjust it dynamically.

[shinrich]

In my PR (#8203) and my testing, I've gone with explicit session window proxy.config.http2.session_initial_window_size_in.

While I agree it would be nice if we could automatically adjust the stream window sizes as streams are created and destroyed, I think implementing that would be quite complex. If I understand your proposal correctly, as a new stream is created, new setting frames would need to be sent to the existing stream frames to adjust down their stream windows. Similarly as a stream is closed, the remaining streams would need setting frames to increase their stream windows.

Also, with an automatic scheme, it would be quite easy to run into a scenario where each stream gets a tiny amount of window.

I think an explicit session window gives us something simple to test various scenarios. If the session window is not set, nothing is changed. As we move to outbound, the values could be adjusted via remap rules. We could also adjust inbound window sizes based on SNI values.

Once we gain some experience, we could later add an automatic mode to reallocate a global session window across the streams existing at the moment.

[maskit]

If I understand your proposal correctly, as a new stream is created, new setting frames would need to be sent to the existing stream frames to adjust down their stream windows. Similarly as a stream is closed, the remaining streams would need setting frames to increase their stream windows.

SETTINGS frame is allowed only on stream 0. One SETTING frame affects all existing and future streams. SETTINGS frame would still need to be sent again and again on stream 0, but it doesn't necessarily need to be done on every stream count change. I'd say once every a few seconds would be enough for many cases.

Also, with an automatic scheme, it would be quite easy to run into a scenario where each stream gets a tiny amount of window.

That depends on your configuration. The minimum stream window size would be initial_window_size_in / max_concurrent_streams_in. You shouldn't accept too many streams.

I think an explicit session window gives us something simple to test various scenarios. If the session window is not set, nothing is changed. As we move to outbound, the values could be adjusted via remap rules. We could also adjust inbound window sizes based on SNI values.

I'm not sure here. Session window size is explicit on my proposal too. Adding a switch to enable/disable the auto sizing should be easy if someone want to keep current behavior. Overriding session window size based on remap rules and SNI values should be possible with either.

Once we gain some experience, we could later add an automatic mode to reallocate a global session window across the streams existing at the moment.

That still doesn't solve the issue that a stream cannot fully utilize a connection.

Since my proposal (variable stream window size) would not conflict with your proposal (separate setting for session window size), I can live with your proposal. However, having the both would complicate window size settings and I think we should avoid it if possible. I'd say my proposal would work for the stall issue and we don't need a new setting for session window size, but I don't have time to implement it right now. So maybe I should approve your PR for now to move forward. What do you think, @shinrich, @bryancall ? Do you think we should have either of our proposals for simplicity, or accept the complexity for flexibility?

[shinrich]

I would like to move forward. I think we will need some time in the field to figure out what is best. What is practical. I'm guessing after a year, we'll want to circle up an simplify the window settings.

[rob05c]

The data window size adjustment seems like the kind of thing that'd be extremely useful to have as an API func and/or hook that plugins could call.

Then, you could write completely dynamic algorithms, in plugins, for that on-the-fly adjustment.

I suspect the same is true for client H2.

[maskit]

I've already commented my stance (-0 to go with Susan's proposal) on the PR, but I visualized current and proposed behavior just to make sure we all are on the same page. Let me know if my understanding is wrong.

Although adjusting window sizes by plugins via TS API would be the most flexible and potentially the best solution, I still think my proposal works better than Susan's IF I understand the issue correctly. One thing that still doesn't make sense to me is just increasing window size without code change didn't work for the issue. It should result in the most permissive settings in terms window sizes.