Propagate downstream timeout to upstream through multiple Envoys

[snowp]

When running Envoy on both egress and ingress, the client will provide a timeout header to the egress Envoy, which will propagate the expected upstream timeout in x-envoy-expected-rq-timeout-ms. The upstream Envoy will not read this header, so it will resolve a new timeout value that is set as the expected timeout for the upstream service. This means that the deadline expected by the egress Envoy is ignored in favor of the ingress Envoy, resulting in the upstream service having an incorrect view of the actual deadline.

It seems like either

1. inserting x-envoy-rq-timeout-ms with the expected timeout on egress
2. parsing x-envoy-expected-rq-timeout-ms as the deadline on the ingress side

would solve the issue (likely guarded by a config flag).

Thoughts?

[mattklein123]

Yeah agreed I think either would solve. I don't have a super strong opinion on which one.

This does bring up the general topic of deadline propagation which we haven't really tackled yet in a holistic way. I've thought at some point we may also want to stick the deadline in trace context baggage since this would also propagate through app calls, though that is a larger problem than you are trying to solve here.

[ramaraochavali]

+1. It would be very useful. Inserting x-envoy-rq-timeout-ms on the ingress side may be good idea. That might require users to set use_remote_address to ensure that it is not sanitized?

[stale]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or other activity occurs. Thank you for your contributions.

[nezdolik]

@snowp would like to help out with this.

[snowp]

@nezdolik Great, I'll assign this one to you. Feel free to ping me with any questions