HTTPRoute Timeouts

[mpstefan]

As a user of NGF
I want to specify timeouts per request for my HTTPRoutes
So that my client applications can get quick feedback that the request will not complete

Background:

NGINX provides the following directives for timeouts:

• proxy_connect_timeout
• proxy_read_timeout
• proxy_send_timeout
• proxy_next_upstream_timeout

All of these timeouts are based on packet transfer to the upstream instead of the entire request. As such, we will need to create an NJS module to support a complete HTTP timeout that is scoped to each request instead of measuring when a packet was last received from the upstream.

This feature is defined in the Gateway API here as an extended feature.

Acceptance

• The user can define a "request" duration for a timeout in an HTTPRoute rule and functions as defined here:

  Request specifies the maximum duration for a gateway to respond to an HTTP request. If the gateway has not been able to respond before this deadline is met, the gateway MUST return a timeout error.

  For example, setting the rules.timeouts.request field to the value 10s in an HTTPRoute will cause a timeout if a client request is taking longer than 10 seconds to complete.

  Setting a timeout to the zero duration (e.g. “0s”) SHOULD disable the timeout completely. Implementations that cannot completely disable the timeout MUST instead interpret the zero duration as the longest possible value to which the timeout can be set.

  This timeout is intended to cover as close to the whole request-response transaction as possible although an implementation MAY choose to start the timeout after the entire request stream has been received instead of immediately after the transaction is initiated by the client.

  The value of Request is a Gateway API Duration string as defined by GEP-2257. When this field is unspecified, request timeout behavior is implementation-specific.

• The user can define a "backendRequest" duration for a timeout in an HTTPRoute rule and functions as defined here:

  BackendRequest specifies a timeout for an individual request from the gateway to a backend. This covers the time from when the request first starts being sent from the gateway to when the full response has been received from the backend.

  Setting a timeout to the zero duration (e.g. “0s”) SHOULD disable the timeout completely. Implementations that cannot completely disable the timeout MUST instead interpret the zero duration as the longest possible value to which the timeout can be set.

  An entire client HTTP transaction with a gateway, covered by the Request timeout, may result in more than one call from the gateway to the destination backend, for example, if automatic retries are supported.

  The value of BackendRequest must be a Gateway API Duration string as defined by GEP-2257. When this field is unspecified, its behavior is implementation-specific; when specified, the value of BackendRequest must be no more than the value of the Request timeout (since the Request timeout encompasses the BackendRequest timeout).

[chrisakker]

None of these timeouts are packet based. They are all TCP connections, or HTTP request/response timeouts, and LB selection timeouts. For further review, read the actual text of the Directives that you linked to. There is no need for NJS, Nginx core does this by default.

[even0306]

None of these timeouts are packet based. They are all TCP connections, or HTTP request/response timeouts, and LB selection timeouts. For further review, read the actual text of the Directives that you linked to. There is no need for NJS, Nginx core does this by default.

How to configure proxy_read_timeout currently?

[mpstefan]

Let me clarify a bit on the response we got from Igor on the core team... what these timeouts measure is the time it takes to complete a part of the whole request. Here's another definition of these three timeouts:

proxy_connect_timeout: Time to wait for the upstream to accept the connection. Does not apply if there is already a connection present from a previous request.

proxy_send_timeout: If a single send operation in the buffer takes longer than this time, the request times out. Each request will have multiple send operations depending on the size of the request, so we're essentially measuring the rate of the send operations here.

proxy_read_timeout: This is measuring the time between responses sent from the server for the whole connection, likely down to the packet level. If it takes longer than this time to receive any byte from the response, it times out.

Here's the full example from Igor that explains why an entire request timeout is currently not possible within nginx with the directives we have:

Let me make an example: let's say a client requests pi value from a server. And a server will generate it digit by digit,

A client will send a request like 'GET /pi ...'. Nginx will choose a configuration for that. There will be various timeouts associated with that configuration

Say, proxy_connect_timeout 3s; proxy_read_timeout 2s; proxy_send_timeout 1s;

nginx chooses a configuration. Finds a server to connect to. If there is a connection in keepalive pool - connect timeout is skipped as we are already connected

If there is no connection in a cache or all connections are used - we establish a new connection for this request specifically
3s timeout is the time when an upstream has to accept a connection.

We will then start sending a request to an upstream

It may take us several operations to send a request

We form a buffer, send it into a connection and start waiting for that 1s timeout till next send operation

If an upstream is too slow reading our request and we cant send the second part of a request within 1 second - a request timeouts, upstream coonection closed, we send 50x response to a client,

If we have sent a request in full we start reading response

Starting with headers

Say, our server sends out whole header part immediately and then 1 digit a second

In this case our read timeout is 2 seconds, we won't exceed it

So we will be reading an upstream response forever. as we never hit that read timeout

(or until we reach other conditions like content-length, connection:close or whatever)

It's more likely that a client side connection will timeout (a browser will stop waiting in, say, 5 minutes)

Do you see how there is no way to impose a limit on a request time overall?

[sjberman]

@even0306 For your question, you should be able to configure this using the SnippetsFilter API.

This doc shows an example using rate limiting, but you can use timeouts instead; something like:

apiVersion: gateway.nginx.org/v1alpha1
kind: SnippetsFilter
metadata:
  name: timeouts
spec:
  snippets:
    - context: http.server.location
      value: proxy_read_timeout 60s;

[even0306]

@even0306For your question, you should be able to configure this using theSnippetsFilter API.

This doc shows an example using rate limiting, but you can use timeouts instead; something like:

apiVersion: gateway.nginx.org/v1alpha1
kind: SnippetsFilter
metadata:
  name: timeouts
spec:
  snippets:
    - context: http.server.location
      value: proxy_read_timeout 60s;

Thank you