Brief description
An untrusted remote client may send HTTP/2 requests that write to the heap outside of the request buffers when the upstream is HTTP/1.
CVSS
CVSS:3.0/AV:N/AC:H/PR:N/UI:N/S:C/C:H/I:H/A:H
(CVSS score 9.0, Critical)
Affected version(s)
Envoy 1.12.1 and before.
Affected component(s)
HTTP/1.x codec
Attack vector(s)
An untrusted remote client may send HTTP/2 requests that write to the heap outside of the request buffers when the upstream is HTTP/1. This may be used to corrupt nearby heap contents, leading to a query-of-death scenario and may be used to bypass Envoy’s access control mechanisms such as path based routing. An attacker can also modify requests from other users that happen to be proximal temporarily and spatially.
Discover(s)/Credits
Harvey Tuch, Google
Example exploit or proof-of-concept
Configure envoy proxy with HTTP/1 upstream and allow HTTP/2 requests from client. Send client request with the ":method" header larger than 4Kb.
Details
The encoder in Envoy’s HTTP/1 codec creates HTTP/1 wire content from Envoy’s internal representation of request headers. While doing so, it assumes that the contents of the :method header field, which is written out at the beginning of the request encoding, is < 4KB when allocating buffers. However, the HTTP/2 codec does not impose a limit on this header’s value or size, as there is no restriction on the value of these headers. As a result, untrusted input from an HTTP/2 client can be used to overflow the buffer allocated for the encoding on the heap.
There are implications for availability, since heap corruption allows for a query-of-death, and also for confidentiality and integrity, since corruption of unrelated requests is possible. The severity is somewhat tempered by the fact that the contents that may be placed outside of bounds are limited to printable ASCII characters as per RFC constraints on valid header values (https://tools.ietf.org/html/rfc7230#section-3.2.6). The range is also limited to roughly the length of the path and method header fields, which is bounded to 64KB by default on Envoy. While we are not aware of any known remote code execution exploits derived from this overflow, there does exist a proof-of-concept for bypassing Envoy’s path based access control
Simple heap shaping can be used to reliably compromise Envoy’s access control mechanisms. A proof-of-concept exists in which two attacker requests delivered with some attacker controlled timing are able to result in bypass of Envoy’s access control. After some heap shaping by preceding requests, the first request is primed with some safe :path contents, e.g. /, and held in-flight in memory, e.g. via Envoy’s buffer filter. The second request overflows its encoding allocation to modify the path after the HTTP connection manager and filters such as the external authorization filter have viewed the path, allow access to arbitrary content in the backend.
Mitigations
- Disable upstream HTTP/1.
- Reduce header size limits to below 2KB. It might be possible to configure as high as somewhere near 4KB, but 2KB is a conservative size. The relevant configuration is HTTP connection manager’s max_request_headers_kb field.
Detection
It’s likely attempts to exploit this bug will result in elevated rates of Envoy crashes, since they are likely to corrupt aspects of heap state, and a successful attack will typically requires some heap shaping. Another potential detection mechanism would be to examine access logs for sign of irregular requests that resemble the nested HTTP/1 requests demonstrated in the proof-of-concept exploit.
References
CVE-2019-18801
Brief description
An untrusted remote client may send HTTP/2 requests that write to the heap outside of the request buffers when the upstream is HTTP/1.
CVSS
CVSS:3.0/AV:N/AC:H/PR:N/UI:N/S:C/C:H/I:H/A:H
(CVSS score 9.0, Critical)
Affected version(s)
Envoy 1.12.1 and before.
Affected component(s)
HTTP/1.x codec
Attack vector(s)
An untrusted remote client may send HTTP/2 requests that write to the heap outside of the request buffers when the upstream is HTTP/1. This may be used to corrupt nearby heap contents, leading to a query-of-death scenario and may be used to bypass Envoy’s access control mechanisms such as path based routing. An attacker can also modify requests from other users that happen to be proximal temporarily and spatially.
Discover(s)/Credits
Harvey Tuch, Google
Example exploit or proof-of-concept
Configure envoy proxy with HTTP/1 upstream and allow HTTP/2 requests from client. Send client request with the ":method" header larger than 4Kb.
Details
The encoder in Envoy’s HTTP/1 codec creates HTTP/1 wire content from Envoy’s internal representation of request headers. While doing so, it assumes that the contents of the :method header field, which is written out at the beginning of the request encoding, is < 4KB when allocating buffers. However, the HTTP/2 codec does not impose a limit on this header’s value or size, as there is no restriction on the value of these headers. As a result, untrusted input from an HTTP/2 client can be used to overflow the buffer allocated for the encoding on the heap.
There are implications for availability, since heap corruption allows for a query-of-death, and also for confidentiality and integrity, since corruption of unrelated requests is possible. The severity is somewhat tempered by the fact that the contents that may be placed outside of bounds are limited to printable ASCII characters as per RFC constraints on valid header values (https://tools.ietf.org/html/rfc7230#section-3.2.6). The range is also limited to roughly the length of the path and method header fields, which is bounded to 64KB by default on Envoy. While we are not aware of any known remote code execution exploits derived from this overflow, there does exist a proof-of-concept for bypassing Envoy’s path based access control
Simple heap shaping can be used to reliably compromise Envoy’s access control mechanisms. A proof-of-concept exists in which two attacker requests delivered with some attacker controlled timing are able to result in bypass of Envoy’s access control. After some heap shaping by preceding requests, the first request is primed with some safe :path contents, e.g. /, and held in-flight in memory, e.g. via Envoy’s buffer filter. The second request overflows its encoding allocation to modify the path after the HTTP connection manager and filters such as the external authorization filter have viewed the path, allow access to arbitrary content in the backend.
Mitigations
Detection
It’s likely attempts to exploit this bug will result in elevated rates of Envoy crashes, since they are likely to corrupt aspects of heap state, and a successful attack will typically requires some heap shaping. Another potential detection mechanism would be to examine access logs for sign of irregular requests that resemble the nested HTTP/1 requests demonstrated in the proof-of-concept exploit.
References