Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

http: create an option for setting a maximum size for uri parsing #26553

Closed
wants to merge 8 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions doc/api/http.md
Original file line number Diff line number Diff line change
Expand Up @@ -1959,6 +1959,17 @@ added: v11.6.0
Read-only property specifying the maximum allowed size of HTTP headers in bytes.
Defaults to 8KB. Configurable using the [`--max-http-header-size`][] CLI option.

## http.maxUriSize
<!-- YAML
added: REPLACEME
-->

* {number}

Read-only property specifying the maximum allowed size of HTTP request uri
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Read-only property specifying the maximum allowed size of HTTP request uri
Read-only property specifying the maximum allowed size of HTTP request URI

in bytes. Defaults to 7KB. Configurable using the
[`--max-http-uri-size`][] CLI option.
BridgeAR marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing bottom references. Will this option be documented later?


## http.request(options[, callback])
## http.request(url[, options][, callback])
<!-- YAML
Expand Down
17 changes: 14 additions & 3 deletions lib/_http_server.js
Original file line number Diff line number Diff line change
Expand Up @@ -509,6 +509,10 @@ const badRequestResponse = Buffer.from(
`HTTP/1.1 400 ${STATUS_CODES[400]}${CRLF}` +
`Connection: close${CRLF}${CRLF}`, 'ascii'
);
const uriTooLongResponse = Buffer.from(
`HTTP/1.1 414 ${STATUS_CODES[414]}${CRLF}` +
`Connection: close${CRLF}${CRLF}`, 'ascii'
);
const requestHeaderFieldsTooLargeResponse = Buffer.from(
`HTTP/1.1 431 ${STATUS_CODES[431]}${CRLF}` +
`Connection: close${CRLF}${CRLF}`, 'ascii'
Expand All @@ -520,9 +524,16 @@ function socketOnError(e) {

if (!this.server.emit('clientError', e, this)) {
if (this.writable) {
const response = e.code === 'HPE_HEADER_OVERFLOW' ?
requestHeaderFieldsTooLargeResponse : badRequestResponse;
this.write(response);
switch (e.code) {
case 'HPE_URI_OVERFLOW':
this.write(uriTooLongResponse);
break;
case 'HPE_HEADER_OVERFLOW':
this.write(requestHeaderFieldsTooLargeResponse);
break;
default:
this.write(badRequestResponse);
}
}
this.destroy(e);
}
Expand Down
14 changes: 14 additions & 0 deletions lib/http.js
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ const {
ServerResponse
} = require('_http_server');
let maxHeaderSize;
let maxUriSize;

function createServer(opts, requestListener) {
return new Server(opts, requestListener);
Expand Down Expand Up @@ -76,6 +77,19 @@ Object.defineProperty(module.exports, 'maxHeaderSize', {
}
});

Object.defineProperty(module.exports, 'maxUriSize', {
configurable: true,
enumerable: true,
get() {
if (maxUriSize === undefined) {
const { getOptionValue } = require('internal/options');
maxUriSize = getOptionValue('--max-http-uri-size');
}

return maxUriSize;
}
});

Object.defineProperty(module.exports, 'globalAgent', {
configurable: true,
enumerable: true,
Expand Down
16 changes: 15 additions & 1 deletion src/node_http_parser_impl.h
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ class Parser : public AsyncWrap, public StreamListener {


int on_url(const char* at, size_t length) {
int rv = TrackHeader(length);
int rv = TrackURI(length);
if (rv != 0) {
return rv;
}
Expand Down Expand Up @@ -251,6 +251,7 @@ class Parser : public AsyncWrap, public StreamListener {

int on_headers_complete() {
#ifdef NODE_EXPERIMENTAL_HTTP
uri_nread_ = 0;
header_nread_ = 0;
#endif /* NODE_EXPERIMENTAL_HTTP */

Expand Down Expand Up @@ -813,6 +814,7 @@ class Parser : public AsyncWrap, public StreamListener {
void Init(parser_type_t type) {
#ifdef NODE_EXPERIMENTAL_HTTP
llhttp_init(&parser_, type, &settings);
uri_nread_ = 0;
header_nread_ = 0;
#else /* !NODE_EXPERIMENTAL_HTTP */
http_parser_init(&parser_, type);
Expand All @@ -825,6 +827,17 @@ class Parser : public AsyncWrap, public StreamListener {
got_exception_ = false;
}

int TrackURI(size_t len) {
#ifdef NODE_EXPERIMENTAL_HTTP
uri_nread_ += len;
if (uri_nread_ >= per_process::cli_options->max_http_uri_size) {
llhttp_set_error_reason(&parser_,
"HPE_URI_OVERFLOW:URI overflow");
return HPE_USER;
}
#endif /* NODE_EXPERIMENTAL_HTTP */
return 0;
}

int TrackHeader(size_t len) {
#ifdef NODE_EXPERIMENTAL_HTTP
Expand Down Expand Up @@ -869,6 +882,7 @@ class Parser : public AsyncWrap, public StreamListener {
#ifdef NODE_EXPERIMENTAL_HTTP
unsigned int execute_depth_ = 0;
bool pending_pause_ = false;
uint64_t uri_nread_ = 0;
uint64_t header_nread_ = 0;
#endif /* NODE_EXPERIMENTAL_HTTP */

Expand Down
4 changes: 4 additions & 0 deletions src/node_options.cc
Original file line number Diff line number Diff line change
Expand Up @@ -426,6 +426,10 @@ PerProcessOptionsParser::PerProcessOptionsParser(
"set the maximum size of HTTP headers (default: 8KB)",
&PerProcessOptions::max_http_header_size,
kAllowedInEnvironment);
AddOption("--max-http-uri-size",
"set the maximum size of HTTP request uri (default: 7KB)",
&PerProcessOptions::max_http_uri_size,
kAllowedInEnvironment);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fwiw, this should not be a per-process but rather a per-Environment option.

AddOption("--v8-pool-size",
"set V8's thread pool size",
&PerProcessOptions::v8_thread_pool_size,
Expand Down
1 change: 1 addition & 0 deletions src/node_options.h
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,7 @@ class PerProcessOptions : public Options {
std::string trace_event_categories;
std::string trace_event_file_pattern = "node_trace.${rotation}.log";
uint64_t max_http_header_size = 8 * 1024;
uint64_t max_http_uri_size = 7 * 1024;
int64_t v8_thread_pool_size = 4;
bool zero_fill_all_buffers = false;
bool debug_arraybuffer_allocations = false;
Expand Down
38 changes: 30 additions & 8 deletions test/parallel/test-http-header-overflow.js
Original file line number Diff line number Diff line change
@@ -1,18 +1,34 @@
// Flags: --expose-internals
'use strict';
const { expectsError, mustCall } = require('../common');
const assert = require('assert');
const { createServer, maxHeaderSize } = require('http');
const { createConnection } = require('net');
const { expectsError, mustCall } = require('../common');

const { getOptionValue } = require('internal/options');

const currentParser = getOptionValue('--http-parser');

const sumOfLengths = (strings) => strings
.map((string) => string.length)
.reduce((a, b) => a + b);

const getBreakingLength = () => {
if (currentParser === 'llhttp') {
return maxHeaderSize - HEADER_NAME.length + 1;
} else {
return maxHeaderSize - HEADER_NAME.length - HEADER_SEPARATOR -
(2 * CRLF.length) + 1;
}
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add some comments in this function?


const CRLF = '\r\n';
const DUMMY_HEADER_NAME = 'Cookie: ';
const DUMMY_HEADER_VALUE = 'a'.repeat(
// Plus one is to make it 1 byte too big
maxHeaderSize - DUMMY_HEADER_NAME.length - (2 * CRLF.length) + 1
);
const HEADER_NAME = 'Cookie';
const HEADER_SEPARATOR = ': ';
const HEADER_VALUE = 'a'.repeat(getBreakingLength());
const PAYLOAD_GET = 'GET /blah HTTP/1.1';
const PAYLOAD = PAYLOAD_GET + CRLF +
DUMMY_HEADER_NAME + DUMMY_HEADER_VALUE + CRLF.repeat(2);
HEADER_NAME + HEADER_SEPARATOR + HEADER_VALUE + CRLF.repeat(2);

const server = createServer();

Expand All @@ -21,7 +37,13 @@ server.on('connection', mustCall((socket) => {
type: Error,
message: 'Parse Error',
code: 'HPE_HEADER_OVERFLOW',
bytesParsed: maxHeaderSize + PAYLOAD_GET.length,
bytesParsed: sumOfLengths([
PAYLOAD_GET,
CRLF,
HEADER_NAME,
HEADER_SEPARATOR,
HEADER_VALUE
]) + 1,
rawPacket: Buffer.from(PAYLOAD)
}));
}));
Expand Down
46 changes: 46 additions & 0 deletions test/parallel/test-http-uri-overflow.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
'use strict';
const { expectsError, mustCall } = require('../common');
const assert = require('assert');
const { createServer, maxUriSize } = require('http');
const { createConnection } = require('net');

const CRLF = '\r\n';
const REQUEST_METHOD_AND_SPACE = 'GET ';
const DUMMY_URI = '/' + 'a'.repeat(
maxUriSize
); // The slash makes it just 1 byte too long.

const PAYLOAD = REQUEST_METHOD_AND_SPACE + DUMMY_URI + CRLF;

const server = createServer();

server.on('connection', mustCall((socket) => {
socket.on('error', expectsError({
type: Error,
message: 'Parse Error',
code: 'HPE_URI_OVERFLOW',
bytesParsed: REQUEST_METHOD_AND_SPACE.length + DUMMY_URI.length,
rawPacket: Buffer.from(PAYLOAD)
}));
}));

server.listen(0, mustCall(() => {
const c = createConnection(server.address().port);
let received = '';

c.on('connect', mustCall(() => {
c.write(PAYLOAD);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is highly possible that the payload comes out of the other side of the socket as a single Buffer. Can you please add a test where the URL is split into two or more chunks?

}));
c.on('data', mustCall((data) => {
received += data.toString();
}));
c.on('end', mustCall(() => {
assert.strictEqual(
received,
'HTTP/1.1 414 URI Too Long\r\n' +
'Connection: close\r\n\r\n'
);
c.end();
}));
c.on('close', mustCall(() => server.close()));
}));
4 changes: 2 additions & 2 deletions test/sequential/test-http-max-http-headers.js
Original file line number Diff line number Diff line change
Expand Up @@ -122,8 +122,8 @@ const test2 = common.mustCall(() => {
'Agent: nod2\r\n' +
'X-CRASH: ';

// /, Host, localhost, Agent, node, X-CRASH, a...
const currentSize = 1 + 4 + 9 + 5 + 4 + 7;
// Host, localhost, Agent, node, X-CRASH, a...
const currentSize = 4 + 9 + 5 + 4 + 7;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please explain this change?

Copy link
Author

@caiolrm caiolrm Mar 21, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

currentSize is being used by fillHeaders to subtract from the max header size and generate a header value with a length that makes the total header size hit the limit. Since the implementation at node_http_parser_impl.h when using llhttp doesn't count the url size towards the header size after the changes, there's no point considering the "/" size when using fillHeaders anymore.

headers = fillHeaders(headers, currentSize);

const server = http.createServer(common.mustNotCall());
Expand Down