Skip to content

Commit

Permalink
http: strictly forbid invalid characters from headers
Browse files Browse the repository at this point in the history
PR-URL: nodejs-private/node-private#26
Reviewed-By: Rod Vagg <r@va.gg>
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
  • Loading branch information
jasnell committed Feb 9, 2016
1 parent 4f4c8ab commit 7bef1b7
Show file tree
Hide file tree
Showing 5 changed files with 92 additions and 97 deletions.
15 changes: 9 additions & 6 deletions doc/api/http.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -649,8 +649,8 @@ response.addTrailers({'Content-MD5': '7895bf4b8828b55ceaf47747b4bca667'});
response.end();
```

Attempting to set a trailer field name that contains invalid characters will
result in a [`TypeError`][] being thrown.
Attempting to set a header field name or value that contains invalid characters
will result in a [`TypeError`][] being thrown.

### response.end([data][, encoding][, callback])

Expand Down Expand Up @@ -721,8 +721,8 @@ or
response.setHeader('Set-Cookie', ['type=ninja', 'language=javascript']);
```

Attempting to set a header field name that contains invalid characters will
result in a [`TypeError`][] being thrown.
Attempting to set a header field name or value that contains invalid characters
will result in a [`TypeError`][] being thrown.

When headers have been set with [`response.setHeader()`][], they will be merged with
any headers passed to [`response.writeHead()`][], with the headers passed to
Expand Down Expand Up @@ -836,8 +836,8 @@ response.writeHead(200, {
This method must only be called once on a message and it must
be called before [`response.end()`][] is called.

If you call [`response.write()`][] or [`response.end()`][] before calling this, the
implicit/mutable headers will be calculated and call this function for you.
If you call [`response.write()`][] or [`response.end()`][] before calling this,
the implicit/mutable headers will be calculated and call this function for you.

When headers have been set with [`response.setHeader()`][], they will be merged with
any headers passed to [`response.writeHead()`][], with the headers passed to
Expand All @@ -860,6 +860,9 @@ should be used to determine the number of bytes in a given encoding.
And Node.js does not check whether Content-Length and the length of the body
which has been transmitted are equal or not.

Attempting to set a header field name or value that contains invalid characters
will result in a [`TypeError`][] being thrown.

## Class: http.IncomingMessage

An `IncomingMessage` object is created by [`http.Server`][] or
Expand Down
17 changes: 17 additions & 0 deletions lib/_http_common.js
Original file line number Diff line number Diff line change
Expand Up @@ -231,3 +231,20 @@ function checkIsHttpToken(val) {
return typeof val === 'string' && token.test(val);
}
exports._checkIsHttpToken = checkIsHttpToken;

/**
* True if val contains an invalid field-vchar
* field-value = *( field-content / obs-fold )
* field-content = field-vchar [ 1*( SP / HTAB ) field-vchar ]
* field-vchar = VCHAR / obs-text
**/
function checkInvalidHeaderChar(val) {
val = '' + val;
for (var i = 0; i < val.length; i++) {
const ch = val.charCodeAt(i);
if (ch === 9) continue;
if (ch <= 31 || ch > 255 || ch === 127) return true;

This comment has been minimized.

Copy link
@silentroach

silentroach Feb 15, 2016

Contributor

Inconsistency detected.

comment:

field-vchar    = VCHAR / obs-text

obs-text is %x80-FF

http_parser.c:

/**
 * Verify that a char is a valid visible (printable) US-ASCII
 * character or %x80-FF
 **/
#define IS_HEADER_CHAR(ch)                                                     \
  (ch == CR || ch == LF || ch == 9 || (ch > 31 && ch != 127))

Real example (nginx + maxmind) with GEOIP_CITY: Düsseldorf - got an empty reply, so nginx will send the request to next upstream and then it will end with 502 bad gateway cause nobody replied.

ping @jasnell

Newly defined header fields SHOULD limit their field values to
   US-ASCII octets.  A recipient SHOULD treat other octets in field
   content (obs-text) as opaque data.

newly only. but there shuld be compatibility though.

This comment has been minimized.

Copy link
@bnoordhuis

bnoordhuis Feb 15, 2016

Member

How is the ü in Düsseldorf encoded? If you can get the raw bytes for that header, that would be helpful.

This comment has been minimized.

Copy link
@silentroach

silentroach Feb 15, 2016

Contributor
$ curl -vvv http://localhost:8000/something -H 'GEOIP_CITY: Düsseldorf'
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8000 (#0)
> GET /something HTTP/1.1
> Host: localhost:8000
> User-Agent: curl/7.43.0
> Accept: */*
> GEOIP_CITY: Düsseldorf
>
* Empty reply from server
* Connection #0 to host localhost left intact

empty reply with node v5.6.0 for this request and normal response with v5.5.0

This comment has been minimized.

Copy link
@ChALkeR

ChALkeR Feb 15, 2016

Member

For me, curl uses UTF-8 (perhaps the system encoding actually, not sure), and ü looks like [0xC3, 0xBC].
Verified with Wireshark.

This comment has been minimized.

Copy link
@silentroach

silentroach Feb 15, 2016

Contributor

yep (c3 bc)

...
0030: 55 73 65 72 2d 41 67 65 6e 74 3a 20 63 75 72 6c User-Agent: curl
0040: 2f 37 2e 34 33 2e 30 0d 0a 41 63 63 65 70 74 3a /7.43.0..Accept:
0050: 20 2a 2f 2a 0d 0a 47 45 4f 49 50 5f 43 49 54 59  */*..GEOIP_CITY
0060: 3a 20 44 c3 bc 73 73 65 6c 64 6f 72 66 0d 0a 0d : D..sseldorf...
0070: 0a                                              .
== Info: Empty reply from server
== Info: Connection #0 to host localhost left intact

This comment has been minimized.

Copy link
@ChALkeR

ChALkeR Feb 15, 2016

Member

@silentroach This is most certainly not the commit that you are looking for, btw — it deals with outgoing headers, and your issue is with incoming headers, it seems.

This comment has been minimized.

Copy link
@silentroach

silentroach Feb 15, 2016

Contributor

seems like you are right, shame on me.

will try to find the reason.

98fde2f6a75bc979bd7c749c59bbe3193adae2a1 is the first bad commit
commit 98fde2f6a75bc979bd7c749c59bbe3193adae2a1
Author: James M Snell <jasnell@gmail.com>
Date:   Wed Feb 3 17:28:48 2016 -0800
}
return false;
}
exports._checkInvalidHeaderChar = checkInvalidHeaderChar;
15 changes: 11 additions & 4 deletions lib/_http_outgoing.js
Original file line number Diff line number Diff line change
Expand Up @@ -305,8 +305,10 @@ function storeHeader(self, state, field, value) {
throw new TypeError(
'Header name must be a valid HTTP Token ["' + field + '"]');
}
value = escapeHeaderValue(value);
state.messageHeader += field + ': ' + value + CRLF;
if (common._checkInvalidHeaderChar(value) === true) {
throw new TypeError('The header content contains invalid characters');
}
state.messageHeader += field + ': ' + escapeHeaderValue(value) + CRLF;

if (connectionExpression.test(field)) {
state.sentConnectionHeader = true;
Expand Down Expand Up @@ -341,8 +343,10 @@ OutgoingMessage.prototype.setHeader = function(name, value) {
if (value === undefined)
throw new Error('"value" required in setHeader("' + name + '", value)');
if (this._header)
throw new Error('Can\'t set headers after they are sent');

throw new Error('Can\'t set headers after they are sent.');
if (common._checkInvalidHeaderChar(value) === true) {
throw new TypeError('The header content contains invalid characters');
}
if (this._headers === null)
this._headers = {};

Expand Down Expand Up @@ -515,6 +519,9 @@ OutgoingMessage.prototype.addTrailers = function(headers) {
throw new TypeError(
'Trailer name must be a valid HTTP Token ["' + field + '"]');
}
if (common._checkInvalidHeaderChar(value) === true) {
throw new TypeError('The header content contains invalid characters');
}
this._trailer += field + ': ' + escapeHeaderValue(value) + CRLF;
}
};
Expand Down
87 changes: 0 additions & 87 deletions test/parallel/test-http-header-response-splitting.js

This file was deleted.

55 changes: 55 additions & 0 deletions test/parallel/test-http-response-splitting.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
'use strict';

const common = require('../common');
const http = require('http');
const net = require('net');
const url = require('url');
const assert = require('assert');

// Response splitting example, credit: Amit Klein, Safebreach
const str = '/welcome?lang=bar%c4%8d%c4%8aContent­Length:%200%c4%8d%c4%8a%c' +
'4%8d%c4%8aHTTP/1.1%20200%20OK%c4%8d%c4%8aContent­Length:%202' +
'0%c4%8d%c4%8aLast­Modified:%20Mon,%2027%20Oct%202003%2014:50:18' +
'%20GMT%c4%8d%c4%8aContent­Type:%20text/html%c4%8d%c4%8a%c4%8' +
'd%c4%8a%3chtml%3eGotcha!%3c/html%3e';

// Response splitting example, credit: Сковорода Никита Андреевич (@ChALkeR)
const x = 'fooഊSet-Cookie: foo=barഊഊ<script>alert("Hi!")</script>';
const y = 'foo⠊Set-Cookie: foo=bar';

var count = 0;

const server = http.createServer((req, res) => {
switch (count++) {
case 0:
const loc = url.parse(req.url, true).query.lang;
assert.throws(common.mustCall(() => {
res.writeHead(302, {Location: `/foo?lang=${loc}`});
}));
break;
case 1:
assert.throws(common.mustCall(() => {
res.writeHead(200, {'foo' : x});
}));
break;
case 2:
assert.throws(common.mustCall(() => {
res.writeHead(200, {'foo' : y});
}));
break;
default:
assert.fail(null, null, 'should not get to here.');
}
if (count === 3)
server.close();
res.end('ok');
});
server.listen(common.PORT, () => {
const end = 'HTTP/1.1\r\n\r\n';
const client = net.connect({port: common.PORT}, () => {
client.write(`GET ${str} ${end}`);
client.write(`GET / ${end}`);
client.write(`GET / ${end}`);
client.end();
});
});

2 comments on commit 7bef1b7

@akras14
Copy link

@akras14 akras14 commented on 7bef1b7 Apr 5, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this commit has broken a good number of implementations that have used invalid characters in headers.

Would you mind sharing some links to the spec that should be followed and, if possible, some tips on how to make the code compliant?

Found this link which appears to point to the spec: http://stackoverflow.com/questions/19028068/illegal-characters-in-http-headers

@jasnell
Copy link
Member Author

@jasnell jasnell commented on 7bef1b7 Apr 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The spec is RFC 7230 (https://tools.ietf.org/html/rfc7230). Specifically, section 3.2 (https://tools.ietf.org/html/rfc7230#section-3.2). The key reason for being strict here is that there are a number of very real security issues related to being too lenient on header parsing. Specifically, code that uses multi-byte character encodings for values in header field names or values can end up with request splitting or response smuggling vulnerabilities because the multi-byte encodings get interpreted as single-byte encodings by http implementations.

Please sign in to comment.