res.redirect unpredictable sometimes with Unicode and even ANSI/ASCII #2897
Description
We get on occasion with § the escaped version (e.g. only %A7 a.k.a ASCII 245) during a redirect() (shouldn't be but encodeURI/encodeURIComponent gives %C2%A7)... I've spent about 3 hours trying different things... unfortunately it's not an easy bug to find. I've poured over our code in a commit that I'm attempting to get pushed to our organization when it's complete... but I'm having to ensure at the very least encodeURI is applied to the path... currently I am defaulting to encodeURIComponent if needed for express routing... even though it's already applied to the String already in between the slashes when needed... same as GitHub does all the time with encodeURIComponent. Some redirects work like a charm depending on their content... others don't.
Anyhow... hope some of this info helps track this down a little better... the work around so far is to double encodeURI or encodeURIComponent on each individual p/a/t/h/to which I know will eventually fail down the line.
Providing a reduced test case will be nearly impossible I think... but I am able to reproduce this with node@4.3.x and express@4.13.4 in our project.
Here's a few stack traces:
Stringy is exactly /scripts/Marti/RFC_2606§3_-_Hello,_World! before redirect() and passed as aRes.redirect(Stringy) with defaulting to 200 status code... tried forcing .status(200).redirect(Stringy) too... no luck.
URIError: Failed to decode param 'RFC_2606%A73_-_Hello,_World!'
at decodeURIComponent (native)
at decode_param (~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express/lib/router/layer.js:167:12)
at Layer.match (~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express/lib/router/layer.js:143:15)
at matchLayer (~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express/lib/router/index.js:557:18)
at next (~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express/lib/router/index.js:216:15)
at ~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/less-middleware/lib/middleware.js:315:14
at Layer.handle [as handle_request] (~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express/lib/router/layer.js:95:5)
at trim_prefix (~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express/lib/router/index.js:312:13)
at ~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express/lib/router/index.js:280:7
at Function.process_params (~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express/lib/router/index.js:330:12)
at next (~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express/lib/router/index.js:271:10)
at ~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/app.js:255:3
at Layer.handle [as handle_request] (~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express/lib/router/layer.js:95:5)
at trim_prefix (~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express/lib/router/index.js:312:13)
at ~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express/lib/router/index.js:280:7
at Function.process_params (~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express/lib/router/index.js:330:12)
at next (~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express/lib/router/index.js:271:10)
at express_minify_middleware (~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express-minify/index.js:185:5)
at Layer.handle [as handle_request] (~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express/lib/router/layer.js:95:5)
at trim_prefix (~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express/lib/router/index.js:312:13)
at ~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express/lib/router/index.js:280:7
at Function.process_params (~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express/lib/router/index.js:330:12)As it can be seen here the "Stringy" with this one is /users/テスト
GET /users/%E3%83%86%E3%82%B9%E3%83%88 200 468.579 ms - -
_http_outgoing.js:351
throw new TypeError('The header content contains invalid characters');
^
TypeError: The header content contains invalid characters
at ServerResponse.OutgoingMessage.setHeader (_http_outgoing.js:351:13)
at ServerResponse.header (~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express/lib/response.js:718:10)
at ServerResponse.location (~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express/lib/response.js:835:8)
at ServerResponse.redirect (~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express/lib/response.js:874:8)
at Query.<anonymous> (~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/controllers/admin.js:504:10)
at ~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/mongoose/node_modules/kareem/index.js:177:19
at ~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/mongoose/node_modules/kareem/index.js:109:16
at nextTickCallbackWith0Args (node.js:415:9)
at process._tickDomainCallback (node.js:385:13)Tested UserAgents:
- Mozilla/5.0 (X11; Linux x86_64; rv:42.0) Gecko/20100101 SeaMonkey/2.39
- Mozilla/5.0 (X11; Linux x86_64; rv:44.0) Gecko/20100101 Firefox/44.0
- Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.97 Safari/537.36
- Probably more when it's this many browsers
Anchor tags with these exact same values work 100%... just not redirect() in express's res Object... so it's a bit baffling.
Thanks for you patience and assistance with the provided intel. :)