querystring: don't inherit from Object.prototype #6055

mscdex · 2016-04-05T12:25:29Z

Pull Request check-list

Does make -j8 test (UNIX) or vcbuild test nosign (Windows) pass with
this change (including linting)?
Is the commit message formatted according to CONTRIBUTING.md?
If this change fixes a bug (or a performance problem), is a regression
test (or a benchmark) included?

Affected core subsystem(s)

querystring

Description of change

This commit safely allows querystring keys that are named the same as
properties that are ordinarily inherited from Object.prototype such
as proto. Additionally, this commit provides a bit of a speed
improvement (~25% in the querystring-parse 'manypairs' benchmark)
when there are many unique keys.

Fixes: #5642

mscdex · 2016-04-05T12:26:51Z

/cc @jasnell @WebReflection

cjihrig · 2016-04-05T12:31:53Z

lib/querystring.js

@@ -5,6 +5,18 @@
 const QueryString = exports;
 const Buffer = require('buffer').Buffer;

+// This constructor is used to store parsed query string values. Instantiating
+// this is faster than explicitly calling `Object.create(null)` to get a
+// "clean" empty object


Could you add a note about the version of V8 that this applies to.

cjihrig · 2016-04-05T12:32:09Z

LGTM pending CI

WebReflection · 2016-04-05T12:32:39Z

lib/querystring.js

+// "clean" empty object
+function ParsedQueryString() {}
+ParsedQueryString.prototype = Object.create(null, {
+  constructor: {


what's with the constructor if I might ask? why bothering?

Also, I think you still need to return an instanceof Object to avoid breaking code using this module.

to be more specific:

function ParsedQueryString() {} ParsedQueryString.prototype = Object.create(null); // before returning, if we don't want outer code to break return Object.setPrototypeOf(obj, Object.prototype);

constructor is removed now.

IMHO I think more people are going to be using typeof foo === 'object' rather than foo instanceof Object. Perhaps @ChALkeR can give us some insight about this?

It's not about instanceof per-se, it's about .hasOwnProperty and other
methods from Object.prototype that most devs give, wrongly, for granted
with objects. This won't affect my code but it will break every code with
for/in loops. I personally don't care
On Apr 5, 2016 1:59 PM, "Brian White" notifications@github.com wrote:

In lib/querystring.js
#6055 (comment):

@@ -5,6 +5,18 @@
const QueryString = exports;
const Buffer = require('buffer').Buffer;

+// This constructor is used to store parsed query string values. Instantiating
+// this is faster than explicitly calling Object.create(null) to get a
+// "clean" empty object
+function ParsedQueryString() {}
+ParsedQueryString.prototype = Object.create(null, {

constructor: {

constructor is removed now.

IMHO I think more people are going to be using typeof foo === 'object'
rather than foo instanceof Object. Perhaps @ChALkeR
https://github.com/ChALkeR can give us some insight about this?

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/nodejs/node/pull/6055/files/ef5f84a85820794a572ba0deb7e15a8b1622b9f4#r58532723

I'm a bit concerned about not setting the object prototype also. We need to get some insight into just how significant of a change that is. Either way, this is definitely semver-major because of that change.

jasnell · 2016-04-05T13:08:27Z

LGTM but I'd like to get more insight into what impact not setting the prototype on return would have.

mscdex · 2016-04-05T13:24:21Z

It'd definitely be a semver-major thing. I just ran some benchmarks with and without setting the prototype on return and there is definitely a noticeable performance regression when setting the prototype on return in almost all of the benchmark cases.

jasnell · 2016-04-05T13:26:35Z

Yep, I was consistently seeing the same thing. I'm happy with the change, just want to make sure we know what'll break

mscdex · 2016-04-05T13:31:56Z

Also for completeness here are benchmark results I am getting as this PR currently stands (no setting prototype on return) with the number of iterations bumped from 1e6 to 10e6:

querystring/querystring-parse.js type=noencode n=10000000: ./node: 689650 ./node-pre-qs: 597360 ...... 15.45%
querystring/querystring-parse.js type=multicharsep n=10000000: ./node: 633240 ./node-pre-qs: 534580 .. 18.46%
querystring/querystring-parse.js type=encodemany n=10000000: ./node: 367970 ./node-pre-qs: 301350 .... 22.11%
querystring/querystring-parse.js type=encodelast n=10000000: ./node: 563560 ./node-pre-qs: 424940 .... 32.62%
querystring/querystring-parse.js type=multivalue n=10000000: ./node: 540460 ./node-pre-qs: 502680 ..... 7.52%
querystring/querystring-parse.js type=multivaluemany n=10000000: ./node: 243350 ./node-pre-qs: 232420 . 4.70%
querystring/querystring-parse.js type=manypairs n=10000000: ./node: 189290 ./node-pre-qs: 151430 ..... 25.00%

Fishrock123 · 2016-04-05T14:16:44Z

Interesting. If this works out well, we should look into doing a similar thing for events, per #728.

Does this break any sort of enumeration i.e. Object.getOwnPropertyNames()?

Another thought, what if we were able to expose ourselves something like Object.create(null) more directly from v8?

mscdex · 2016-04-05T14:29:20Z

@Fishrock123 RE: EventEmitter, I already had that in mind, was going to test it later today.

WebReflection · 2016-04-05T15:12:23Z

Does this break any sort of enumeration i.e. Object.getOwnPropertyNames()?

No, it doesn't affect any Object or Reflect operation.

what if we were able to expose ourselves something like Object.create(null) more directly from v8?

I don't know why Dictionary or Null constructors aren't natively available, these both have been used for ages to create null objects, having them native would be awesome.

// forcing `new Dictionary` initialization
class Dictionary {}
delete Object.setPrototypeOf(
  Dictionary.prototype,
  null
).constructor;

Sinewyk · 2016-04-05T16:55:46Z

Related: #6044

Fishrock123 · 2016-04-07T14:32:57Z

(same as #6092 (comment)) Hmmm, can this be back-ported? Theoretically, this could be considered a security issue for unvalidated input.

mscdex · 2016-04-07T16:31:04Z

citgm: https://ci.nodejs.org/job/thealphanerd-smoker/186/

jasnell · 2016-04-07T16:42:17Z

This LGTM if CI and CITGM look good. I'm currently -1 on backporting until we can get a better sense about what breaks with this change.

mscdex · 2016-04-07T18:28:31Z

CI: https://ci.nodejs.org/job/node-test-pull-request/2208/

mscdex · 2016-04-18T23:20:55Z

CI one last time: https://ci.nodejs.org/job/node-test-pull-request/2304/

jasnell · 2016-04-18T23:22:32Z

LGTM but we should also get a CITGM run on it. /cc @thealphanerd

mscdex · 2016-04-18T23:23:26Z

I ran citgm 11 days ago, but we can do it again I suppose.

jasnell · 2016-04-18T23:27:16Z

oh! ha! missed that comment :-)

jasnell · 2016-04-19T01:11:54Z

CI is green

Trott · 2016-04-19T22:20:18Z

This PR appears to introduce an unfortunate side effect in lib/url.js where sometimes the query property will have a prototype (if there is no query string in the URL) and sometimes it won't. I'll be offering a fix for it in #6213, but if that doesn't land, we should still fix it...

This commit safely allows querystring keys that are named the same as properties that are ordinarily inherited from Object.prototype such as __proto__. Additionally, this commit provides a bit of a speed improvement (~25% in the querystring-parse 'manypairs' benchmark) when there are many unique keys. Fixes: nodejs#5642 PR-URL: nodejs#6055 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

This commit safely allows querystring keys that are named the same as properties that are ordinarily inherited from Object.prototype such as __proto__. Additionally, this commit provides a bit of a speed improvement (~25% in the querystring-parse 'manypairs' benchmark) when there are many unique keys. Fixes: #5642 PR-URL: #6055 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

The default behaviour was changed with node version 6. https://github.com/nodejs/node/blob/master/doc/changelogs/CHANGELOG_V6.md nodejs/node#6055

Don't use Object.prototpye.hasOwnProperty on object returned by querystring.parse Since it does not inherit from Object.prototype since nodejs/node#6055 (node v6.0.0+)

Previously, we assumed that whatever object the user passed to the object assertion helpers had its prototype set to Object and would thus have things like a hasOwnProperty property. However, it turns out that this is not a valid assumption. This isn't limited to just corner cases; Node 6.x changed the behavior of its `querystring` module (which is also used by its `url` module) to explicitly set the prototype of querystring objects to `null` in order to prevent key collisions if someone sent a URL query string parameter with the same key as a property on the prototype. See nodejs/node#6055 for where this change was made. Therefore, we use Object.prototype.hasOwnProperty, which is guaranteed to always exist, and call it on the object the user passed in. We do the same for propertyIsEnumerable as applicable.

After V8 5.6, using Object.create(null) directly is now faster than using a constructor for map-like objects. PR-URL: #11930 Refs: emberjs/ember.js#15001 Refs: https://crrev.com/532c16eca071df3ec8eed394dcebb932ef584ee6 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

franciscolourenco · 2020-10-13T18:01:47Z

Did this solve a security issue, or just about avoiding name conflicts and supporting more property names?

mscdex added the querystring Issues and PRs related to the built-in querystring module. label Apr 5, 2016

cjihrig reviewed Apr 5, 2016
View reviewed changes

WebReflection reviewed Apr 5, 2016
View reviewed changes

mscdex force-pushed the querystring-parse-empty-prototype branch 2 times, most recently from ab2ca7c to 31d52c8 Compare April 5, 2016 12:57

jasnell added the semver-major PRs that contain breaking changes and should be released in the next major version. label Apr 5, 2016

ChALkeR added the performance Issues and PRs related to the performance of Node.js. label Apr 5, 2016

Sinewyk mentioned this pull request Apr 5, 2016

querystring: allow querystring parse to handle __proto__ #6044

Closed

3 tasks

mscdex added this to the 6.0.0 milestone Apr 6, 2016

mscdex removed the performance Issues and PRs related to the performance of Node.js. label Apr 7, 2016

mscdex mentioned this pull request Apr 7, 2016

http: don't inherit from Object.prototype #6102

Closed

3 tasks

mscdex deleted the querystring-parse-empty-prototype branch April 19, 2016 01:44

jasnell mentioned this pull request Apr 19, 2016

querystring: avoid conflicts with Object.prototype and __proto__ #5722

Closed

navarroaxel mentioned this pull request Apr 28, 2016

Is not compatible with nodejs 6.0 prerender/prerender#308

Closed

artembykov mentioned this pull request Apr 29, 2016

TypeError: params.hasOwnProperty is not a function havard/node-openid#148

Closed

iyuq mentioned this pull request Apr 29, 2016

fix to pass the test at node v6.x koajs/koa#723

Merged

rickharrison mentioned this pull request May 27, 2016

this.query no longer has hasOwnProperty method in node 6.2 koajs/koa#747

Closed

robinbullocks4rb mentioned this pull request May 30, 2016

Error thrown when input object does not inherit from Object.prototype jonschlinkert/rename-keys#7

Closed

teohhanhui mentioned this pull request Jun 17, 2016

Don't use Object.prototpye.hasOwnProperty on query finboxio/koa-prerender#1

Merged

kashav mentioned this pull request Aug 6, 2016

Use hasOwnProperty from Object prototype (Node v6.x) winstonjs/winston-mongodb#71

Merged

AdriVanHoudt mentioned this pull request Sep 6, 2016

isPlainObject node 4 vs 6 lodash/lodash#2635

Closed

tony-kerz mentioned this pull request Sep 11, 2016

Error: obj.hasOwnProperty is not a function axios/axios#436

Closed

targos mentioned this pull request Sep 23, 2016

6.x querystring.parse returns object for which instanceof Object is false (was true for 4.x) #8737

Closed

alexcorre mentioned this pull request Nov 28, 2016

Node v6 - TypeError thrown when validating parsed querystring objects mac-/ratify#51

Closed

rainesinternationaldev mentioned this pull request Dec 8, 2016

Fix exception on Node 6.x bibanul/koa-prerender#1

Open

naugtur mentioned this pull request Jun 20, 2017

Is it time to deprecate Object.prototype extension? shouldjs/should.js#146

Open

emilymcafee mentioned this pull request Apr 8, 2018

upgrade to node6 mapbox/pagerduty#4

Merged

3 tasks

abovedave mentioned this pull request Jun 7, 2018

HTTP2 error on req.headers dadi/logger#56

Closed

This was referenced Jun 19, 2020

ctx.query is not a plain object while ctx.params and ctx.request.body is koajs/koa#1472

Closed

Node6 incompatibilities with querystrings js-data/js-data#571

Open

franciscolourenco mentioned this pull request Oct 13, 2020

Cannot read property 'name' of undefined steelbrain/php-serialize#258

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

querystring: don't inherit from Object.prototype #6055

querystring: don't inherit from Object.prototype #6055

mscdex commented Apr 5, 2016

mscdex commented Apr 5, 2016

cjihrig Apr 5, 2016

cjihrig commented Apr 5, 2016

WebReflection Apr 5, 2016

WebReflection Apr 5, 2016

mscdex Apr 5, 2016

WebReflection Apr 5, 2016

jasnell Apr 5, 2016

jasnell commented Apr 5, 2016

mscdex commented Apr 5, 2016

jasnell commented Apr 5, 2016

mscdex commented Apr 5, 2016

Fishrock123 commented Apr 5, 2016

mscdex commented Apr 5, 2016

WebReflection commented Apr 5, 2016

Sinewyk commented Apr 5, 2016

Fishrock123 commented Apr 7, 2016

mscdex commented Apr 7, 2016

jasnell commented Apr 7, 2016

mscdex commented Apr 7, 2016

mscdex commented Apr 18, 2016

jasnell commented Apr 18, 2016

mscdex commented Apr 18, 2016

jasnell commented Apr 18, 2016

jasnell commented Apr 19, 2016

Trott commented Apr 19, 2016

franciscolourenco commented Oct 13, 2020

querystring: don't inherit from Object.prototype #6055

querystring: don't inherit from Object.prototype #6055

Conversation

mscdex commented Apr 5, 2016

Pull Request check-list

Affected core subsystem(s)

Description of change

mscdex commented Apr 5, 2016

cjihrig Apr 5, 2016

Choose a reason for hiding this comment

cjihrig commented Apr 5, 2016

WebReflection Apr 5, 2016

Choose a reason for hiding this comment

WebReflection Apr 5, 2016

Choose a reason for hiding this comment

mscdex Apr 5, 2016

Choose a reason for hiding this comment

WebReflection Apr 5, 2016

Choose a reason for hiding this comment

jasnell Apr 5, 2016

Choose a reason for hiding this comment

jasnell commented Apr 5, 2016

mscdex commented Apr 5, 2016

jasnell commented Apr 5, 2016

mscdex commented Apr 5, 2016

Fishrock123 commented Apr 5, 2016

mscdex commented Apr 5, 2016

WebReflection commented Apr 5, 2016

Sinewyk commented Apr 5, 2016

Fishrock123 commented Apr 7, 2016

mscdex commented Apr 7, 2016

jasnell commented Apr 7, 2016

mscdex commented Apr 7, 2016

mscdex commented Apr 18, 2016

jasnell commented Apr 18, 2016

mscdex commented Apr 18, 2016

jasnell commented Apr 18, 2016

jasnell commented Apr 19, 2016

Trott commented Apr 19, 2016

franciscolourenco commented Oct 13, 2020