Performance regression of instanceof in v7 and master

[mcollina]

• Version: v7 and master
• Platform: Mac
• Subsystem:

instanceof checks has become almost 100 times slower in Node v7+ (and current master)

var r = /hello/

console.time('instanceof RegExp')
for (var i = 0; i < 100000000; i++) {
  r instanceof RegExp
}
console.timeEnd('instanceof RegExp')

var o = {}

console.time('instanceof Object')
for (var i = 0; i < 100000000; i++) {
  o instanceof Object
}
console.timeEnd('instanceof Object')

In node v6:

$ node instanceof.js
instanceof RegExp: 133.519ms
instanceof Object: 134.572ms

In node v7:

$ node instanceof.js
instanceof RegExp: 9858.616ms
instanceof Object: 9839.696ms

I know this is a problem in V8, but I think it's good to track it here as well.

v8 issue: https://bugs.chromium.org/p/v8/issues/detail?id=5640

cc @fhinkel

[targos]

Could you also create a V8 bug and link to it here ?

[mcollina]

@targos done, thanks.

[targos]

I cannot reproduce this regression in Chrome 54.

[mcollina]

Now that I test it... neither can I, and in theory they use the same v8 version.
Maybe there is a patch to backport to the v8 version we are using.

[bnoordhuis]

Found the cause: commit 2a4b068 from last September, cc @addaleax.

Chrome 54 ships the exact same version as node.js master, 5.4.500.41, and I can confirm that it's fast in Chrome and d8 (./configure --enable-d8) but not in Node.js.

I checked with perf(1) and it showed that V8 was spending a phenomenal amount of time in v8::internal::Object::InstanceOf(), more specifically the code path that checks for the presence of a @@hasInstance method. Here is a minimal test case:

function F() {}
const hasInstance = Function.prototype[Symbol.hasInstance];
Object.defineProperty(F, Symbol.hasInstance, { value: hasInstance });
for (var o = {}, i = 0; i < 1e8; i++) o instanceof Object;

Basically, if V8 sees that @@hasInstance is used, it starts emitting conservative slow path code.

[mcollina]

Oh no! :(

We have some possible solutions:

a. revert that fix, but probably it's semver-major. If we could do it semver-minor it will be the best solution.
b. wait until there is a patch from upstream V8, and apply that.

It would be nice to a quick post-mortem on this one. How can we spot those before releasing?

[addaleax]

Huh… yeah, I did not see that coming.

[addaleax]

(sorry, mis-clicked and posted the comment too early)

How can we spot those before releasing?

• Does V8 have benchmarks for this kind of thing? Is running them inside Node helpful?
• Would people from the V8 team have spotted that this is problematic?
• @mcollina I’m curious – how did you spot this?

[mcollina]

@mcollina I’m curious – how did you spot this?

I found out that https://github.com/mcollina/bloomrun was 10 times slower in v7 rather than v6. It boiled down to instanceof, I did not do a git bisect on core, and I should have (that's awesome @bnoordhuis!!!).

[bnoordhuis]

Truthfully, I didn't bisect. I had a pretty good idea from looking at the perf data where the problem came from and reverting the commit confirmed that. :-)

[bmeurer]

I think I can come up with a small(ish) mitigation for the problem. Let me have a look.

[mcollina]

@bmeurer I can confirm the patch works as expected, and it's fairly straightforward to apply.

[bmeurer]

@mcollina Awesome. I'm currently fixing the issue in TurboFan as well. The patch is slightly more involved there for a couple of reasons. But you should be fine with the Crankshaft fix for 99% of the cases.

[targos]

I'm currently fixing the issue in TurboFan as well.

About that, I wonder: is TurboFan enabled by default in V8? If so, how do we know what parts of it are shipped without a flag? I'm asking because I think I never saw a [compiling method ... using TurboFan] message without the --turbo flag and things are usually slower with the flag enabled.

[bmeurer]

It is for language features that Crankshaft cannot deal with, i.e. try-catch, try-finally, for-of, etc.

[mhdawson]

Would this have been caught by an octane run ? If so it might make sense to set that up as one of our nightly benchmarks. Now that we have our own v8 branch were all of the changes applied I'm guessing it should re relatively easy to do that.

[bnoordhuis]

I did a quick check with the EarleyBoyer benchmark, it has a lot of instanceof checks. Before/after numbers when run in node (not d8):

EarleyBoyer: 5219  # master
EarleyBoyer: 24491  # with 2a4b068 reverted

So an unequivocal 'yes' in this particular case. :-)

[mhdawson]

Will look at getting octane added as part of our nightly benchmarks in nodejs/benchmarking#75.

[mcollina]

Good idea!

[mcollina]

The issue is marked as fixed in v8. Can we backport the fix here? What is the process for v8 patches and backports?

[targos]

@fhinkel any chance for it to be backported to V8 5.4 or 5.5 ?

[fhinkel]

@natorion Can we backport performance improvements to 5.4 or only bug fixes?

If we can't backport it to V8, we can always cherry-pick the commits to Node. We probably want to wait a few days though, in case there are any issues with the commits.

[natorion]

For 5.4 it is probably to late. For 5.5 it should be fine (the CS part) as this is still in beta. Need to check though. What is the benefit for Node if this is merged to 5.5 though?

BTW, this also needs to be fixed on 5.6.

[fhinkel]

Should we wait until #9697 has landed?

[mcollina]

We need to get this into node v7.

[bnoordhuis]

We will probably upgrade to V8 5.5 during the v7 release cycle if we can work out the kinks, see #9618.

[bengl]

It looks like this might have regressed back to a 50(ish)x slowdown for RegExp on the Node 8 nightlies, and for non-builtins (which I added a test for in @mcollina's original example code), it seems to be a 10(ish)x slowdown on both Node 7 and Node 8 nightlies. (Should I open a new issue?)

System: Linux 4.9.11 x86_64

$ cat instanceof.js 
var r = /hello/

console.time('instanceof RegExp')
for (var i = 0; i < 100000000; i++) {
  r instanceof RegExp
}
console.timeEnd('instanceof RegExp')

var o = {}

console.time('instanceof Object')
for (var i = 0; i < 100000000; i++) {
  o instanceof Object
}
console.timeEnd('instanceof Object')

function F () {}
var f = new F

console.time('instanceof F')
for (var i = 0; i < 100000000; i++) {
  f instanceof F
}
console.timeEnd('instanceof F')
$ ~/.nvm/versions/node/v6.10.2/bin/node instanceof.js 
instanceof RegExp: 127.840ms
instanceof Object: 129.514ms
instanceof F: 635.539ms
$ ~/.nvm/versions/node/v7.9.0/bin/node instanceof.js 
instanceof RegExp: 163.946ms
instanceof Object: 120.818ms
instanceof F: 6786.285ms
$ ~/.nvm/versions/node/v8.0.0-nightly2017042158066d16d5/bin/node instanceof.js 
instanceof RegExp: 7501.615ms
instanceof Object: 121.429ms
instanceof F: 7219.541ms

[addaleax]

Should I open a new issue?

@bengl Yes, that seems like a good idea (and ping nodejs/v8 in it:))

[hashseed]

This is https://bugs.chromium.org/p/v8/issues/detail?id=5902

I'll merge this on Monday to 5.8.

[hashseed]

Merged to V8 version 5.8: https://chromium.googlesource.com/v8/v8.git/+/e62656ffe685408e06d175ff5f371c8fd2556b58

[hashseed]

Also see https://bugs.chromium.org/p/v8/issues/detail?id=6295

[mcollina]

@hashseed fantastic!