dns: fix issues in dns benchmark #14936

ian-perkins · 2017-08-19T03:47:01Z

The benchmark script for dns contained functions with args declared
but never used. This fix removes those arguments from the function
signatures.

No test existed for the dns benchmark so one was added to the
parallel suite.

The dns benchmark uses the core dns.lookup() function which does not
access the network but instead uses "an operating system facility
that can associate names with addresses, and vice versa" e.g. similar
to ping; however, it is a synchronous call which runs on the libuv
threadpool - the number of test calls was therefore reduced to 5e4
(from 5e6).

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
commit message follows commit guidelines

Affected core subsystem(s)

Trott · 2017-08-19T04:04:00Z

test/parallel/test-benchmark-dns.js

+                          { NODEJS_BENCHMARK_ZERO_ALLOWED: 1 });
+
+const child = fork(runjs,
+                   ['--set', 'n=5e4',


We just want to make sure the thing runs so I think this could be n=1.

Could you also add:

'--set', 'all=false', '--set', 'name=127.0.0.1',

so as to minimize the breadth of the benchmark even further.

Trott · 2017-08-19T04:04:09Z

The most important fix is that this changes the script from one that doesn't work to one that can actually run. :-D

Trott · 2017-08-19T04:05:01Z

Before this commit:

$ ./node benchmark/run.js dns

dns/lookup.js
/Users/trott/io.js/benchmark/common.js:80
        throw new TypeError(`configuration "${key}" had type ${typeof value}`);
        ^

TypeError: configuration "all" had type boolean
    at recursive (/Users/trott/io.js/benchmark/common.js:80:15)
    at recursive (/Users/trott/io.js/benchmark/common.js:91:9)
    at Benchmark._queue (/Users/trott/io.js/benchmark/common.js:99:5)
    at new Benchmark (/Users/trott/io.js/benchmark/common.js:18:21)
    at Object.exports.createBenchmark (/Users/trott/io.js/benchmark/common.js:7:10)
    at Object.<anonymous> (/Users/trott/io.js/benchmark/dns/lookup.js:6:22)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)
    at tryModuleLoad (module.js:470:12)
$

After this commit, it actually runs.

Trott · 2017-08-19T04:06:07Z

And after:

$ ./node benchmark/run.js dns

dns/lookup.js
dns/lookup.js n=5000000 all="true" name="": 4,485,516.774544048
dns/lookup.js n=5000000 all="false" name="": 4,614,725.86397749
dns/lookup.js n=5000000 all="true" name="127.0.0.1": 2,519,219.886801595
dns/lookup.js n=5000000 all="false" name="127.0.0.1": 2,523,608.1753037865
dns/lookup.js n=5000000 all="true" name="::1": 2,633,747.082621402
dns/lookup.js n=5000000 all="false" name="::1": 2,705,813.055286177
$

🎉 Thanks, @ian-perkins!

Trott

LGTM. Welcome @ian-perkins and thanks.

Trott · 2017-08-19T04:08:40Z

Tiny nit: The subsystem here should likely be benchmark rather than dns. If you don't mind changing it in the commit message, you'll save whoever lands this a few keystrokes.

Trott · 2017-08-19T04:09:03Z

CI: https://ci.nodejs.org/job/node-test-pull-request/9747/

BridgeAR · 2017-08-19T04:23:01Z

benchmark/dns/lookup.js

@@ -5,7 +5,7 @@ const lookup = require('dns').lookup;

 const bench = common.createBenchmark(main, {
  name: ['', '127.0.0.1', '::1'],
-  all: [true, false],
+  all: ['true', 'false'],


This actually breaks the intention of the Boolean. The value will be coerced to a Boolean again in line 15 and !!'false' === true. So you might just change this to [1, 0] instead.

@BridgeAR Oh, good catch.

Another possibility is to leave them as strings but change the assignment to something like: const all = conf.all === 'true' ? true : false;

To me, this has the advantage of being more readable and also seems to be at least a little bit in line with what some other benchmarks do. Of course, @BridgeAR's suggestion works too if that seems preferable.

Trott

Need to address the !!'false' === true thing noted by @BridgeAR. LGTM otherwise.

refack

LGTM % strings && 5e4 comments addressed

BridgeAR · 2017-08-23T19:26:27Z

@ian-perkins thanks a lot for following up on the comments. I must ask you to rebase though as described here as we can otherwise not cleanly apply your changes. Keep up the good work!

Looks like requests have been addressed, clearing my request for changes.

Trott · 2017-08-23T23:27:38Z

@BridgeAR Looks like this has no conflicts with master anymore. PTAL.

jasnell · 2017-08-23T23:36:59Z

hmm..looks like the conflicts were resolved by merging and not rebasing.

Trott · 2017-08-24T01:12:56Z

hmm..looks like the conflicts were resolved by merging and not rebasing.

Ah! Yes, extra commits, now I see. Yeah, ignore me, and @ian-perkins please rebase when you get a chance. :-D

ian-perkins · 2017-08-24T06:40:44Z

I'll try to do this today but can I first check what the rebase should do? Is it to squash all the commits I made into one? I read the referred doc and it mentions commits being squashed on landing so I suspect that I need to do something different in this case? Sorry for the basic nature of my questions...

Trott · 2017-08-24T06:50:39Z

@ian-perkins The rebase is to synch up your branch with the master branch on the upstream repository, and then re-apply your changes on top of that. (Hope that answers your question, but if not, feel free to ask clarifying questions here or hop on IRC to see if someone can help you!)

The benchmark script for dns contained functions with args declared but never used. This fix removes those arguments from the function signatures. No test existed for the dns benchmark so one was added to the parallel suite. The dns benchmark uses the core dns.lookup() function which does not access the network but instead uses "an operating system facility that can associate names with addresses, and vice versa" e.g. similar to ping; however, it is a synchronous call which runs on the libuv threadpool - the number of test calls was therefore reduced to 5e4 (from 5e6).

The benchmark script for dns contained functions with args declared but never used. This fix removes those arguments from the function signatures. No test existed for the dns benchmark so one was added to the parallel suite. The dns benchmark uses the core dns.lookup() function which does not access the network. To improve performance the tests are limited to 1 invocation to a single endpoint.

ian-perkins · 2017-08-24T07:29:17Z

I did the rebase and pushed to my fork - hopefully it's ok...

Trott · 2017-08-24T22:02:05Z

@BridgeAR PTAL

BridgeAR

LGTM

Trott · 2017-08-24T22:23:42Z

CI: https://ci.nodejs.org/job/node-test-pull-request/9826/

The benchmark script for dns contained functions with args declared but never used. This fix removes those arguments from the function signatures. No test existed for the dns benchmark so one was added to the parallel suite. To improve performance the tests are limited to 1 invocation to a single endpoint. PR-URL: #14936 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Refael Ackermann <refack@gmail.com>

BridgeAR · 2017-08-25T22:02:36Z

Landed in 15ac1ea

@ian-perkins congratulations on your first contribution! 🎉

The benchmark script for dns contained functions with args declared but never used. This fix removes those arguments from the function signatures. No test existed for the dns benchmark so one was added to the parallel suite. To improve performance the tests are limited to 1 invocation to a single endpoint. PR-URL: nodejs/node#14936 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Refael Ackermann <refack@gmail.com>

The benchmark script for dns contained functions with args declared but never used. This fix removes those arguments from the function signatures. No test existed for the dns benchmark so one was added to the parallel suite. To improve performance the tests are limited to 1 invocation to a single endpoint. PR-URL: #14936 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Refael Ackermann <refack@gmail.com>

nodejs-github-bot added benchmark Issues and PRs related to the benchmark subsystem. dns Issues and PRs related to the dns subsystem. test Issues and PRs related to the tests. labels Aug 19, 2017

Trott reviewed Aug 19, 2017

View reviewed changes

Trott approved these changes Aug 19, 2017

View reviewed changes

BridgeAR requested changes Aug 19, 2017

View reviewed changes

Trott previously requested changes Aug 19, 2017

View reviewed changes

refack approved these changes Aug 19, 2017

View reviewed changes

ian-perkins force-pushed the dns-benchmark branch from d2c9298 to 2d3e175 Compare August 24, 2017 07:21

ian-perkins force-pushed the dns-benchmark branch from 2d3e175 to 83c6d6b Compare August 24, 2017 07:37

BridgeAR approved these changes Aug 24, 2017

View reviewed changes

BridgeAR self-assigned this Aug 25, 2017

BridgeAR closed this Aug 25, 2017

This was referenced Sep 1, 2017

benchmark: refactor to remove duplicate code #15121

Closed

test, benchmark: create shared runBenchmark func #15004

Closed

MylesBorins mentioned this pull request Sep 10, 2017

v8.5.0 proposal #15308

Merged

MylesBorins added the dont-land-on-v6.x label Sep 20, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dns: fix issues in dns benchmark #14936

dns: fix issues in dns benchmark #14936

ian-perkins commented Aug 19, 2017 •

edited by refack

Loading

Trott Aug 19, 2017

refack Aug 19, 2017

Trott commented Aug 19, 2017

Trott commented Aug 19, 2017

Trott commented Aug 19, 2017

Trott left a comment

Trott commented Aug 19, 2017

Trott commented Aug 19, 2017

BridgeAR Aug 19, 2017

Trott Aug 19, 2017

Trott Aug 19, 2017 •

edited

Loading

Trott left a comment

refack left a comment

BridgeAR commented Aug 23, 2017

Trott commented Aug 23, 2017

jasnell commented Aug 23, 2017

Trott commented Aug 24, 2017

ian-perkins commented Aug 24, 2017

Trott commented Aug 24, 2017

ian-perkins commented Aug 24, 2017

Trott commented Aug 24, 2017

BridgeAR left a comment

Trott commented Aug 24, 2017

BridgeAR commented Aug 25, 2017

dns: fix issues in dns benchmark #14936

dns: fix issues in dns benchmark #14936

Conversation

ian-perkins commented Aug 19, 2017 • edited by refack Loading

Checklist

Affected core subsystem(s)

Trott Aug 19, 2017

Choose a reason for hiding this comment

refack Aug 19, 2017

Choose a reason for hiding this comment

Trott commented Aug 19, 2017

Trott commented Aug 19, 2017

Trott commented Aug 19, 2017

Trott left a comment

Choose a reason for hiding this comment

Trott commented Aug 19, 2017

Trott commented Aug 19, 2017

BridgeAR Aug 19, 2017

Choose a reason for hiding this comment

Trott Aug 19, 2017

Choose a reason for hiding this comment

Trott Aug 19, 2017 • edited Loading

Choose a reason for hiding this comment

Trott left a comment

Choose a reason for hiding this comment

refack left a comment

Choose a reason for hiding this comment

BridgeAR commented Aug 23, 2017

Trott commented Aug 23, 2017

jasnell commented Aug 23, 2017

Trott commented Aug 24, 2017

ian-perkins commented Aug 24, 2017

Trott commented Aug 24, 2017

ian-perkins commented Aug 24, 2017

Trott commented Aug 24, 2017

BridgeAR left a comment

Choose a reason for hiding this comment

Trott commented Aug 24, 2017

BridgeAR commented Aug 25, 2017

ian-perkins commented Aug 19, 2017 •

edited by refack

Loading

Trott Aug 19, 2017 •

edited

Loading