benchmark: add --format csv option #7961

adrian-nitu-92 · 2016-08-03T14:20:49Z

Checklist

make -j4 test (UNIX), or vcbuild test nosign (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

Affected core subsystem(s)

benchmark

Description of change

Added the option of using --format csv when outputting data.

AndreasMadsen · 2016-08-03T15:34:33Z

I don't think we want to add extra arguments to each individual benchmark. That is a lot of extra complexity in parsing and output and it is not needed since benchmark/run.js or benchmark/scatter.js can be used to get the same result. Modifying only benchmark/run.js should be sufficient, I'm sorry if that wasn't clear.

jasnell · 2016-08-03T16:23:30Z

+1 to what @AndreasMadsen is saying here. I think there's an easier way to do this.

adrian-nitu-92 · 2016-08-04T09:03:57Z

Fair point :), reduced the change set.
I added a secondary commit that fixes some typos and adds a set of missing {}.

addaleax · 2016-08-04T11:14:00Z

benchmark/run.js

 `, {
  arrayArgs: ['set']
 });
 const benchmarks = cli.benchmarks();
+const validFormats = ['csv', 'default'];


Should this be 'simple'?

It could also be removed :))), but I wanted to add something to not let 'csv' alone :D. Anything should be good, it is an option I doubt would be used. Should I make the change?

Huh? I meant, in the usage text you added --format as simple/csv, but this array doesn’t seem to match that. Sorry if that’s been unclear (or I’m just misunderstanding this in general?)

Oh, yea, my bad, thank you for spotting the inconsistency :D

AndreasMadsen · 2016-08-04T12:57:29Z

@adrian-nitu-92 Yes, this is much better. I've sugested some changes, you will also have to:

remove console.log(filename); in the csv case.
add csv header: console.log('"filename", "configuration", "rate", "time"'); should do.

edit: could you @intel.com email to your GitHub account. I don't know if we require the commits to link to your user account, but it would be nice.

adrian-nitu-92 · 2016-08-04T16:04:03Z

Updated the PR with all changes, except removing the variable validFormats :)

AndreasMadsen · 2016-08-04T20:33:06Z

benchmark/run.js

 (function recursive(i) {
  const filename = benchmarks[i];
  const child = fork(path.resolve(__dirname, filename), cli.optional.set);

  console.log();
-  console.log(filename);
+  if (format == 'csv') {
+    console.log('"filename","configuration","rate","time"');


This is going to output a csv header for every file. A csv header should only appear once.

adrian-nitu-92 · 2016-08-05T10:23:11Z

Updated PR :)

AndreasMadsen · 2016-08-05T10:28:29Z

Great. Be sure to run make lint, I did see at least one lint error.

AndreasMadsen · 2016-08-06T16:30:56Z

CI: https://ci.nodejs.org/job/node-test-pull-request/3554/

AndreasMadsen · 2016-08-06T16:35:40Z

benchmark/run.js

+  return;
+}
+
+if (format == 'csv') {


@Trott Does this really match our lint rules? The linter says it does. https://ci.nodejs.org/job/node-test-linter/3613/console

Yes, as far as I can tell, that is consistent with our lint rules. (I would prefer this be === rather than == but that's not lint-enforced.)

So would I. @adrian-nitu-92 could you fix this, I think there is another one too.

AndreasMadsen · 2016-08-06T17:58:07Z

/cc @nodejs/benchmarking @addaleax @mscdex who participated in the original issue. I need to run this to be sure (can't right now), but besides a "lint" nit this looks good.

Refs: nodejs#7961 (comment)

adrian-nitu-92 · 2016-08-08T10:27:44Z

Ou, I get it now :D. Made the requested changes :)

AndreasMadsen · 2016-08-08T14:33:48Z

benchmark/run.js

@@ -10,31 +10,54 @@ const cli = CLI(`usage: ./node run.js [options] [--] <category> ...

  --filter pattern          string to filter benchmark scripts
  --set    variable=value   set benchmark variable (can be repeated)
+  --format [simple|csv]      optional value that specifies the output format


Could you align the descriptions. ``optional => optional

AndreasMadsen · 2016-08-08T14:38:25Z

@adrian-nitu-92 awesome work. I have tried it out and it looks good.

As I'm not a very experienced node.js reviewer, thus I will give this 48 hours before merging, such other contributors have time to come with last remarks.

jasnell · 2016-08-08T14:51:13Z

LGTM

adrian-nitu-92 · 2016-08-09T08:20:10Z

benchmark/run.js

-
-    console.log(`${data.name}${conf}: ${data.rate}`);
+    // delete first space of the configuration
+    conf = conf.slice(1);


This line can be removed if I reformat 57&59. I like this version better because I don't think you should rely on spaces in strings you are concatenating.

Agreed, just keep it.

Refs: nodejs#7961 (comment) PR-URL: nodejs#8000 Reviewed-By: Brian White <mscdex@mscdex.net> Reviewed-By: Andreas Madsen <amwebdk@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Johan Bergström <bugs@bergstroem.nu>

Refs: #7961 (comment) PR-URL: #8000 Reviewed-By: Brian White <mscdex@mscdex.net> Reviewed-By: Andreas Madsen <amwebdk@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Johan Bergström <bugs@bergstroem.nu>

AndreasMadsen · 2016-08-11T10:15:12Z

LGTM

CI: https://ci.nodejs.org/job/node-test-pull-request/3621/

will merge, when ci is green.

AndreasMadsen · 2016-08-11T11:00:26Z

benchmark/run.js

+
+const validFormats = ['csv', 'simple'];
+const format = cli.optional.format || 'simple';
+if (!validFormats.includes(cli.optional.format)) {


should be !validFormats.includes(format) otherwise it will print Invalid format detected when there are no --format argument.

AndreasMadsen · 2016-08-11T11:01:56Z

I found a bug, I will merge when that gets fixed.

I noticed some typos and the lack of {} following an if. Signed-off-by: Adrian Nitu <adrian.nitu@intel.com>

Node documentation recommends using process.exitCode = x and returning as a way to exit. Signed-off-by: Adrian Nitu <adrian.nitu@intel.com>

Added the option of using --format csv when outputting data. Signed-off-by: Adrian Nitu <adrian.nitu@intel.com>

adrian-nitu-92 · 2016-08-12T10:40:01Z

Updated the PR and the branch merging point ( pulled nodejs latest and rebased on top of it, no conflicts detected). Also ran the whole tests again just to make sure I don't have additional slip-ups :D

Thanks Andreas for your help :D

AndreasMadsen · 2016-08-12T11:28:20Z

Thanks @adrian-nitu-92. Landed in 9e7fd8e 4b527a4 474e629

nodejs-github-bot added the benchmark Issues and PRs related to the benchmark subsystem. label Aug 3, 2016

adrian-nitu-92 mentioned this pull request Aug 3, 2016

OUTPUT_FORMAT=csv option is missing from benchmark/common.js #7890

Closed

addaleax reviewed Aug 4, 2016
View reviewed changes

AndreasMadsen reviewed Aug 4, 2016
View reviewed changes

AndreasMadsen reviewed Aug 6, 2016
View reviewed changes

Trott added a commit to Trott/io.js that referenced this pull request Aug 6, 2016

benchmark: favor === over ==

9712cd2

Refs: nodejs#7961 (comment)

Trott mentioned this pull request Aug 6, 2016

benchmark: favor === over == #8000

Closed

2 tasks

AndreasMadsen reviewed Aug 8, 2016
View reviewed changes

adrian-nitu-92 reviewed Aug 9, 2016
View reviewed changes

AndreasMadsen reviewed Aug 11, 2016
View reviewed changes

adrian-nitu-92 added 3 commits August 12, 2016 13:37

benchmark: fix comment typos and code format

032b872

I noticed some typos and the lack of {} following an if. Signed-off-by: Adrian Nitu <adrian.nitu@intel.com>

benchmark: update compare.js exit method

5b64f9b

Node documentation recommends using process.exitCode = x and returning as a way to exit. Signed-off-by: Adrian Nitu <adrian.nitu@intel.com>

benchmark: add --format csv option

fbcd137

Added the option of using --format csv when outputting data. Signed-off-by: Adrian Nitu <adrian.nitu@intel.com>

AndreasMadsen closed this Aug 12, 2016

AndreasMadsen added dont-land-on-v4.x labels Aug 15, 2016

gibfahn mentioned this pull request Jun 15, 2017

Auditing for 6.11.1 nodejs/Release#230

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

benchmark: add --format csv option #7961

benchmark: add --format csv option #7961

adrian-nitu-92 commented Aug 3, 2016

AndreasMadsen commented Aug 3, 2016

jasnell commented Aug 3, 2016

adrian-nitu-92 commented Aug 4, 2016

addaleax Aug 4, 2016

adrian-nitu-92 Aug 4, 2016 •

edited

Loading

addaleax Aug 4, 2016

adrian-nitu-92 Aug 4, 2016

AndreasMadsen commented Aug 4, 2016 •

edited

Loading

adrian-nitu-92 commented Aug 4, 2016 •

edited

Loading

AndreasMadsen Aug 4, 2016

adrian-nitu-92 commented Aug 5, 2016

AndreasMadsen commented Aug 5, 2016

AndreasMadsen commented Aug 6, 2016

AndreasMadsen Aug 6, 2016

Trott Aug 6, 2016

AndreasMadsen Aug 6, 2016

AndreasMadsen commented Aug 6, 2016

adrian-nitu-92 commented Aug 8, 2016

AndreasMadsen Aug 8, 2016

AndreasMadsen commented Aug 8, 2016

jasnell commented Aug 8, 2016

adrian-nitu-92 Aug 9, 2016 •

edited

Loading

AndreasMadsen Aug 9, 2016

AndreasMadsen commented Aug 11, 2016 •

edited

Loading

AndreasMadsen Aug 11, 2016

AndreasMadsen commented Aug 11, 2016

adrian-nitu-92 commented Aug 12, 2016

AndreasMadsen commented Aug 12, 2016

benchmark: add --format csv option #7961

benchmark: add --format csv option #7961

Conversation

adrian-nitu-92 commented Aug 3, 2016

Checklist

Affected core subsystem(s)

Description of change

AndreasMadsen commented Aug 3, 2016

jasnell commented Aug 3, 2016

adrian-nitu-92 commented Aug 4, 2016

Choose a reason for hiding this comment

adrian-nitu-92 Aug 4, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AndreasMadsen commented Aug 4, 2016 • edited Loading

adrian-nitu-92 commented Aug 4, 2016 • edited Loading

Choose a reason for hiding this comment

adrian-nitu-92 commented Aug 5, 2016

AndreasMadsen commented Aug 5, 2016

AndreasMadsen commented Aug 6, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AndreasMadsen commented Aug 6, 2016

adrian-nitu-92 commented Aug 8, 2016

Choose a reason for hiding this comment

AndreasMadsen commented Aug 8, 2016

jasnell commented Aug 8, 2016

adrian-nitu-92 Aug 9, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AndreasMadsen commented Aug 11, 2016 • edited Loading

Choose a reason for hiding this comment

AndreasMadsen commented Aug 11, 2016

adrian-nitu-92 commented Aug 12, 2016

AndreasMadsen commented Aug 12, 2016

adrian-nitu-92 Aug 4, 2016 •

edited

Loading

AndreasMadsen commented Aug 4, 2016 •

edited

Loading

adrian-nitu-92 commented Aug 4, 2016 •

edited

Loading

adrian-nitu-92 Aug 9, 2016 •

edited

Loading

AndreasMadsen commented Aug 11, 2016 •

edited

Loading