test: added `common.mustCallAtLeast` #12935

refack · 2017-05-09T21:24:40Z

added common.mustCallAtLeast for calling more than minimum times.

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

Affected core subsystem(s)

test

addaleax · 2017-05-09T21:27:37Z

test/common/index.js

+/**
+ * @function mustCall
+ * @param fn "The code that must be called
+ * @param {?number|string} expected "Expected times to be called, or '+' for more then 0. Default = 1"


typo: then → than

refack · 2017-05-09T21:28:08Z

CI: https://ci.nodejs.org/job/node-test-commit/9765/
/cc @nodejs/testing

cjihrig · 2017-05-09T21:28:06Z

test/common/index.js

+ * @returns {Function}
+ * @export
+ */
+exports.mustCall = function mustCall(fn, expected) {


Can you undo this block of changes.

the JSDocs?
It's usefull for people with IDEs that can parse these (VSCode / Webstorm)

Other than lib/punycode.js which is vendored, there are only 9 @param matches in all of lib, src, and test. It's more consistent to drop them.

cjihrig · 2017-05-09T21:28:28Z

test/parallel/test-test-common.js

@@ -0,0 +1,16 @@
+'use strict';


This code should live in test/parallel/test-common.js I think.

didn't see that one...

cjihrig · 2017-05-09T21:29:05Z

I think this needs a documentation update.

refack · 2017-05-09T21:43:32Z

Docs added, nit's addressed. @cjihrig I'd rather keep the JSDocs, unless you have a strong objection.

cjihrig · 2017-05-09T21:45:07Z

I wouldn't say strong objection, but definitely -1.

refack · 2017-05-09T21:47:22Z

I wouldn't say strong objection, but definitely -1.

Gone.
I'll do a PR with JSDocs for the whole file once 😉

Fishrock123 · 2017-05-09T21:51:59Z

Could you point to somewhere that this would be better suited for rather than the explicit number of calls?

Fishrock123 · 2017-05-09T21:52:37Z

I generally think this is a bad idea because if something is being called more times than you expect you've probably got a bug.

refack · 2017-05-09T21:52:45Z

Could you point to somewhere that this would be better suited for rather than the explicit number of calls?

#12930 (comment)

refack · 2017-05-09T21:55:49Z

I generally think this is a bad idea because if something is being called more times than you expect you've probably got a bug.

I tend to agree, but some things are either OS dependent or non deterministic. In those cases we tend to skip the mustCall all together which IMHO is worse.
I agree this should be used sparingly, and usage should receive justification in a comment.

jasnell

I've been wanting this for a while. Once @cjihrig is happy with it, I'm LGTM

mscdex · 2017-05-09T22:32:06Z

-1 I don't think it is a good idea to reuse the same parameter (or even the same function) for this.

jasnell · 2017-05-09T22:33:14Z

@Fishrock123 ... this came up with regards to adding mustCall() around a data event handler, which could be reasonably called any number of times.

@mscdex ... the other option I had in mind was a common.mustCallAtLeast(n) method that would allow a minimum number to be specified.

mscdex · 2017-05-09T22:39:09Z

@jasnell I would much prefer a separate method so that the intention is clear.

refack · 2017-05-09T22:57:44Z

@jasnell I would much prefer a separate method so that the intention is clear.

@mscdex I was thinking about that.
Will do.

Trott

-0 from me. I dislike magic values. I prefer that things in common have simple, self-explanatory, and intuitively obvious interfaces. This adds more cognitive overhead, especially for newcomers.

Trott · 2017-05-09T23:08:46Z

test/common/README.md

-fail.
+Returns a function that calls `fn`. If the returned function has not been
+called exactly `expected` number of times, or at least once if
+`expected === '+'`, when the test is complete, then the test will fail.


Nit: This documentation change results in a confusing run-on sentence:

If the returned function has not been called exactly expected number of times, or at least once if
expected === '+', when the test is complete, then the test will fail.

Would be better as something like:

If the returned function has not been called exactly expected number of times when the test is complete, then the test will fail. If expected is '+'`, then the test must run at least once.

gibfahn · 2017-05-09T23:13:31Z

the other option I had in mind was a common.mustCallAtLeast(n)

+1 to that

-0 from me. I dislike magic values

@Trott thoughts on @jasnell's proposed common.mustCallAtLeast(2)?

Trott · 2017-05-09T23:19:07Z

@Trott thoughts on @jasnell's proposed common.mustCallAtLeast(2)?

Eh, still pretty +/-0 on it. I think there's enough API surface area in common and we quickly reach a point of diminishing returns.

Moreover, I'm not really sure this is solving a very common problem. I imagine it's mostly for data callbacks in tests that are called once but could theoretically be called more than once. However, those almost never need to actually be wrapped to make sure they're called because they usually have something like data += chunk and then the value of data is checked somewhere, so you don't really need to check that the callback was called. The test will fail if it wasn't. (Will there be a small number of empty callbacks for data that we want to check? Sure. Is it worth adding a whole other function to the common monolith for it? I doubt it.)

But if others feel this has big value, I won't stop it.

refack · 2017-05-09T23:28:32Z

Replace magic '+' with mustCallAtLeast PTAL

refack · 2017-05-09T23:38:33Z

P.S. why don't we adopt something like sinon.
I'm a big believer that better tooling begets better tests.

Trott · 2017-05-10T04:16:03Z

P.S. why don't we adopt something like sinon.

Biggest reason is probably because no one's bothered to do it yet.

That said, a PR adding sinon very well might not get accepted. Speaking only for myself, I have grown wary of things that raise the barrier to entry for people working on tests. So to the extent that we avoid extra tools and unneeded abstractions in the common module, I'm happy. But that said, if the benefit would be considerable and obvious, I'd be for it.

Others in @nodejs/testing might have other opinions.

refack · 2017-05-19T17:40:04Z

Pre land CI: https://ci.nodejs.org/job/node-test-commit/10013/

refack · 2017-05-19T19:24:35Z

landed in fccc0bf

MylesBorins · 2017-07-17T14:51:25Z

Should this land on v6.x?

refack · 2017-07-17T15:49:34Z

Should this land on v6.x?

Sure. It's a semver-minor change in the test harness.

refack · 2017-07-17T18:09:40Z

#14327 depends on this

PR-URL: #12935 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

nodejs-github-bot added the test Issues and PRs related to the tests. label May 9, 2017

refack self-assigned this May 9, 2017

refack requested review from jasnell and addaleax May 9, 2017 21:25

addaleax approved these changes May 9, 2017

View reviewed changes

cjihrig requested changes May 9, 2017

View reviewed changes

refack mentioned this pull request May 9, 2017

test: improve numerous http tests #12930

Closed

3 tasks

jasnell approved these changes May 9, 2017

View reviewed changes

Trott reviewed May 9, 2017

View reviewed changes

refack force-pushed the mustCall+ branch from 95517cc to a6bb177 Compare May 9, 2017 23:30

refack changed the title ~~test: allow mustCall '+' as expected~~ test: added common.mustCallAtLeast May 9, 2017

refack mentioned this pull request May 10, 2017

create public nodejs/collaborators repo nodejs/TSC#243

Closed

refack force-pushed the mustCall+ branch from 141dc1f to a5b2396 Compare May 19, 2017 17:39

refack closed this May 19, 2017

refack force-pushed the mustCall+ branch from a5b2396 to fccc0bf Compare May 19, 2017 19:24

refack deleted the mustCall+ branch May 19, 2017 19:24

refack merged commit fccc0bf into nodejs:master May 19, 2017

jasnell mentioned this pull request May 28, 2017

8.0.0 Release Proposal #12220

Closed

refack mentioned this pull request Jun 3, 2017

test: add hijackStdout and hijackStderr #13439

Closed

4 tasks

refack removed their assignment Jun 12, 2017

gibfahn mentioned this pull request Jun 15, 2017

Auditing for 6.11.1 nodejs/Release#230

Closed

3 tasks

MylesBorins added the lts-watch-v6.x label Jul 17, 2017

MylesBorins added the semver-minor PRs that contain new features and should be released in the next minor version. label Jul 17, 2017

refack mentioned this pull request Jul 17, 2017

[v6.x backport] test: harden test-dgram-bind-shared-ports #14327

Closed

3 tasks

MylesBorins added land-on-v6.x and removed lts-watch-v6.x semver-minor PRs that contain new features and should be released in the next minor version. labels Aug 14, 2017

MylesBorins pushed a commit that referenced this pull request Aug 14, 2017

test: add mustCallAtLeast

ce86b8e

PR-URL: #12935 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

MylesBorins pushed a commit that referenced this pull request Aug 16, 2017

test: add mustCallAtLeast

f578c9b

PR-URL: #12935 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

MylesBorins mentioned this pull request Aug 16, 2017

v6.11.3-proposal #14852

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: added `common.mustCallAtLeast` #12935

test: added `common.mustCallAtLeast` #12935

refack commented May 9, 2017 •

edited

Loading

addaleax May 9, 2017

refack commented May 9, 2017

cjihrig May 9, 2017

refack May 9, 2017

cjihrig May 9, 2017

cjihrig May 9, 2017

refack May 9, 2017 •

edited

Loading

cjihrig commented May 9, 2017

refack commented May 9, 2017

cjihrig commented May 9, 2017

refack commented May 9, 2017

Fishrock123 commented May 9, 2017

Fishrock123 commented May 9, 2017

refack commented May 9, 2017

refack commented May 9, 2017

jasnell left a comment

mscdex commented May 9, 2017 •

edited

Loading

jasnell commented May 9, 2017

mscdex commented May 9, 2017

refack commented May 9, 2017

Trott left a comment

Trott May 9, 2017

gibfahn commented May 9, 2017

Trott commented May 9, 2017

refack commented May 9, 2017

refack commented May 9, 2017

Trott commented May 10, 2017

refack commented May 19, 2017

refack commented May 19, 2017

MylesBorins commented Jul 17, 2017

refack commented Jul 17, 2017

refack commented Jul 17, 2017

test: added common.mustCallAtLeast #12935

test: added common.mustCallAtLeast #12935

Conversation

refack commented May 9, 2017 • edited Loading

Checklist

Affected core subsystem(s)

addaleax May 9, 2017

Choose a reason for hiding this comment

refack commented May 9, 2017

cjihrig May 9, 2017

Choose a reason for hiding this comment

refack May 9, 2017

Choose a reason for hiding this comment

cjihrig May 9, 2017

Choose a reason for hiding this comment

cjihrig May 9, 2017

Choose a reason for hiding this comment

refack May 9, 2017 • edited Loading

Choose a reason for hiding this comment

cjihrig commented May 9, 2017

refack commented May 9, 2017

cjihrig commented May 9, 2017

refack commented May 9, 2017

Fishrock123 commented May 9, 2017

Fishrock123 commented May 9, 2017

refack commented May 9, 2017

refack commented May 9, 2017

jasnell left a comment

Choose a reason for hiding this comment

mscdex commented May 9, 2017 • edited Loading

jasnell commented May 9, 2017

mscdex commented May 9, 2017

refack commented May 9, 2017

Trott left a comment

Choose a reason for hiding this comment

Trott May 9, 2017

Choose a reason for hiding this comment

gibfahn commented May 9, 2017

Trott commented May 9, 2017

refack commented May 9, 2017

refack commented May 9, 2017

Trott commented May 10, 2017

refack commented May 19, 2017

refack commented May 19, 2017

MylesBorins commented Jul 17, 2017

refack commented Jul 17, 2017

refack commented Jul 17, 2017

test: added `common.mustCallAtLeast` #12935

test: added `common.mustCallAtLeast` #12935

refack commented May 9, 2017 •

edited

Loading

refack May 9, 2017 •

edited

Loading

mscdex commented May 9, 2017 •

edited

Loading