-
Notifications
You must be signed in to change notification settings - Fork 29.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test: add common.mustSucceed #35086
test: add common.mustSucceed #35086
Conversation
My immediate reaction is that I prefer the explicitness and clarity of the current pattern. But that's my reaction without looking much and I have to admit you make some good points in your explanation. I'll take a closer look later, but for now: /ping @nodejs/testing |
Having spent a little more time with this, I think I'm a convert. This seems like a good idea to me. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The new function needs to be documented in test/common/README.md
. Also, might be worthwhile looking for common.mustCall()
instances in our internal docs that need to be changed. I'm thinking specifically of doc/guides/writing-tests.md
but there may be others.
I agree, and I think this does not help much with cases where It's test writer responsibility to make sure errors are not ignored with or without this change. |
Personally, I think that the We have something remotely similar for
From my perspective, simply having
I actually started this because I noticed a lot of tests using |
Kindly asking @jasnell, @BridgeAR, @cjihrig, @addaleax for input due to their experience with these particular tests.
|
My initial thought would be that this only really makes sense if it is enforced through a linter rule, because I would expect that this pattern does look surprising to people who are new to our test suite (the same obviously goes for most patterns that are specific to it). |
I think we could use a linter rule to prevent |
@tniessen Yeah, any |
@addaleax I added the rule and fixed the resulting linter problems. However, I restricted it to only apply if the function body begins with something(common.mustCall((err) => {
if (err && err.code === 'ERR_SOMETHING') {
// Do something
return;
}
assert.ifError(err);
}); |
This has been open for more than three weeks and has neither been approved nor rejected nor otherwise reviewed. Ping @nodejs/testing (hopefully that's a good choice?). |
I go back and forth between thinking it's a good idea and might help catch some bugs in our tests...then thinking it may not be worth the churn/additional API...and then thinking that it doesn't matter that much either way because we can do whatever we want with the tests and then change it if we don't like it after a month or whatever. Let's give this another 3 days or so and if no one else either approves or objects, I'll approve it and you can rebase and land. I want to give people with stronger opinions (and who are putting more thought into it) a chance to weigh in. But I think I like this change. |
Replaced the following pattern with "mustSucceed()": mustCall\([ \n]*(\(erro?r?\)[ \n]*=>[ \n]*\{?|function[ \n]*\(erro?r?\)[ \n]*\{)?[ \n]*assert\.ifError(\(erro?r?\))?;?[ \n]*\}?[ \n]*\)
Replaced the following pattern with "mustSucceed(($2)$3=>$4{\n": mustCall\([\n ]*\(erro?r?(,[\n ]*([^\)]*))?\)([\n ]*)=>([\n ]*)\{[\n ]+assert\.ifError\(erro?r?\);\n
Replaced the following pattern with "mustSucceed(function($2)$3{\n": mustCall\([\n ]*function[ \n]*\(erro?r?(, *([^\)]*))?\)([ \t\n]*)\{[\n ]+assert\.ifError\(erro?r?\);\n
Convert statements of the form assert(!err) to mustSucceed.
@Trott It's been another two weeks, I will rebase this, and you (or anyone else) can either approve or close this. |
df0224e
to
d5c3a36
Compare
d5c3a36
to
e43195b
Compare
Ping @Trott. Sorry for pinging you again, I just don't want this to be forgotten for another few weeks and require more rebases. Feel free to ignore or close if you don't think approving it is the right thing to do. |
This is awesome @tniessen! Love it that you provided the eslint rules and the PR even catchs a bunch of scenarios of I'm a big proponent of less abstractions in tests but all things weighted this is def one worth having 👍 |
Commit Queue failed- Loading data for nodejs/node/pull/35086 ✔ Done loading data for nodejs/node/pull/35086 ----------------------------------- PR info ------------------------------------ Title test: add common.mustSucceed (#35086) ⚠ Could not retrieve the email or name of the PR author's from user's GitHub profile! Branch tniessen:add-mustsucceed -> nodejs:master Labels author ready, test Commits 9 - test: add common.mustSucceed - test: convert to mustSucceed - test: convert to mustSucceed - test: convert to mustSucceed - test: convert to mustSucceed - test: apply manual formatting to previous commits - test: use mustSucceed instead of mustCall - doc,test: document common.mustSucceed - tools,test: add linter rule for mustSucceed Committers 1 - Tobias Nießen PR-URL: https://github.com/nodejs/node/pull/35086 Reviewed-By: Ruy Adorno ------------------------------ Generated metadata ------------------------------ PR-URL: https://github.com/nodejs/node/pull/35086 Reviewed-By: Ruy Adorno -------------------------------------------------------------------------------- ✔ Last GitHub Actions successful ℹ Last Full PR CI on 2020-10-14T18:53:56Z: https://ci.nodejs.org/job/node-test-pull-request/33623/ - Querying data for job/node-test-pull-request/33623/ ✔ Build data downloaded ✔ Last Jenkins CI successful ℹ This PR was created on Mon, 07 Sep 2020 01:32:28 GMT ✔ Approvals: 1 ✔ - Ruy Adorno (@ruyadorno): https://github.com/nodejs/node/pull/35086#pullrequestreview-510585252 -------------------------------------------------------------------------------- ✔ No git cherry-pick in progress ✔ No git am in progress ✔ No git rebase in progress -------------------------------------------------------------------------------- - Bringing origin/master up to date... From https://github.com/nodejs/node * branch master -> FETCH_HEAD ✔ origin/master is now up-to-date - Downloading patch for 35086 From https://github.com/nodejs/node * branch refs/pull/35086/merge -> FETCH_HEAD ✔ Fetched commits as 7f25fe8b67ee..e43195b5cbd1 -------------------------------------------------------------------------------- Auto-merging test/common/index.js [master c9f901e678] test: add common.mustSucceed Author: Tobias Nießen Date: Sun Sep 6 22:27:07 2020 +0200 1 file changed, 9 insertions(+) [master 49e28d63f3] test: convert to mustSucceed Author: Tobias Nießen Date: Sun Sep 6 23:39:24 2020 +0200 15 files changed, 23 insertions(+), 60 deletions(-) Auto-merging test/parallel/test-fs-rmdir-recursive.js Auto-merging test/parallel/test-dns-resolveany.js [master 8c7dfc7bb5] test: convert to mustSucceed Author: Tobias Nießen Date: Mon Sep 7 00:27:11 2020 +0200 77 files changed, 193 insertions(+), 387 deletions(-) [master 8148783856] test: convert to mustSucceed Author: Tobias Nießen Date: Mon Sep 7 01:13:37 2020 +0200 32 files changed, 64 insertions(+), 129 deletions(-) [master cf6c14ea88] test: convert to mustSucceed Author: Tobias Nießen Date: Mon Sep 7 01:22:49 2020 +0200 8 files changed, 28 insertions(+), 54 deletions(-) Auto-merging test/parallel/test-fs-rmdir-recursive.js [master 6bd8dd3ce2] test: apply manual formatting to previous commits Author: Tobias Nießen Date: Mon Sep 7 02:24:09 2020 +0200 43 files changed, 58 insertions(+), 116 deletions(-) [master 1058c28ee6] test: use mustSucceed instead of mustCall Author: Tobias Nießen Date: Mon Sep 7 02:07:13 2020 +0200 13 files changed, 40 insertions(+), 41 deletions(-) [master 826bc32c7b] doc,test: document common.mustSucceed Author: Tobias Nießen Date: Mon Sep 7 11:56:00 2020 +0200 2 files changed, 14 insertions(+) [master b4ad75875a] tools,test: add linter rule for mustSucceed Author: Tobias Nießen Date: Tue Sep 8 18:16:02 2020 +0200 20 files changed, 161 insertions(+), 97 deletions(-) create mode 100644 test/parallel/test-eslint-prefer-common-mustsucceed.js create mode 100644 tools/eslint-rules/prefer-common-mustsucceed.js ✔ Patches applied There are 9 commits in the PR. Attempting autorebase. Rebasing (2/18) Commit Queue action: https://github.com/nodejs/node/actions/runs/311127512 |
PR-URL: #35086 Reviewed-By: Ruy Adorno <ruyadorno@github.com> Reviewed-By: Rich Trott <rtrott@gmail.com>
Landed in 30fb4a0 |
Thank you for reviewing @ruyadorno and @Trott, and thank you for landing @aduh95. |
PR-URL: #35086 Reviewed-By: Ruy Adorno <ruyadorno@github.com> Reviewed-By: Rich Trott <rtrott@gmail.com>
Our increasingly large collection of tests makes frequent use of this pattern:
This PR changes those to this pattern:
This simple change allows us to remove more than 300 lines of code from tests. Another aspect is that many tests really should be using
mustSucceed()
instead ofmustCall()
, becausemustCall()
silently ignores errors passed to it. I changed the crypto tests manually where I found such cases, and a few other test files, but there's probably dozens or hundreds of such cases in other test files.To make this easier to review, I separated the changes into multiple commits.
mustCall
tomustSucceed
.mustCall\([ \n]*(\(erro?r?\)[ \n]*=>[ \n]*\{?|function[ \n]*\(erro?r?\)[ \n]*\{)?[ \n]*assert\.ifError(\(erro?r?\))?;?[ \n]*\}?[ \n]*\)
withmustSucceed()
. These are trivial cases, e.g,mustCall(assert.ifError)
,mustCall((err) => assert.ifError(err))
,mustCall(function(err) { assert.ifError(err); })
etc.mustCall\([\n ]*\(erro?r?(,[\n ]*([^\)]*))?\)([\n ]*)=>([\n ]*)\{[\n ]+assert\.ifError\(erro?r?\);\n
withmustSucceed(($2)$3=>$4{\n
. These are arrow functions that take an error argument and begin withassert.ifError
.mustCall\([\n ]*function[ \n]*\(erro?r?(, *([^\)]*))?\)([ \t\n]*)\{[\n ]+assert\.ifError\(erro?r?\);\n
withmustSucceed(function($2)$3{\n
. These are regular functions that take an error argument and begin withassert.ifError
.assert.ok(!err)
tomustSucceed
, where possible.mustCall
withmustSucceed
where it seems to make sense.Checklist
make -j4 test
(UNIX), orvcbuild test
(Windows) passes