Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: fix test-cluster-worker-init.js flakyness #8703

Closed
wants to merge 1 commit into from

Conversation

imyller
Copy link
Member

@imyller imyller commented Sep 21, 2016

Checklist
  • make -j4 test (UNIX), or vcbuild test nosign (Windows) passes
  • tests and/or benchmarks are included
  • commit message follows commit guidelines
Affected core subsystem(s)

test, cluster

Description of change

Update test to match current test guidelines and use common.mustCall
instead of unref'd timer.

Fixes: #8700

@nodejs-github-bot nodejs-github-bot added the test Issues and PRs related to the tests. label Sep 21, 2016
@imyller imyller added the cluster Issues and PRs related to the cluster subsystem. label Sep 21, 2016
@imyller
Copy link
Member Author

imyller commented Sep 21, 2016

/cc @nodejs/testing

@imyller
Copy link
Member Author

imyller commented Sep 21, 2016

CI stress test with pi1:
100 runs: https://ci.nodejs.org/job/node-stress-single-test-pi1-fanned/17/ (passed)
999 runs: https://ci.nodejs.org/job/node-stress-single-test-pi1-fanned/19/ (passed, but aborted after 328 runs for unrelated reasons)

Copy link
Contributor

@cjihrig cjihrig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm very +1 for removing timers from tests.

timer.unref();

worker.on('message', function(message) {
worker.on('message', common.mustCall((message) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the only common.mustCall() you need.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer to keep the other common.mustCall()'s because they are actually mandatory for passing the test too. At least it gives more granularity to error msg if test fails if we have them.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, earlier we had no clear picture what phase actually failed. I wanted to improve that situation.

Copy link
Contributor

@cjihrig cjihrig Sep 21, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand. But I also think that introduces a slippery slope where literally every callback in every test is inside of a common.mustCall() (since it shouldn't be in the test if it isn't necessary).

EDIT: With the exception of common.fail cases.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I'll modify and I understand your concern.

My personal opinion still remains that in cases where verifying sequence of events is important, common.mustCall can be present in the middle-steps too. I categorized this test to be one of those cases.

timer.unref();

worker.on('message', function(message) {
worker.on('message', common.mustCall((message) => {
assert(message, 'did not receive expected message');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please change this to assert.strictEqual() to prevent surprises.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure thing. It will be in the format

assert.strictEqual(message, true, 'did not receive expected message');

because worker sends true as message value.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. I just wanted to rule out any other truthy value somehow showing up. Thanks for making the changes.

@imyller
Copy link
Member Author

imyller commented Sep 21, 2016

Update test to match current test guidelines and use common.mustCall
instead of unref'd timer.

Fixes: nodejs#8700
@imyller
Copy link
Member Author

imyller commented Sep 21, 2016

@cjihrig Please re-review at your convenience. The changes you requested are in.

Copy link
Contributor

@cjihrig cjihrig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change to arrow functions seems pointless, but I won't fight it. LGTM

@imyller
Copy link
Member Author

imyller commented Sep 21, 2016

New CI after requested modifications: https://ci.nodejs.org/job/node-test-pull-request/4202/

@Trott
Copy link
Member

Trott commented Sep 22, 2016

LGTM

@imyller
Copy link
Member Author

imyller commented Sep 22, 2016

@rvagg had to boot CI Pi's (for unrelated reasons) so stress test aborted after 328 runs. Total we have 428 successfull stress test runs.

Do you feel that we need to complete full 999 cycle in the stress test CI for this one?

@Trott
Copy link
Member

Trott commented Sep 22, 2016

Do you feel that we need to complete full 999 cycle in the stress test CI for this one?

Not in this case. I think it's fine.

@Trott
Copy link
Member

Trott commented Sep 22, 2016

Stress test against master: https://ci.nodejs.org/job/node-stress-single-test-pi1-binary/15/label=pi1-raspbian-wheezy/console

Failed 10 out of 100 runs with current master. So 328 successful runs with 0 failures using the code change in this PR seems pretty convincing.

Copy link
Member

@santigimeno santigimeno left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@imyller
Copy link
Member Author

imyller commented Sep 22, 2016

@Trott Just asking: do we apply 48 hour window to this PR or does arm-fanned CI flakyness require more urgent landing? LGTMs are there and CI is green.

@Trott
Copy link
Member

Trott commented Sep 22, 2016

I'm inclined to wait. There's still plenty of other armv6 failures to sort out, so waiting another 24 hours on this one isn't going to hurt anything. However, if anyone on @nodejs/testing feels differently, I will defer to them on it.

@imyller imyller self-assigned this Sep 23, 2016
@imyller
Copy link
Member Author

imyller commented Sep 23, 2016

Landing:

  • Three LGTMs
  • No objections
  • Requested changes have been made
  • CI test passed (only the known CI failures)

@imyller
Copy link
Member Author

imyller commented Sep 23, 2016

landed in 66369d0

@imyller imyller closed this Sep 23, 2016
@imyller imyller removed their assignment Sep 23, 2016
imyller added a commit that referenced this pull request Sep 23, 2016
Update test to match current test guidelines and use common.mustCall
instead of unref'd timer.

PR-URL: #8703
Fixes: #8700
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
jasnell pushed a commit that referenced this pull request Sep 29, 2016
Update test to match current test guidelines and use common.mustCall
instead of unref'd timer.

PR-URL: #8703
Fixes: #8700
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Fishrock123 pushed a commit that referenced this pull request Oct 11, 2016
Update test to match current test guidelines and use common.mustCall
instead of unref'd timer.

PR-URL: #8703
Fixes: #8700
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
MylesBorins pushed a commit that referenced this pull request Nov 18, 2016
Update test to match current test guidelines and use common.mustCall
instead of unref'd timer.

PR-URL: #8703
Fixes: #8700
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cluster Issues and PRs related to the cluster subsystem. test Issues and PRs related to the tests.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Investigate flaky test-cluster-worker-init on Raspberry Pi 1
6 participants