Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: make stdout buffer test more robust #6633

Closed
wants to merge 2 commits into from
Closed

Conversation

Trott
Copy link
Member

@Trott Trott commented May 7, 2016

Checklist
  • tests and code linting passes
  • the commit message follows commit guidelines
Affected core subsystem(s)

test

Description of change

test-stdout-buffer-flush-on-exit is unfortunately non-deterministic. It
will, every so often, pass when it is supposed to fail. This is
currently guarded against by running the test with three different long
strings. This change increases it to five to reduce the false negatives.

test-stdout-buffer-flush-on-exit is unfortunately non-deterministic. It
will, every so often, pass when it is supposed to fail. This is
currently guarded against by running the test with three different long
strings. This change increases it to five to reduce the false negatives.
@Trott Trott added test Issues and PRs related to the tests. known issue test labels May 7, 2016
@mscdex mscdex added the process Issues and PRs related to the process subsystem. label May 7, 2016
@@ -15,7 +15,7 @@ if (process.argv[2] === 'child') {
process.exit();
}

[16, 18, 20].forEach((exponent) => {
[16, 18, 20, 21, 22].forEach((exponent) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should it be adding 22 and 24 to stay consistent here? or are 21 and 22 magic somehow?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

22 is the biggest integer you can use before you start getting RangeError failures for Invalid string length.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, figured there was a reason

@Trott
Copy link
Member Author

Trott commented May 7, 2016

Stress test of current version:

Stress test with the version in this PR:

Weirdly and annoyingly, it's not any more reliable.

Closing for now in the hopes of figuring it out and re-opening...

@Trott Trott closed this May 7, 2016
@Trott Trott reopened this May 8, 2016
@Trott
Copy link
Member Author

Trott commented May 8, 2016

Reopened. Increasing the number of tests to 14 seems to pump up the reliability quite a bit.

Current master stress test: https://ci.nodejs.org/job/node-stress-single-test/703/nodes=centos5-64/console

This PR: https://ci.nodejs.org/job/node-stress-single-test/704/nodes=centos5-64/console

@Trott
Copy link
Member Author

Trott commented May 8, 2016

Current version fails about 1/3 of the time. Version in this PR didn't fail at all after 9999 runs.

@Trott
Copy link
Member Author

Trott commented May 9, 2016

CI: https://ci.nodejs.org/job/node-test-pull-request/2541/

@bnoordhuis
Copy link
Member

LGTM

1 similar comment
@jasnell
Copy link
Member

jasnell commented May 9, 2016

LGTM

@santigimeno
Copy link
Member

LGTM

@MylesBorins
Copy link
Contributor

landed in 0912b88

MylesBorins pushed a commit that referenced this pull request May 10, 2016
test-stdout-buffer-flush-on-exit is unfortunately non-deterministic. It
will, every so often, pass when it is supposed to fail. This is
currently guarded against by running the test with three different long
strings. This change increases it to five to reduce the false negatives.

PR-URL: #6633
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
evanlucas pushed a commit that referenced this pull request May 17, 2016
test-stdout-buffer-flush-on-exit is unfortunately non-deterministic. It
will, every so often, pass when it is supposed to fail. This is
currently guarded against by running the test with three different long
strings. This change increases it to five to reduce the false negatives.

PR-URL: #6633
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
@MylesBorins
Copy link
Contributor

@Trott lts?

@Trott
Copy link
Member Author

Trott commented Jun 2, 2016

@thealphanerd Yes.

One thing about this and the other known issues PRs is that they will have to be landed in order. I think numerical order (by PR ID) ascending should be good.

MylesBorins pushed a commit that referenced this pull request Jul 11, 2016
test-stdout-buffer-flush-on-exit is unfortunately non-deterministic. It
will, every so often, pass when it is supposed to fail. This is
currently guarded against by running the test with three different long
strings. This change increases it to five to reduce the false negatives.

PR-URL: #6633
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
MylesBorins pushed a commit that referenced this pull request Jul 11, 2016
test-stdout-buffer-flush-on-exit is unfortunately non-deterministic. It
will, every so often, pass when it is supposed to fail. This is
currently guarded against by running the test with three different long
strings. This change increases it to five to reduce the false negatives.

PR-URL: #6633
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
MylesBorins pushed a commit that referenced this pull request Jul 12, 2016
test-stdout-buffer-flush-on-exit is unfortunately non-deterministic. It
will, every so often, pass when it is supposed to fail. This is
currently guarded against by running the test with three different long
strings. This change increases it to five to reduce the false negatives.

PR-URL: #6633
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
MylesBorins pushed a commit that referenced this pull request Jul 12, 2016
test-stdout-buffer-flush-on-exit is unfortunately non-deterministic. It
will, every so often, pass when it is supposed to fail. This is
currently guarded against by running the test with three different long
strings. This change increases it to five to reduce the false negatives.

PR-URL: #6633
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
@MylesBorins MylesBorins mentioned this pull request Jul 12, 2016
MylesBorins pushed a commit that referenced this pull request Jul 14, 2016
test-stdout-buffer-flush-on-exit is unfortunately non-deterministic. It
will, every so often, pass when it is supposed to fail. This is
currently guarded against by running the test with three different long
strings. This change increases it to five to reduce the false negatives.

PR-URL: #6633
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
MylesBorins pushed a commit that referenced this pull request Jul 14, 2016
test-stdout-buffer-flush-on-exit is unfortunately non-deterministic. It
will, every so often, pass when it is supposed to fail. This is
currently guarded against by running the test with three different long
strings. This change increases it to five to reduce the false negatives.

PR-URL: #6633
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
@Trott Trott deleted the know branch January 13, 2022 22:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
process Issues and PRs related to the process subsystem. test Issues and PRs related to the tests.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants