-
Notifications
You must be signed in to change notification settings - Fork 29.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test: make stdout buffer test more robust #6633
Conversation
test-stdout-buffer-flush-on-exit is unfortunately non-deterministic. It will, every so often, pass when it is supposed to fail. This is currently guarded against by running the test with three different long strings. This change increases it to five to reduce the false negatives.
@@ -15,7 +15,7 @@ if (process.argv[2] === 'child') { | |||
process.exit(); | |||
} | |||
|
|||
[16, 18, 20].forEach((exponent) => { | |||
[16, 18, 20, 21, 22].forEach((exponent) => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should it be adding 22 and 24 to stay consistent here? or are 21 and 22 magic somehow?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
22 is the biggest integer you can use before you start getting RangeError
failures for Invalid string length
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cool, figured there was a reason
Stress test of current version: Stress test with the version in this PR: Weirdly and annoyingly, it's not any more reliable. Closing for now in the hopes of figuring it out and re-opening... |
Reopened. Increasing the number of tests to 14 seems to pump up the reliability quite a bit. Current master stress test: https://ci.nodejs.org/job/node-stress-single-test/703/nodes=centos5-64/console This PR: https://ci.nodejs.org/job/node-stress-single-test/704/nodes=centos5-64/console |
Current version fails about 1/3 of the time. Version in this PR didn't fail at all after 9999 runs. |
LGTM |
1 similar comment
LGTM |
LGTM |
landed in 0912b88 |
test-stdout-buffer-flush-on-exit is unfortunately non-deterministic. It will, every so often, pass when it is supposed to fail. This is currently guarded against by running the test with three different long strings. This change increases it to five to reduce the false negatives. PR-URL: #6633 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
test-stdout-buffer-flush-on-exit is unfortunately non-deterministic. It will, every so often, pass when it is supposed to fail. This is currently guarded against by running the test with three different long strings. This change increases it to five to reduce the false negatives. PR-URL: #6633 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
@Trott lts? |
@thealphanerd Yes. One thing about this and the other known issues PRs is that they will have to be landed in order. I think numerical order (by PR ID) ascending should be good. |
test-stdout-buffer-flush-on-exit is unfortunately non-deterministic. It will, every so often, pass when it is supposed to fail. This is currently guarded against by running the test with three different long strings. This change increases it to five to reduce the false negatives. PR-URL: #6633 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
test-stdout-buffer-flush-on-exit is unfortunately non-deterministic. It will, every so often, pass when it is supposed to fail. This is currently guarded against by running the test with three different long strings. This change increases it to five to reduce the false negatives. PR-URL: #6633 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
test-stdout-buffer-flush-on-exit is unfortunately non-deterministic. It will, every so often, pass when it is supposed to fail. This is currently guarded against by running the test with three different long strings. This change increases it to five to reduce the false negatives. PR-URL: #6633 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
test-stdout-buffer-flush-on-exit is unfortunately non-deterministic. It will, every so often, pass when it is supposed to fail. This is currently guarded against by running the test with three different long strings. This change increases it to five to reduce the false negatives. PR-URL: #6633 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
test-stdout-buffer-flush-on-exit is unfortunately non-deterministic. It will, every so often, pass when it is supposed to fail. This is currently guarded against by running the test with three different long strings. This change increases it to five to reduce the false negatives. PR-URL: #6633 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
test-stdout-buffer-flush-on-exit is unfortunately non-deterministic. It will, every so often, pass when it is supposed to fail. This is currently guarded against by running the test with three different long strings. This change increases it to five to reduce the false negatives. PR-URL: #6633 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Checklist
Affected core subsystem(s)
test
Description of change
test-stdout-buffer-flush-on-exit is unfortunately non-deterministic. It
will, every so often, pass when it is supposed to fail. This is
currently guarded against by running the test with three different long
strings. This change increases it to five to reduce the false negatives.