test: parallelize long-running test #3287

Trott · 2015-10-08T22:20:11Z

I got terribly sad seeing test-stringbytes-external fail on Raspberry Pi in CI again and again and again, so I did the easiest laziest thing I could think of to fix it, which was to split the long-running test into multiple files so it wouldn't time out.

Fixes (hopefull) #2370

evanlucas · 2015-10-08T22:25:50Z

test/parallel/test-stringbytes-external-at-max.js

+  const kStringMaxLength = process.binding('buffer').kStringMaxLength;
+
+  try {
+    new Buffer(kStringMaxLength * 3);


do we need to allocate this much?

I ask mainly because from all of the tests I've run on a Pi 2, this will throw (every time from what I've seen). But it always seemed to get past this part a Pi 1

The reason for the over allocation is to also judge the amount of extra space the heap has available. If we only allocate enough for the test then the toString() operations may make v8 abort because of unavailable memory.

ah that's right

Trott · 2015-10-08T23:03:01Z

CI: https://ci.nodejs.org/job/node-test-commit/774/

It's not red! First time in weeks, I think! I'm so happy, I'm going to put an exclamation point after every sentence!

evanlucas · 2015-10-08T23:04:21Z

Nice!!!!

trevnorris · 2015-10-08T23:04:25Z

test/parallel/test-stringbytes-external-at-max.js

+  const buf = new Buffer(kStringMaxLength);
+
+  var maxString = buf.toString();
+  assert.equal(maxString.length, kStringMaxLength);


Not a good test. The memory needs to be filled or else it may contain random multi-byte characters and the string length might vary. Example:

let b = new Buffer(4).fill('a'); b[0] = 0xc8; b[1] = 0xa2; b.toString().length === 3;

So as much as I hate to do this, fill the above buffer with 0x61 (numbers are currently optimized better than strings). This way you get reliable results.

@trevnorris This would seem to be a bug that exists in the current test and not something that was introduced in the refactoring I did, right? (I'm still happy to fix it, but I want to make sure I actually understand correctly. If I did cause this, I want to know where I goofed.)

This is actually my fault. When these tests were added I specified to skip the zero fill. Discounting the utf-8 case. Though, in reality, the utf-8 case is not necessary. Since the length of a utf-8 string will always be <= a binary string. By filling it we're essentially doing a binary conversion internally since no multi-byte characters will be detected.

So I'd say we drop this one and skip the zero fill.

@trevnorris Fixed in 55a1d94. PTAL.

trevnorris · 2015-10-09T16:50:49Z

Did another review. Thanks for working this out. The test has been a serious thorn.

Trott · 2015-10-09T21:19:59Z

Ref: #3303

Trott · 2015-10-10T05:26:20Z

I think this is good to go for our current CI. PTAL @trevnorris @evanlucas

I want to get the AIX issue in #3303 fixed too, but I don't want that to hold this up for the CI so I'd prefer to get this merged and then I (or someone else) can tackle #3303.

CI: https://ci.nodejs.org/job/node-test-pull-request/473/

Trott · 2015-10-10T05:36:25Z

Looks like the updating of the error message assertion broke the Pi 2 tests:

not ok 100 test-stringbytes-external-at-max.js
#
#assert.js:89
#  throw new assert.AssertionError({
#  ^
#AssertionError: 'Invalid array buffer length' == 'Invalid typed array length'
#    at Object.<anonymous> (/home/iojs/build/workspace/node-test-binary-arm/RUN_SUBSET/0/nodes/pi2-raspbian-wheezy/test/parallel/test-stringbytes-external-at-max.js:13:10)
#    at Module._compile (module.js:425:26)
#    at Object.Module._extensions..js (module.js:432:10)
#    at Module.load (module.js:356:32)
#    at Function.Module._load (module.js:311:12)
#    at Function.Module.runMain (module.js:457:10)
#    at startup (node.js:134:18)
#    at node.js:961:3
  ---
  duration_ms: 1.429

I'm inclined to roll back 3b9e2be.

Trott · 2015-10-10T05:44:43Z

Reverted the error message check, trying again on CI: https://ci.nodejs.org/job/node-test-pull-request/474/

Trott · 2015-10-10T06:20:10Z

CI looks good now. arm-fanned passing! If this is sufficiently better than the current situation to merge, I think test-child-process-emfile.js on 32-bit FreeBSD is the next target on Failing Test Whack-A-Mole.

targos · 2015-10-10T07:41:45Z

We could backport v8/v8@60f8317 to have a better error message.

evanlucas · 2015-10-10T12:51:22Z

LGTM

trevnorris · 2015-10-10T18:13:08Z

LGTM

jasnell · 2015-10-11T00:26:23Z

LGTM... still not sure if this fixes the AIX issue /cc @mhdawson
will get this landed in master tho

Fixes a persistently troublesome failing test by splitting it out into multiple parallel tests. Reviewed By: Evan Lucas <evanlucas@me.com> Reviewed By: Trevor Norris <trev.norris@gmail.com> Reviewed By: James M Snell <jasnell@gmail.com> PR-URL: #3287

jasnell · 2015-10-11T00:37:18Z

Landed in 31c971d

jasnell · 2015-10-11T00:37:51Z

@mhdawson ... as soon as you get the chance, please let me know if this fixes the AIX problem

mscdex · 2015-10-11T17:52:22Z

It looks like at least test-stringbytes-external-exceed-max-by-1.js is timing out occasionally on pi1-raspbian-wheezy.

Trott · 2015-10-11T21:45:30Z

@mscdex test-stringbytes-external-exceed-max.js can take over a minute on pi1-raspbian-wheezy and it only has one call to .toString(). test-stringbytes-external-exceed-max-by-1.js has eight .toString() calls. The solution might be just to split test-stringbytes-external-exceed-max-by-1.js into multiple test files. (Splitting a monolithic test file into multiple test files is what got this from always-failing on pi1-raspbian-wheezy to only-failing-sometimes.)

mscdex · 2015-10-11T21:51:25Z

@Trott Yeah I'm thinking it should be split up more (one for each .toString()?).

Trott · 2015-10-12T03:56:02Z

@mscdex Let's try this: #3323

Fixes a persistently troublesome failing test by splitting it out into multiple parallel tests. Reviewed By: Evan Lucas <evanlucas@me.com> Reviewed By: Trevor Norris <trev.norris@gmail.com> Reviewed By: James M Snell <jasnell@gmail.com> PR-URL: nodejs#3287

Fixes a persistently troublesome failing test by splitting it out into multiple parallel tests. Reviewed By: Evan Lucas <evanlucas@me.com> Reviewed By: Trevor Norris <trev.norris@gmail.com> Reviewed By: James M Snell <jasnell@gmail.com> PR-URL: #3287

jasnell · 2015-10-28T15:11:38Z

Landed in v4.x-staging in c76ef6c

Fixes a persistently troublesome failing test by splitting it out into multiple parallel tests. Reviewed By: Evan Lucas <evanlucas@me.com> Reviewed By: Trevor Norris <trev.norris@gmail.com> Reviewed By: James M Snell <jasnell@gmail.com> PR-URL: #3287

Trott mentioned this pull request Oct 8, 2015

test: parallel/test-stringbytes-external is too slow #2370

Closed

Trott force-pushed the borked-test branch from e1c50ff to 6a8a8b5 Compare October 8, 2015 22:23

evanlucas reviewed Oct 8, 2015
View reviewed changes

mscdex added the test Issues and PRs related to the tests. label Oct 8, 2015

trevnorris reviewed Oct 8, 2015
View reviewed changes

Trott mentioned this pull request Oct 9, 2015

stringbytes-external fails on AIX. #3303

Closed

test: parallelize long-running test

c3672b0

Trott force-pushed the borked-test branch from 6a8a8b5 to c3672b0 Compare October 9, 2015 22:30

Trott added 2 commits October 9, 2015 15:34

fixup: update error message

3b9e2be

fixup

55a1d94

fixup: revert error message check

21e054b

jasnell closed this Oct 11, 2015

Trott mentioned this pull request Oct 12, 2015

test: split up buffer tests for reliability #3323

Closed

This was referenced Oct 27, 2015

Release proposal: v5.0.0 #3466

Merged

Preparing v4.2.2 LTS Maintenance nodejs/Release#50

Closed

jasnell added the lts-watch-v4.x label Oct 28, 2015

jasnell added land-on-v4.x and removed lts-watch-v4.x labels Oct 28, 2015

jasnell removed the land-on-v4.x label Nov 3, 2015

Trott deleted the borked-test branch January 13, 2022 22:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: parallelize long-running test #3287

test: parallelize long-running test #3287

Trott commented Oct 8, 2015

evanlucas Oct 8, 2015

evanlucas Oct 8, 2015

trevnorris Oct 8, 2015

evanlucas Oct 8, 2015

Trott commented Oct 8, 2015

evanlucas commented Oct 8, 2015

trevnorris Oct 8, 2015

Trott Oct 9, 2015

trevnorris Oct 9, 2015

Trott Oct 9, 2015

trevnorris commented Oct 9, 2015

Trott commented Oct 9, 2015

Trott commented Oct 10, 2015

Trott commented Oct 10, 2015

Trott commented Oct 10, 2015

Trott commented Oct 10, 2015

targos commented Oct 10, 2015

evanlucas commented Oct 10, 2015

trevnorris commented Oct 10, 2015

jasnell commented Oct 11, 2015

jasnell commented Oct 11, 2015

jasnell commented Oct 11, 2015

mscdex commented Oct 11, 2015

Trott commented Oct 11, 2015

mscdex commented Oct 11, 2015

Trott commented Oct 12, 2015

jasnell commented Oct 28, 2015

test: parallelize long-running test #3287

test: parallelize long-running test #3287

Conversation

Trott commented Oct 8, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Trott commented Oct 8, 2015

evanlucas commented Oct 8, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

trevnorris commented Oct 9, 2015

Trott commented Oct 9, 2015

Trott commented Oct 10, 2015

Trott commented Oct 10, 2015

Trott commented Oct 10, 2015

Trott commented Oct 10, 2015

targos commented Oct 10, 2015

evanlucas commented Oct 10, 2015

trevnorris commented Oct 10, 2015

jasnell commented Oct 11, 2015

jasnell commented Oct 11, 2015

jasnell commented Oct 11, 2015

mscdex commented Oct 11, 2015

Trott commented Oct 11, 2015

mscdex commented Oct 11, 2015

Trott commented Oct 12, 2015

jasnell commented Oct 28, 2015