CI Failures: 20180817 #14

joyeecheung · 2018-08-17T20:17:11Z

Failures in node-test-pull-request/16453 to node-test-pull-request/16516 that failed more than 2 PRs

JSTest Failure

Reason	`parallel/test-crypto-scrypt`
Type	JS_TEST_FAILURE
Failed PR	7 (nodejs/node#22330, nodejs/node#22128, nodejs/node#22290, nodejs/node#22287, nodejs/node#22265, nodejs/node#22329, nodejs/node#22281)
Appeared	test-joyent-ubuntu1604_sharedlibs_container-x64-4, test-digitalocean-ubuntu1604_sharedlibs_container-x64-6
First CI	https://ci.nodejs.org/job/node-test-pull-request/16457/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/16474/

Example

not ok 319 parallel/test-crypto-scrypt
  ---
  duration_ms: 1.128
  severity: fail
  exitcode: 1
  stack: |-
    /home/iojs/build/workspace/node-test-commit-linux-containered/nodes/ubuntu1604_sharedlibs_openssl110_x64/test/common/index.js:737
            throw new assert.AssertionError({
            ^
    
    AssertionError [ERR_ASSERTION]: Expected inputs to be strictly equal:
    + actual - expected ... Lines skipped
    
      Comparison {
    +   message: 'error:060B50AC:digital envelope routines:EVP_PBE_scrypt:memory limit exceeded',
    -   code: 'ERR_CRYPTO_SCRYPT_INVALID_PARAMETER',
    -   message: 'Invalid scrypt parameter',
        type: [Function: Error] {
    ...

Reason	`parallel/test-tls-passphrase`
Type	JS_TEST_FAILURE
Failed PR	7 (nodejs/node#22330, nodejs/node#22128, nodejs/node#22290, nodejs/node#22287, nodejs/node#22265, nodejs/node#22329, nodejs/node#22281)
Appeared	test-joyent-ubuntu1604_sharedlibs_container-x64-4, test-digitalocean-ubuntu1604_sharedlibs_container-x64-6
First CI	https://ci.nodejs.org/job/node-test-pull-request/16457/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/16474/

Example

not ok 1742 parallel/test-tls-passphrase
  ---
  duration_ms: 0.415
  severity: fail
  exitcode: 1
  stack: |-
    assert.js:650
        throw actual;
        ^
    
    Error: error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt
        at Object.createSecureContext (_tls_common.js:144:17)
        at Object.connect (_tls_wrap.js:1136:48)
        at /home/iojs/build/workspace/node-test-commit-linux-containered/nodes/ubuntu1604_sharedlibs_openssl110_x64/test/parallel/test-tls-passphrase.js:228:7
        at getActual (assert.js:563:5)
        at Function.throws (assert.js:680:24)
        at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/nodes/ubuntu1604_sharedlibs_openssl110_x64/test/parallel/test-tls-passphrase.js:227:8)
        at Module._compile (internal/modules/cjs/loader.js:689:30)
        at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
        at Module.load (internal/modules/cjs/loader.js:599:32)
        at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
  ...

Reason	`parallel/test-gc-http-client`
Type	JS_TEST_FAILURE
Failed PR	3 (nodejs/node#22340, nodejs/node#22350, nodejs/node#21611)
Appeared	test-digitalocean-freebsd11-x64-2, test-digitalocean-ubuntu1604_sharedlibs_container-x64-7, test-azure_msft-win2016-x64-4
First CI	https://ci.nodejs.org/job/node-test-pull-request/16468/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/16484/

Example

not ok 728 parallel/test-gc-http-client
  ---
  duration_ms: 0.371
  severity: fail
  exitcode: 1
  stack: |-
    We should do 500 requests
    Done: 0/500
    Collected: 0/198
    events.js:167
          throw er; // Unhandled 'error' event
          ^
    
    Error: connect ECONNRESET 127.0.0.1:46410 - Local (127.0.0.1:46638)
        at internalConnect (net.js:933:16)
        at defaultTriggerAsyncIdScope (internal/async_hooks.js:294:19)
        at GetAddrInfoReqWrap.emitLookup [as callback] (net.js:1080:9)
        at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:62:10)
    Emitted 'error' event at:
        at Socket.socketErrorListener (_http_client.js:399:9)
        at Socket.emit (events.js:182:13)
        at emitErrorNT (internal/streams/destroy.js:82:8)
        at emitErrorAndCloseNT (internal/streams/destroy.js:50:3)
        at process._tickCallback (internal/process/next_tick.js:63:19)
  ...

Reason	`parallel/test-stdio-pipe-redirect`
Type	JS_TEST_FAILURE
Failed PR	3 (nodejs/node#22320, nodejs/node#22202, nodejs/node#21593)
Appeared	test-azure_msft-win10-x64-1, test-rackspace-win2008r2-x64-1, test-azure_msft-win2012r2-x64-2, test-rackspace-win2012r2-x64-2, test-azure_msft-win2016-x64-2, test-azure_msft-win10-x64-4, test-rackspace-win2012r2-x64-1, test-azure_msft-win2016-x64-4, test-azure_msft-win10-x64-3, test-rackspace-win2012r2-x64-9, test-azure_msft-win2012r2-x64-1
First CI	https://ci.nodejs.org/job/node-test-pull-request/16458/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/16489/

Example

not ok 360 parallel/test-stdio-pipe-redirect
  ---
  duration_ms: 0.599
  severity: crashed
  stack: |-
    oh no!
    exit code: CRASHED
  ...

Reason	`sequential/test-http2-large-file`
Type	JS_TEST_FAILURE
Failed PR	3 (nodejs/node#22356, nodejs/node#21840, nodejs/node#22350)
Appeared	test-rackspace-win2008r2-x64-6
First CI	https://ci.nodejs.org/job/node-test-pull-request/16477/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/16499/

Example

not ok 573 sequential/test-http2-large-file
  ---
  duration_ms: 2.60
  severity: fail
  exitcode: 1
  stack: |-
    (node:7836) ExperimentalWarning: The http2 module is an experimental API.
    events.js:167
          throw er; // Unhandled 'error' event
          ^
    
    Error: read ECONNRESET
        at TCP.onread (net.js:661:25)
    Emitted 'error' event at:
        at emitErrorNT (internal/streams/destroy.js:82:8)
        at emitErrorAndCloseNT (internal/streams/destroy.js:50:3)
        at process._tickCallback (internal/process/next_tick.js:63:19)
  ...

Reason	`addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-binary`
Type	JS_TEST_FAILURE
Failed PR	2 (nodejs/node#22320, nodejs/node#21593)
Appeared	test-digitalocean-freebsd11-x64-2
First CI	https://ci.nodejs.org/job/node-test-pull-request/16458/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/16475/

Example

not ok 1904 addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-binary
  ---
  duration_ms: 49.461
  severity: crashed
  stack: |-
    oh no!
    exit code: CRASHED (Signal: 9)
  ...

Reason	`parallel/test-timers-throw-reschedule`
Type	JS_TEST_FAILURE
Failed PR	2 (nodejs/node#21593, nodejs/node#22202)
Appeared	test-linuxonecc-rhel72-s390x-1, test-rackspace-ubuntu1604-x64-2, test-linuxonecc-rhel72-s390x-2
First CI	https://ci.nodejs.org/job/node-test-pull-request/16475/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/16486/

Example

not ok 1510 parallel/test-timers-throw-reschedule
  ---
  duration_ms: 0.228
  severity: fail
  stack: |-
    assert.js:42
      throw new errors.AssertionError({
      ^
    
    AssertionError [ERR_ASSERTION]: false == true
        at Timeout.common.mustCall (/data/iojs/build/workspace/node-test-commit-linuxone/nodes/rhel72-s390x/test/parallel/test-timers-throw-reschedule.js:20:5)
        at Timeout._onTimeout (/data/iojs/build/workspace/node-test-commit-linuxone/nodes/rhel72-s390x/test/common/index.js:478:15)
        at ontimeout (timers.js:498:11)
        at Timer.unrefdHandle (timers.js:611:5)
  ...

Reason	`sequential/test-performance`
Type	JS_TEST_FAILURE
Failed PR	2 (nodejs/node#22318, nodejs/node#22288)
Appeared	test-digitalocean-ubuntu1604_sharedlibs_container-x64-7, test-requireio_andineck-debian9-armv6l_pi1p-1
First CI	https://ci.nodejs.org/job/node-test-pull-request/16459/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/16462/

Example

not ok 2289 sequential/test-performance
  ---
  duration_ms: 4.558
  severity: fail
  exitcode: 1
  stack: |-
    assert.js:338
        throw err;
        ^
    
    AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
    
      assert(Math.abs(delta) < 1000)
    
        at checkNodeTiming (/home/iojs/build/workspace/node-test-commit-linux-containered/nodes/ubuntu1604_sharedlibs_debug_x64/test/sequential/test-performance.js:64:7)
        at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/nodes/ubuntu1604_sharedlibs_debug_x64/test/sequential/test-performance.js:73:1)
        at Module._compile (internal/modules/cjs/loader.js:689:30)
        at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
        at Module.load (internal/modules/cjs/loader.js:599:32)
        at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
        at Function.Module._load (internal/modules/cjs/loader.js:530:3)
        at Function.Module.runMain (internal/modules/cjs/loader.js:742:12)
        at startup (internal/bootstrap/node.js:266:19)
        at bootstrapNodeJSCore (internal/bootstrap/node.js:611:3)
  ...

Jenkins Failure

Reason	`Build timed out (after 120 minutes). Marking the build as failed.`
Type	JENKINS_FAILURE
Failed PR	3 (nodejs/node#21593, nodejs/node#22340, nodejs/node#22373)
Appeared	test-digitalocean-fedora27-x64-1
First CI	https://ci.nodejs.org/job/node-test-pull-request/16489/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/16513/

Example

Build timed out (after 120 minutes). Marking the build as failed.

Git Failure

Reason	`Changes not staged for commit:`
Type	GIT_FAILURE
Failed PR	23 (nodejs/node#22269, nodejs/node#22318, nodejs/node#22330, nodejs/node#22300, nodejs/node#22329, nodejs/node#22281, nodejs/node#22356, nodejs/node#22271, nodejs/node#22355, nodejs/node#22345, nodejs/node#22197, nodejs/node#22204, nodejs/node#22104, nodejs/node#21840, nodejs/node#22128, nodejs/node#22290, nodejs/node#22361, nodejs/node#22350, nodejs/node#22287, nodejs/node#22288, nodejs/node#22368, nodejs/node#22328, nodejs/node#22342)
Appeared	test-joyent-smartos16-x64-1, test-joyent-ubuntu1604_sharedlibs_container-x64-3, test-digitalocean-ubuntu1604_sharedlibs_container-x64-1, test-packetnet-centos7-arm64-1, test-scaleway-ubuntu1604-armv7l-1, test-linuxonecc-rhel72-s390x-3, test-softlayer-ubuntu1604_sharedlibs_container-x64-5, test-linuxonecc-rhel72-s390x-1, test-softlayer-ubuntu1604_sharedlibs_container-x64-2, test-digitalocean-ubuntu1604_sharedlibs_container-x64-10, test-digitalocean-ubuntu1604_sharedlibs_container-x64-4, test-packetnet-centos7-arm64-2, test-digitalocean-ubuntu1604_sharedlibs_container-x64-6, test-joyent-smartos16-x64-2, test-osuosl-aix61-ppc64_be-1, test-digitalocean-ubuntu1604_sharedlibs_container-x64-7, test-digitalocean-ubuntu1604_sharedlibs_container-x64-2, test-packetnet-ubuntu1604-arm64-2, test-softlayer-ubuntu1604_sharedlibs_container-x64-1, test-linuxonecc-rhel72-s390x-2
First CI	https://ci.nodejs.org/job/node-test-pull-request/16453/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/16516/

Example

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   deps/v8/third_party/jinja2/LICENSE

Untracked files:
  (use "git add <file>..." to include in what will be committed)

	build/

no changes added to commit

Reason	`ERROR: Error fetching remote repo 'jenkins_tmp'`
Type	GIT_FAILURE
Failed PR	9 (nodejs/node#22287, nodejs/node#22340, nodejs/node#21397, nodejs/node#22348, nodejs/node#22359, nodejs/node#22373, nodejs/node#22368, nodejs/node#22328, nodejs/node#22342)
Appeared	test-azure_msft-win10-x64-3
First CI	https://ci.nodejs.org/job/node-test-pull-request/16502/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/16516/

Example

ERROR: Error fetching remote repo 'jenkins_tmp'
hudson.plugins.git.GitException: Failed to fetch from binary_tmp@147.75.70.237:binary_tmp.git
	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:889)
	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1146)
	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1177)
	at hudson.scm.SCM.checkout(SCM.java:504)

Reason	`FATAL: Could not checkout ...`
Type	GIT_FAILURE
Failed PR	2 (nodejs/node#22281, nodejs/node#22355)
Appeared	test-joyent-freebsd10-x64-1, test-softlayer-centos6-x64-1, test-softlayer-centos7-x64-1, test-rackspace-fedora27-x64-1, test-digitalocean-ubuntu1404-x64-1, test-joyent-ubuntu1710-x64-1, test-digitalocean-ubuntu1804_container-x64-2, test-macstadium-macos10.11-x64-1, test-joyent-smartos16-x64-1, test-joyent-smartos17-x64-1, test-scaleway-ubuntu1604-armv7l-3, test-osuosl-aix61-ppc64_be-2, test-rackspace-centos7-x64-1, test-macstadium-macos10.11-x64-2
First CI	https://ci.nodejs.org/job/node-test-pull-request/16474/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/16479/

Example

FATAL: Could not checkout 1aaf64d889f049a31ae0e1245a7631911b25acc7
hudson.plugins.git.GitException: Command "git checkout -f 1aaf64d889f049a31ae0e1245a7631911b25acc7" returned status code 128:
stdout: 
stderr: fatal: reference is not a tree: 1aaf64d889f049a31ae0e1245a7631911b25acc7

	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2050)

Build Failure

Reason	bash: line 2: syntax error near unexpected token `newline'
Type	BUILD_FAILURE
Failed PR	3 (nodejs/node#22340, nodejs/node#22373, nodejs/node#22368)
Appeared	test-digitalocean-ubuntu1404-x64-1, test-rackspace-ubuntu1604-x64-2, test-digitalocean-ubuntu1404-x86-1, test-macstadium-macos10.11-x64-2
First CI	https://ci.nodejs.org/job/node-test-pull-request/16508/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/16514/

Example

bash: line 2: syntax error near unexpected token `newline'

Reason	`fatal: Couldn't find remote ref refs/heads/v4.x-staging`
Type	BUILD_FAILURE
Failed PR	3 (nodejs/node#22330, nodejs/node#22320, nodejs/node#22318)
Appeared	test-requireio_chrislea-debian9-armv6l_pi1p-1, test-requireio_mininodes-debian9-armv7l_pi2-1, test-requireio_mininodes-debian9-armv6l_pi1p-1, test-requireio_bengl-debian9-armv6l_pi1p-1, test-requireio_continuationlabs-debian9-armv6l_pi1p-1, test-requireio_svincent-debian9-armv7l_pi2-1, test-requireio_mcollina-debian9-armv7l_pi2-1, test-requireio_svincent-debian9-armv7l_pi2-2, test-requireio_joeyvandijk-debian9-armv7l_pi2-1, test-requireio_jasnell-debian9-armv7l_pi2-1
First CI	https://ci.nodejs.org/job/node-test-pull-request/16457/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/16459/

Example

fatal: Couldn't find remote ref refs/heads/v4.x-staging
fatal: The remote end hung up unexpectedly

Reason	`ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?`
Type	BUILD_FAILURE
Failed PR	2 (nodejs/node#22340, nodejs/node#22328)
Appeared	test-joyent-smartos17-x64-1, test-osuosl-aix61-ppc64_be-2, test-osuosl-aix61-ppc64_be-1, test-joyent-smartos16-x64-2
First CI	https://ci.nodejs.org/job/node-test-pull-request/16508/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/16515/

Example

ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
Notifying upstream projects of job completion
Finished: FAILURE

The text was updated successfully, but these errors were encountered:

joyeecheung · 2018-08-17T20:21:04Z

Progress

In Progress:

sequential/test-http2-large-file: Investigate flaky test-http2-large-file node#22327
bash: line 2: syntax error near unexpected token newline': GitHub network inconsistancy build#1458
sequential/test-performance: Investigate flaky parallel/test-performance node#19197 opened when it was in parallel, fix in test: harden sequential/test-performance node#22404
Build timed out (after 120 minutes). Marking the build as failed.: test-digitalocean-fedora27-x64-1 keeps timing out build#1463

Fixed

parallel/test-gc-http-client: Investigate flaky parallel/test-gc-http-client node#22336, fix in test: harden test-gc-http-client node#22373
ERROR: Error fetching remote repo 'jenkins_tmp': Git lock in test-azure_msft-win10-x64-3 exists build#1460
parallel/test-timers-throw-reschedule (v8.x only): Windows intermittent ? - parallel/test-timers-throw-reschedule node#21188 fix in test: move test-timers-throw-reschedule to sequential node#22379 (marked it as flaky)
parallel/test-stdio-pipe-redirect (v8.x only): Investigate flaky parallel/test-stdio-pipe-redirect node#22378
parallel/test-crypto-scrypt by Upgrade to OpenSSL-1.1.0i node#22318
parallel/test-tls-passphrase by Upgrade to OpenSSL-1.1.0i node#22318
addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-binary by test: call gc() explicitly to avoid OOM node#22301 (pending a real fix in Investigate flaky addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-binary on FreeBSD 11 node#22317)
Changes not staged for commit: by deps: fix CRLF in text file not present in upstream node#22340
ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?: should be fixed by deps: fix CRLF in text file not present in upstream node#22340

???

FATAL: Could not checkout ...: looks like some temporary git failure, I think it's gone now?
fatal: Couldn't find remote ref refs/heads/v4.x-staging: see CI failures: 20180815 #13 I think it's gone now

New format is in nodejs/reliability#14 Added a few more patterns and linkify the machine names as well as console of example stacks

1) This adds a better error logging so we are able to address further failures easier. 2) It adds a extra epsilon so the test runs into less issues in case the machine is under heavy load. 3) The epsilon in increased if the CPU is under heavy load. 4) The total startup epsilon was reduced due to recent startup time improvements. PR-URL: nodejs#22404 Fixes: nodejs#19197 Refs: nodejs/reliability#14 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

1) This adds a better error logging so we are able to address further failures easier. 2) It adds a extra epsilon so the test runs into less issues in case the machine is under heavy load. 3) The epsilon in increased if the CPU is under heavy load. 4) The total startup epsilon was reduced due to recent startup time improvements. PR-URL: #22404 Fixes: #19197 Refs: nodejs/reliability#14 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

New format is in nodejs/reliability#14 Added a few more patterns and linkify the machine names as well as console of example stacks

priyank-p pushed a commit to nodejs/node-core-utils that referenced this issue Aug 17, 2018

ci: tweak patterns, improve markdown output

94a7193

New format is in nodejs/reliability#14 Added a few more patterns and linkify the machine names as well as console of example stacks

BridgeAR mentioned this issue Aug 19, 2018

test: harden sequential/test-performance nodejs/node#22404

Closed

4 tasks

joyeecheung closed this as completed Aug 20, 2018

johnfrench3 pushed a commit to johnfrench3/core-utils-node that referenced this issue Nov 2, 2022

ci: tweak patterns, improve markdown output

beeb213

New format is in nodejs/reliability#14 Added a few more patterns and linkify the machine names as well as console of example stacks

renawolford6 added a commit to renawolford6/node-dev-build-core-utils that referenced this issue Nov 10, 2022

ci: tweak patterns, improve markdown output

724bb19

New format is in nodejs/reliability#14 Added a few more patterns and linkify the machine names as well as console of example stacks

Developerarif2 pushed a commit to Developerarif2/node-core-utils that referenced this issue Jan 27, 2023

ci: tweak patterns, improve markdown output

2f50d56

New format is in nodejs/reliability#14 Added a few more patterns and linkify the machine names as well as console of example stacks

gerkai added a commit to gerkai/node-core-utils-project-build that referenced this issue Jan 27, 2023

ci: tweak patterns, improve markdown output

a3a6277

New format is in nodejs/reliability#14 Added a few more patterns and linkify the machine names as well as console of example stacks

patrickm68 added a commit to patrickm68/NodeJS-core-utils that referenced this issue Sep 14, 2023

ci: tweak patterns, improve markdown output

c3b7ef1

New format is in nodejs/reliability#14 Added a few more patterns and linkify the machine names as well as console of example stacks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CI Failures: 20180817 #14

CI Failures: 20180817 #14

joyeecheung commented Aug 17, 2018

joyeecheung commented Aug 17, 2018 •

edited

Loading

CI Failures: 20180817 #14

CI Failures: 20180817 #14

Comments

joyeecheung commented Aug 17, 2018

JSTest Failure

Jenkins Failure

Git Failure

Build Failure

joyeecheung commented Aug 17, 2018 • edited Loading

Progress

joyeecheung commented Aug 17, 2018 •

edited

Loading