Bug: Angular CLI test running module build/chrome concurrently

[SelfishLucho]

Bug Report or Feature Request (mark with an x)

- [x ] bug report -> please search issues before submitting
- [ ] feature request

Versions.

"dependencies": {
"@angular/animations": "4.3.1",
"@angular/common": "4.3.1",
"@angular/compiler": "4.3.1",
"@angular/core": "4.3.1",
"@angular/forms": "4.3.1",
"@angular/http": "4.3.1",
"@angular/platform-browser": "4.3.1",
"@angular/platform-browser-dynamic": "4.3.1",
"@angular/router": "4.3.1",
"core-js": "2.4.1",
"rxjs": "5.1.0",
"zone.js": "0.8.5"
},
"devDependencies": {
"@angular/cli": "1.3.1",
"@angular/compiler-cli": "4.3.1",
"@types/jasmine": "2.5.38",
"@types/node": "6.0.60",
"codelyzer": "2.0.0",
"jasmine-core": "2.5.2",
"jasmine-spec-reporter": "3.2.0",
"json-server": "0.9.6",
"karma": "1.4.1",
"karma-chrome-launcher": "2.0.0",
"karma-cli": "1.0.1",
"karma-coverage-istanbul-reporter": "0.2.0",
"karma-jasmine": "1.1.0",
"karma-jasmine-html-reporter": "0.2.2",
"karma-junit-reporter": "1.2.0",
"karma-remap-istanbul": "0.6.0",
"protractor": "5.1.0",
"ts-node": "2.0.0",
"tslint": "4.5.0",
"typescript": "2.2.0"
}

node: 6.10.1
os: linux x64 (Bash on Ubuntu)

Repro steps.

I'm trying to run tests with following:
"ng test -cc --single-run=true"

In v.1.0.0, when doing ng test, the tests will do the following in serial order:

1. Build modules
2. Start chrome
3. Start the test suite

In v.1.3.1, the tests starts Chrome at the same time it starts to build the modules,
it tries to build modules and start Chrome concurrently. The result of this is that Chrome is waiting too long,
resulting in a "Chrome have not captured in 60000 ms, killing."-error.

The log given by the failure.

ng test -cc --single-run=true

0% compiling�������������� 10% building modules 0/1 modules 1 active ...

.... module building....

active� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� ������������� 71% sealing������������ 72% optimizing��������������� 73% basic module optimization� �� �� �� �� �� ������������������������� 74% module optimization������������������������ 75% advanced module optimization� �� �� �� ������������������������������ 76% basic chunk optimization����������������������������� 76% basic chunk optimization pass 2������������������������������������ 76% basic chunk optimization pass 3� �� �� �� �� �� �� �� �� �� �� �� �� ������������������������ 77% chunk optimization����������������������� 78% advanced chunk optimization�������������������������������� 79% module and chunk tree optimization� �� �� �� �� �� �� �� �������������������������������� 80% chunk modules optimization������������������������������� 81% advanced chunk modules optimization� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� ��������������������� 82% module reviving�������������������� 83% module order optimization� �� �� ���������������������������� 84% module id optimization� �� �� �� �� �� �� �� �������������������� 85% chunk reviving������������������� 86% chunk order optimization� �� �� ��������������������������� 87% chunk id optimization� �� �� �� �� �� �� �� �� �� �� �� �� �� ������������� 88% hashing������������ 89% module assets processing� ����������������������������� 90% chunk assets processing���������������������������� 91% additional chunk assets processing� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� ��������������� 92% recording�������������� 91% additional asset processing� �� �� ������������������������������ 92% chunk asset optimization� �� �� �� �� �� ������������������������ 94% asset optimization�[33m11 09 2017 13:23:46.394:WARN [launcher]: �[39mChrome have not captured in 60000 ms, killing.
� �� �� �� �� �� �� �� �� �� �������������� 95% emitting� �� �� �� �� �� �� �� �� �� �� �� �� ��[32m11 09 2017 13:23:50.802:INFO [launcher]: �[39mTrying to start Chrome again (1/2).
�[32m11 09 2017 13:23:58.517:INFO [Chromium 59.0.3071 (Ubuntu 0.0.0)]: �[39mConnected on socket lEAXE18XTbsYhWwIAAAA with id 21814804
�[33m11 09 2017 13:24:09.216:WARN [Chromium 59.0.3071 (Ubuntu 0.0.0)]: �[39mDisconnected (1 times), because no message in 10000 ms.
Chromium 59.0.3071 (Ubuntu 0.0.0) ERROR
Disconnected, because no message in 10000 ms.
Chromium 59.0.3071 (Ubuntu 0.0.0) ERROR
Disconnected, because no message in 10000 ms.

Chromium 59.0.3071 (Ubuntu 0.0.0) ERROR
Disconnected, because no message in 10000 ms.
Chromium 59.0.3071 (Ubuntu 0.0.0) ERROR
Disconnected, because no message in 10000 ms.

�[33m11 09 2017 13:24:11.505:WARN [launcher]: �[39mChrome was not killed in 2000 ms, sending SIGKILL.

npm ERR! Linux 4.4.0-62-generic
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "run-script" "test-coverage"
npm ERR! node v6.10.1
npm ERR! npm v3.10.10
npm ERR! code ELIFECYCLE
npm ERR! ��������������������������� : ng test -cc --single-run=true
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the ��������������������������� test-coverage script 'ng test -cc --single-run=true'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the ngflash-webgui package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! ng test -cc --single-run=true
npm ERR! You can get information on how to open an issue for this project with:
npm ERR! npm bugs ngflash-webgui
npm ERR! Or if that isn't available, you can get their info via:
npm ERR! npm owner ls ngflash-webgui
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR! ��������������������������� /npm-debug.log
:ngTestCoverage FAILED

FAILURE: Build failed with an exception.

• What went wrong:
  Execution failed for task ':ngTestCoverage'.

Process 'command 'npm'' finished with non-zero exit value 1

• Try:
  Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 6 mins 36.346 secs

Desired functionality.

Finishing running module build before executing Chrome in order to avoid time out.

Mention any other details that might be useful.

Older working package dependencies with previous version 1.0.0:

"dependencies": {
"@angular/animations": "4.0.0",
"@angular/common": "4.0.0",
"@angular/compiler": "4.0.0",
"@angular/core": "4.0.0",
"@angular/forms": "4.0.0",
"@angular/http": "4.0.0",
"@angular/platform-browser": "4.0.0",
"@angular/platform-browser-dynamic": "4.0.0",
"@angular/router": "4.0.0",
"core-js": "2.4.1",
"rxjs": "5.1.0",
"zone.js": "0.8.5"
},
"devDependencies": {
"@angular/cli": "1.0.0",
"@angular/compiler-cli": "4.0.0",
"@types/jasmine": "2.5.38",
"@types/node": "6.0.60",
"codelyzer": "2.0.0",
"jasmine-core": "2.5.2",
"jasmine-spec-reporter": "3.2.0",
"json-server": "0.9.6",
"karma": "1.4.1",
"karma-chrome-launcher": "2.0.0",
"karma-cli": "1.0.1",
"karma-coverage-istanbul-reporter": "0.2.0",
"karma-jasmine": "1.1.0",
"karma-jasmine-html-reporter": "0.2.2",
"karma-junit-reporter": "1.2.0",
"karma-remap-istanbul": "0.6.0",
"protractor": "5.1.0",
"ts-node": "2.0.0",
"tslint": "4.5.0",
"typescript": "2.2.0"
}

[hansl]

Hi @SelfishLucho,

I cannot seem to be able to reproduce this issue with the CLI 1.4.1 (latest version). Could you either try with that version, or provide a project where we could reproduce and investigate?

I'll keep this alive for a few days to wait for your answer.
Thanks!

[SelfishLucho]

Hi @hansl,

I tried with version 1.4.1 and it didn't work out, however stumbled into this issue #7296 and added the flag --sourcemap=false. Still the same behavior as originally but this time it actually ran through the tests successfully, did this give any hints on why this could be?

Let me figure out to reproduce this on a separate project...

[jongunter]

I have the same issue.

ng test --single-run results in Disconnected, because no message in 1000ms errors.
ng test --single-run --sourcemap=false runs all the tests just fine

Oddly enough, ng test works fine too. It just seems to be --single-run causing the problem?

[webstp]

@hansl You can view this behaviour from a new project created using the CLI. Running ng test will start both the webpack process and the browser at the same time.

INFO [launcher]: Starting browser Chrome
 25% building modules 133/144 modules 11 active ...s/modules/core.get-iterator-method.js

If you're application if large enough then you will run into the following

WARN [launcher]: Chrome have not captured in 60000 ms, killing.
WARN [launcher]: Chrome was not killed in 2000 ms, sending SIGKILL.

As a workaround you can add more retries to the karma.conf.js to be able to eventually run the browser after webpack finishes. (by default karma will retry 3 times, and has a captureTimeout of 60 seconds)

module.exports = function (config) {
    config.set({
        /* the rest of it */
        retryLimit: 5
    });
};

Ideally the broswer shouldn't be launched until after wepack finishes

[nathanBrenner]

I'm getting that error message in CircleCI "Chrome have not captured in 60000 ms, killing."

My circle is set for version 1, and this is the test section in circle.yml, which is what is suggested in the angular cli docs:

test:
  override:
    - xvfb-run -a npm run test -- --single-run --no-progress --browser=ChromeNoSandbox

I also tried updating the version of Chrome that circle runs (45), to the latest version (61), but that didn't change the outcome.

[nathanBrenner]

Looks like the issue is with Karma:

jasmine/jasmine#1327

tl;dr try changing the jasmine-core version to 2.5.2, and or changing all beforeEach(() => to beforeEach(async(() =>. I've tried both together and it's still getting that error.

I did however, start a new cli app, and it worked fine, until I copied over all of my components, so I think the main issue is with having too many unit tests which in my case is over 400 atm.

[jongunter]

I mentioned before that ng test --single-run --sourcemap=false worked. After many weeks of working fine, it has now quit working and broke all our CI builds :(

I've used git log any changes to our package.json file and I can't seem to find anything would break the unit testing functionality. I guess my version numbers in package.json aren't strict enough?

As before, ng test --single-run doesn't work.
ng test still works.

[jongunter]

Regarding my previous comment, it looks like if I have ng serve running in another terminal window, ng test --single-run --sourcemap=false will fail, too. If I stop the ng serve process, ng test --single-run --sourcemap=false works great.

ng-test --single-run still broken for me, regardless of if I'm running ng serve in another terminal window.

All of this was discovered on my Win8 PC. I'll need to verify the same behavior occurs next time I'm on my Mac.

[jongunter]

I should also mention that running ng-test --single-run, seems to launch a chrome window with the correct URL (and the tests look like they run in the browser window), but the console application cannot communicate with Chrome, apparently.

[jongunter]

I created a repo that reproduces this issue. Please try to run ng test --single-run with it (I use Windows 8 with NPM 3.10.10 and Node 6.11.10); see if you get the same error.
https://github.com/jongunter/ng-cli-test-issue-repro