Testing directly with karma #10703

krokofant · 2018-05-07T14:27:40Z

Versions

    "@angular-devkit/build-angular": "~0.6.0",
    "@angular-devkit/core": "0.0.29",
    "@angular/cli": "6.0.0",
    "@angular/common": "6.0.0",
    "@angular/compiler": "6.0.0",
    "@angular/compiler-cli": "6.0.0",
    "@angular/core": "6.0.0",
    "@angular/forms": "6.0.0",
    "@angular/http": "6.0.0",
    "@angular/language-service": "6.0.0",
    "@angular/platform-browser": "6.0.0",
    "@angular/platform-browser-dynamic": "6.0.0",
    "@angular/router": "6.0.0",
    "@angular/upgrade": "6.0.0",

Windows 10

Repro steps

Run karma start karma.conf.js

Observed behavior

s\angular_devkit\build_angular\src\angular-cli-files\plugins\karma.ts:53
    throw new Error(`The '@angular-devkit/build-angular/plugins/karma' karma plugin is meant to` +
          ^
Error: The '@angular-devkit/build-angular/plugins/karma' karma plugin is meant to be used from within Angular CLI and will not work correctly outside of it.

Desired behavior

Be able to run test directly with karma as before version 6

This became a problem after upgrading to version 6.

The text was updated successfully, but these errors were encountered:

jbandi · 2018-05-07T22:49:15Z

This also breaks integration with the test runner in WebStorm & IDEA ... it's not possible to run tests from WebStorm any more by pressing the "green triangles" next to the test code :-(

vycoder · 2018-05-08T07:33:44Z

Anybody know any alternative on running the tests? I just migrated to angular 6 as well. Now I can't run our tests. Wondering what's the suggested way to run test now? Do I need some cli configuration or something?

krokofant · 2018-05-08T08:01:34Z

@josephharveyangeles You can tunnel everything via the angular cli like

ng test --karma-config karma.somestuff.conf.js

jbandi · 2018-05-08T12:21:15Z

For the IntelliJ (WebStorm, IDEA) integration the issue is tracked here:
https://youtrack.jetbrains.com/issue/WEB-32653

buster · 2018-05-09T11:08:49Z

Obviously this was broken by #10478 but the issue doesn't say why this is an error now..
Wouldn't the sensible thing to do, to make it a warning instead of an error before turning it into an error?
This currently prevents us from updating a project to angular 6 and i can imagine that the IntelliJ side won't be fixed in the short term..

Also, please vote on https://youtrack.jetbrains.com/issue/WEB-32653 if you are affected.

buster · 2018-05-15T08:58:59Z

FYI, according to the jetbrains ticket, the issue will be fixed in IDEA 2018.1.4

ciarancourtney · 2018-05-17T09:44:57Z

Still broken on:

IntelliJ IDEA 2018.2 EAP (Ultimate Edition)
Build #IU-182.2371.4, built on May 15, 2018

timdoes · 2018-05-18T17:41:38Z

And it is still broken on

Webstorm 2018.2 EAP
Build #WS-182.2371.25, built on May 17, 2018

AlexGS74 · 2018-05-18T17:47:04Z

@timdoes It works if you create a new run configuration. It will detect Cli and use it as Karma engine. If you run an old configuration still pointing to a global/local karma instance, it fails.

maprox · 2018-05-18T19:16:23Z

@AlexGS74 I've used your suggestion but still see "Error: Cannot find module '[myproject]\node_modules\@angular\cli\lib\server.js'". I'm om 2018.1.4

timdoes · 2018-05-18T19:56:47Z

@AlexGS74 That worked. Thanks!

AlexGS74 · 2018-05-19T02:22:16Z

@maprox Latest released version of Webstorm is 2018.1.3. It doesn't support this as it was release before Angular Cli 6 release.
What I describe above works with current Webstorm 2018.2 EAP, which will eventually turn into release.

tschach · 2018-05-22T10:22:09Z

According to the WebStorm Release Notes this should be fixed: https://confluence.jetbrains.com/display/WI/WebStorm+181.5087.27+Release+Notes
I am unable to test it since I am working with the IDEA.
For the IDEA I have opened a separate issue, as this seems to be only fixed in WebStorm: https://youtrack.jetbrains.com/issue/IDEA-192411

tschach · 2018-05-22T13:17:25Z

Hi, I have updated IntelliJ IDEA to 2018.1.4 (Build #IU-181.5087.20, built on May 16, 2018) and the Karma plugin to 181.5087.27 and now it works, provided you point your Karma package to your Angular-CLI package and not to your Karma package (as suggested by @AlexGS74).

ilancohen · 2018-05-30T08:12:25Z

With IntelliJ IDEA 2018.1.4, I'm using the config that @tschach posted, but still getting the error that @maprox got:
"Error: Cannot find module '[myproject]\node_modules@angular\cli\lib\server.js'".
Does anyone have a solution to this?

ilancohen · 2018-05-30T08:23:36Z

@maprox - Fixed it, once I updated the Karma plugin.

Zephilim · 2018-06-11T16:31:46Z

This is not just an intellij issue. I am running into this problem on macOS running "karma start" from the command line and I get this error. I'm using Angular 6.0.4 and karma 2.0.2

"dependencies": {
"@angular/animations": "^6.0.4",
"@angular/common": "^6.0.4",
"@angular/compiler": "^6.0.4",
"@angular/core": "^6.0.4",
"@angular/forms": "^6.0.4",
"@angular/http": "^6.0.4",
"@angular/platform-browser": "^6.0.4",
"@angular/platform-browser-dynamic": "^6.0.4",
"@angular/router": "^6.0.4",
...

"devDependencies": {
"@angular-devkit/build-angular": "~0.6.5",
"@angular/cli": "^6.0.8",
"@angular/compiler-cli": "^6.0.4",
"@angular/language-service": "^6.0.4",
"@types/chartist": "^0.9.37",
"@types/jasmine": "^2.8.8",
"@types/jasminewd2": "~2.0.2",
"@types/jquery": "^3.2.5",
"@types/node": "~6.0.60",
"codelyzer": "^4.3.0",
"jasmine-core": "~2.6.2",
"jasmine-spec-reporter": "^4.2.1",
"karma": "^2.0.2",
"karma-chrome-launcher": "^2.2.0",
"karma-cli": "~1.0.1",
"karma-coverage-istanbul-reporter": "^1.2.1",
"karma-jasmine": "~1.1.0",
"karma-jasmine-html-reporter": "^0.2.2",
...

If there is a work around for this environment, please let me know.

krokofant · 2018-06-12T06:57:15Z

@Zephilim I mentioned this solution in an above comment:

Tunnel the karma config via ng test:

ng test --karma-config karma.somestuff.conf.js

alexis-mrc · 2018-06-12T07:04:36Z

@krokofant does your command solve the problem exposed by @jbandi : "running the test from the green triangles in webstorm next to the test code"
I get empty test suite all the time I start the test from the test runners triangles.

Zephilim · 2018-06-12T08:51:40Z

Thanks krokofant, I guess I didn't really get what you meant by tunnel. I actually discovered (by accident) another way of running the coverage and that is simply "ng test --code-coverage", just not very well documented.

therepo90 · 2018-07-19T14:24:14Z

For me updating webstorm it helped

mscudlik · 2018-07-31T12:43:20Z

doesn't help if you want to use @angular-devkit/build-angular without @angular/cli

nanoTitan · 2018-08-10T20:52:26Z

The original issue as the author has stated is that upgrading to Angular 6 has broken the ability to do:
Run karma start karma.conf.js

This by itself has nothing to do with Webstorm, and many of us don't use that IDE. Is there any resolution to this problem as stated?

nikdo · 2018-08-16T10:50:28Z

@krokofant Tunneling it via ng test does not work for me. The command immediately terminates with no output. Do I need some configuration in angular.json to make it work? Is there an option to make ng test more verbose?

I'm really disappointed that there is no option to go without @angular/cli. We've ejected our Angular project two years ago and now after long and painful upgrade to Angular 6 (there is almost no documentation for upgrading ejected projects) we are left with @angular/cli@1.7.4.

Danieltreiber · 2018-09-11T16:07:03Z

I fixed this problem by removing and re-enabling the Angular CLI Framework. I did this by removing the AngularJS plugin, restarting, enabling it, restarting x2 (not sure why I needed to restart again).

This will tell karma to use node_modules/@angular/cli/bin/ng test vs node_modules/karma/bin/karma start (you can see the command that karma is running in the Karma Server tab)

nikdo · 2018-09-12T07:39:36Z

Well, we have found a workaround: We ditched Karma and replaced it with Jest.

I just wonder why ejected and non-cli solutions are left behind with many issues and no documentation. :(

edarioq · 2018-09-23T22:06:57Z

Having the same issue, really don't want to have to start over with another testing framework, any fixes? We don't use the Angular CLI and we have a hybrid AngularJS/Angular 6 app, reason which we are not using the CLI, the project is too big and too costly for an immediate migration.

peter-snr · 2018-10-15T09:19:09Z

This is just going to encourage developers to ignore tests and they are so difficult to set up and there is so much information out there that is just plain wrong.

I wanted some html reports for our build server, been at it hours now and still nothing?
I have the karma-jasmine-html-reporter installed in my karma.conf but nope, no html reports?

Really lost, this is so easy in JAVA with JUNIT, why so difficult using Angular 6 ???
Seems @nikdo has the solution ditch it and start again with something that actually works, Jest??

Edit: - This might be useful so I have added it here.

https://medium.com/welldone-software/an-overview-of-javascript-testing-in-2018-f68950900bc3

In short, if you want to “just get started” or looking for a fast framework for large projects, go with Jest.

And this will help you get started with Jest.
https://codeburst.io/angular-6-ng-test-with-jest-in-3-minutes-b1fe5ed3417c

james-schwartzkopf · 2018-10-15T15:18:43Z

@peter-snr this issue doesn't seem to be related to your problems. This isn't stack exchange, but karma-jasmine-html-reporter doesn't do what you think it does. It provides a nice interface to show in the browser while your test are running, not a final HTML report. Given your JUnit comment, you might be interested in karma-junit-reporter (remember to also add it to the reporters array).

IonelLupu · 2018-12-10T11:07:01Z

Did anyone encountered ChromeHeadless have not captured in 60000 ms, killing when running from WebStorm/PHPStorm 2018.3? It works fine when I run ng test in my terminal but when creating a run configuration inside the IDE it doesn't work.

cport1 · 2018-12-10T17:33:28Z

Did anyone encountered ChromeHeadless have not captured in 60000 ms, killing when running from WebStorm/PHPStorm 2018.3? It works fine when I run ng test in my terminal but when creating a run configuration inside the IDE it doesn't work.

When you go to Run/Debug Configurations, make sure the Config file is set to your src/karma.conf.js and your Karma Package is pointed at your node_modules/@angular/cli ...

IonelLupu · 2018-12-11T11:31:19Z

@cport1 . I think I've setup all of theese. This is my IDE configuration:

And this is the output:

/frontend/node_modules/@angular/cli/bin/ng test frontend --karma-config ~/.PhpStorm2018.3/config/plugins/js-karma/js_reporter/karma-intellij/lib/intellij.conf.js --source-map
 10% building modules 1/1 modules 0 active(node:13055) DeprecationWarning: Tapable.plugin is deprecated. Use new API on `.hooks` instead
 10% building modules 3/4 modules 1 active /frontend/src sync /\.spec\.ts$/11 12 2018 11:40:55.931:INFO [karma]: Karma v1.7.1 server started at http://0.0.0.0:9876/
11 12 2018 11:40:55.932:INFO [launcher]: Launching browser ChromeKarma with unlimited concurrency
11 12 2018 11:40:55.959:INFO [launcher]: Starting browser ChromeHeadless
Waiting for a captured browser... To capture a browser open http://localhost:9876/
11 12 2018 11:41:55.983:WARN [launcher]: ChromeHeadless have not captured in 60000 ms, killing.
11 12 2018 11:41:55.990:INFO [launcher]: Trying to start ChromeHeadless again (1/2).
11 12 2018 11:42:56.043:WARN [launcher]: ChromeHeadless have not captured in 60000 ms, killing.
11 12 2018 11:42:56.055:INFO [launcher]: Trying to start ChromeHeadless again (2/2).
11 12 2018 11:43:56.114:WARN [launcher]: ChromeHeadless have not captured in 60000 ms, killing.
11 12 2018 11:43:56.120:ERROR [launcher]: ChromeHeadless failed 2 times (timeout). Giving up.

As you can see, even if I set my karma.conf.js it still uses the config/plugins/js-karma/js_reporter/karma-intellij/lib/intellij.conf.js, which is strange

avoerman · 2018-12-13T15:28:07Z

@jcwolf Node might be running out of memory; this started happening to me and I added

--max_old_space_size=5048

to the Node options in the debug configuration. It could also mean a type error. WebStorm occasionally swallows these, so make sure your tests can compile 'ng test --watch=false' from a terminal.

IonelLupu · 2018-12-14T14:04:24Z

@avoerman The tests are running fine from terminal (when I run inside the docker container) but In WebsStorm/PHPStorm it fails with the same error. Maybe I have to set something similar to Remote CLI Interpreters from docker for karma like I set for PHP?

mgechev · 2018-12-14T19:22:36Z

In the CLI we want to provide a well encapsulated build; invoking the tests manually with karma breaks this encapsulation.

I'd recommend you to use the solution from this comment #10703 (comment).

If I've missed something or misunderstood your use case, please open a new issue with details.

marthinus-engelbrecht · 2019-04-13T12:41:28Z

@mgechev Problem is when that encapsulated build has issues. See #11672. The tight coupling here isn't making angular robust. Really don't want to be without test coverage reporting even though I let my tests drive implementation. And yes, I should look more into the issue myself and help fix it but right at this moment, I don't have the time. I'm also sure there are a billion other use cases where you don't want to be forced to use ng test

angular-automatic-lock-bot · 2019-09-09T11:41:30Z

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}

krokofant changed the title ~~Testing directly with karma (without angular-cli)~~ Testing directly with karma May 7, 2018

clydin added the comp: cli/test label May 8, 2018

avbentem mentioned this issue May 17, 2018

Can't run 'ng test' on all apps in a project and provide overrides #10765

Closed

pgingras mentioned this issue May 18, 2018

[feature-request/regression] Allow to run tests/coverage directly from IDE #10755

Closed

ghost mentioned this issue May 24, 2018

Cannot run tests of empty project with plain karma -> integration with JetBrains's IntelliJ is broken angular/devkit#846

Closed

sheriffjohn mentioned this issue Jun 8, 2018

Karma-tester failar slumpmässigt Vastra-Gotalandsregionen/komponentkartan#281

Closed

1 task

filipesilva added area: @angular-devkit/build-angular and removed area: @angular-devkit/build-angular comp: cli/test labels Jun 8, 2018

alexeagle mentioned this issue Sep 4, 2018

Cannot run tests of empty project with plain karma -> integration with JetBrains's IntelliJ is broken #12108

Closed

mgechev closed this as completed Dec 14, 2018

eirslett mentioned this issue May 21, 2019

ERRORS & BUILD FAILURE: When executing tests with karma eirslett/frontend-maven-plugin#819

Open

angular-automatic-lock-bot bot locked and limited conversation to collaborators Sep 9, 2019

Testing directly with karma #10703

Testing directly with karma #10703

Comments

krokofant commented May 7, 2018

Versions

Repro steps

Observed behavior

Desired behavior

jbandi commented May 7, 2018 • edited Loading

vycoder commented May 8, 2018

krokofant commented May 8, 2018

jbandi commented May 8, 2018 • edited Loading

buster commented May 9, 2018 • edited Loading

buster commented May 15, 2018

ciarancourtney commented May 17, 2018

timdoes commented May 18, 2018

AlexGS74 commented May 18, 2018

maprox commented May 18, 2018 • edited Loading

timdoes commented May 18, 2018

AlexGS74 commented May 19, 2018

tschach commented May 22, 2018

tschach commented May 22, 2018

ilancohen commented May 30, 2018

ilancohen commented May 30, 2018

Zephilim commented Jun 11, 2018

krokofant commented Jun 12, 2018

alexis-mrc commented Jun 12, 2018

Zephilim commented Jun 12, 2018

therepo90 commented Jul 19, 2018

mscudlik commented Jul 31, 2018

nanoTitan commented Aug 10, 2018

nikdo commented Aug 16, 2018

Danieltreiber commented Sep 11, 2018 • edited Loading

nikdo commented Sep 12, 2018

edarioq commented Sep 23, 2018

peter-snr commented Oct 15, 2018 • edited Loading

james-schwartzkopf commented Oct 15, 2018

IonelLupu commented Dec 10, 2018

cport1 commented Dec 10, 2018

IonelLupu commented Dec 11, 2018

avoerman commented Dec 13, 2018

IonelLupu commented Dec 14, 2018

mgechev commented Dec 14, 2018

marthinus-engelbrecht commented Apr 13, 2019 • edited Loading

angular-automatic-lock-bot bot commented Sep 9, 2019

jbandi commented May 7, 2018 •

edited

Loading

jbandi commented May 8, 2018 •

edited

Loading

buster commented May 9, 2018 •

edited

Loading

maprox commented May 18, 2018 •

edited

Loading

Danieltreiber commented Sep 11, 2018 •

edited

Loading

peter-snr commented Oct 15, 2018 •

edited

Loading

marthinus-engelbrecht commented Apr 13, 2019 •

edited

Loading