Can't run 'ng test' on all apps in a project and provide overrides

[Nxt3]

Versions

Angular CLI: 6.0.0
Node: 8.9.4
OS: darwin x64
Angular: 6.0.0
... animations, cli, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, platform-server, router

Package                            Version
------------------------------------------------------------
@angular-devkit/architect          0.6.0
@angular-devkit/build-angular      0.6.0
@angular-devkit/build-optimizer    0.6.0
@angular-devkit/core               0.6.0
@angular-devkit/schematics         0.6.0
@angular/cdk                       6.0.1
@angular/material                  6.0.1
@angular/material-moment-adapter   6.0.1
@ngtools/webpack                   6.0.0
@schematics/angular                0.6.0
@schematics/update                 0.6.0
rxjs                               6.1.0
typescript                         2.7.2
webpack                            4.6.0

Repro steps

• Have multiple apps/projects
• Run ng test --watch=false or anything that would provide an override

Observed behavior

ng test works as long as I don't provide overrides.

Before @angular/cli@6.0.0, running ng test --watch=false would work. Now you get the following error:

$ ng test --browsers=ChromeHeadless --watch=false --progress=false --sourcemaps=false

Error: Architect commands with multiple targets cannot specify overrides.'test' would be run on the following projects: app1,app2,app3
    at TestCommand.validate (/Users/user/code/angular_app/node_modules/@angular/cli/models/architect-command.js:91:23)
    at /Users/user/code/angular_app/node_modules/@angular/cli/models/command-runner.js:274:39
    at Generator.next (<anonymous>)
    at /Users/user/code/angular_app/node_modules/@angular/cli/models/command-runner.js:7:71
    at new Promise (<anonymous>)
    at __awaiter (/Users/user/code/angular_app/node_modules/@angular/cli/models/command-runner.js:3:12)
    at validateAndRunCommand (/Users/user/code/angular_app/node_modules/@angular/cli/models/command-runner.js:273:12)
    at Object.<anonymous> (/Users/user/code/angular_app/node_modules/@angular/cli/models/command-runner.js:100:26)
    at Generator.next (<anonymous>)
    at fulfilled (/Users/user/code/angular_app/node_modules/@angular/cli/models/command-runner.js:4:58)

Desired behavior

I should be able to provide overrides on ng test regardless of the number of projects I'm running it on. This worked prior to @angular/cli@6.0.0.

[clydin]

ng test now supports running test across multiple projects. Currently the name of the project needs to be specified to allow the use of additional options. This will be expanded in the future to provide for more flexibility.

ng test [project_name] --browsers=ChromeHeadless --watch=false --progress=false --sourcemaps=false

[Nxt3]

Just to make sure I'm understanding you--I can't run ng test on multiple projects simultaneously. Correct?

[avbentem]

Couldn't defaultProject from angular.json be used to select a single project if none is specified?

(Like WebStorm 2018.2 EAP still fails to run tests, due to the above error, where as the current 2018.1.3 throws "Error: The '@angular-devkit/build-angular/plugins/karma' karma plugin is meant to be used from within Angular CLI and will not work correctly outside of it." due to #10703.)

@Nxt3 I'm not sure if this is what you're asking, but for me, with 3 projects in angular.json, running ng test wil run the tests for all 3 of them. However, not simultaneously but rather sequentially: the next starts after I terminate one by hitting Ctrl+C, due to autoWatch: true in my karma.conf.js.

(But trying something like ng test --watch=false will throw the error you mentioned.)

[rdnscr]

@clydin have you tested that with latest versions of angular/cli? because i have a multiproject workspace and for me it seems like in this case command line arguments are not properly propagated thorugh cli.

ng test [project_name] --browsers=ChromeHeadless --watch=false --progress=false --sourcemaps=false

This line isn't working at all for me, although it would indeed be how i expected it to work. In my case the tests of the given project ([project_name]) are properly executed but all the additional parameters are ignored and the defaults are used instead.

Thanks for letting me know if it works for someone in a multiproject workspace.

[Itrulia]

Is there a way to disable that? I am overriding the ng test command via the builders.json, and I don't want ng test to run it for all projects (my jest.config already specifies that)

[johncrim]

Normally "me too" comments aren't real helpful, I'll try for an exception:

I'm seeing this same problem using the beta of @angular/cli 7.0.0, so it hasn't been accidentally fixed:

Angular CLI: 7.0.0-beta.2
Node: 10.8.0
OS: win32 x64
Angular: 7.0.0-beta.5
... animations, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

Package                            Version
------------------------------------------------------------
@angular-devkit/architect          0.9.0-beta.2
@angular-devkit/build-angular      0.9.0-beta.2
@angular-devkit/build-ng-packagr   0.9.0-beta.2
@angular-devkit/build-optimizer    0.9.0-beta.2
@angular-devkit/build-webpack      0.9.0-beta.2
@angular-devkit/core               0.9.0-beta.2
@angular-devkit/schematics         0.9.0-beta.2
@angular/cdk                       7.0.0-beta.0
@angular/cli                       7.0.0-beta.2
@angular/material                  7.0.0-beta.0
@ngtools/json-schema               1.1.0
@ngtools/webpack                   7.0.0-beta.2
@schematics/angular                0.9.0-beta.2
@schematics/update                 0.9.0-beta.2
ng-packagr                         4.1.1
rxjs                               6.3.2
typescript                         3.0.3
webpack                            4.17.2

The problem is exactly as described - I have a pretty spare multi-project workspace, most of the projects haven't been updated much after they were generated from the cli. And, ng test --watch=false --browsers=ChromeHeadless runs in single-run mode and uses ChromeHeadless for the first project, and uses watch mode and Chrome for all subsequent projects. So this is a semi-blocker for CI builds.

Workarounds appear to be:

1. Run each project's tests separately (a pain for development and build configuration)
2. Try something like nrwl nx for running tests in multiple projects

[escalonn]

ng test [project_name] --browsers=ChromeHeadless --watch=false --progress=false --sourcemaps=false

This should be documented and isn't.

[PapaNappa]

Any progress on this? Running tests on all projects seems to be a nice feature. ng lint also works this way pretty flawless.

[mariohmol]

Had this issue upgrading to v7 and i'vent seen anyone talking about this...

In my case i kept the default name (dev), so i did:

ng test dev --watch=false

[Tomek6789]

Hello guys, can someone help me understand why angular/cli when running ng test is opening and compiling files for each project indywidual ??? This proces cause a lot of time that is redundant.

In my project when I move karma.conf.js, test.ts and tsconfig.spec.json to root level.
I reduce test execution time from 15 minutes to 2 minutes

If someone needs to run all test for all projects (CI for example). I really encourage you to try this approach.

[mihalcan]

@Tomek6789 can you describe the changes required?

[Tomek6789]

@mihalcan You need to create test.ts file that will gather all the test.

// Grab all your tests.
const apps = require.context('./apps', true, /\.spec\.ts$/); // path to your tests.
const libs = require.context('./libs', true, /\.spec\.ts$/); // path to yout tests.
// Load the modules.
apps.keys().map(apps);
libs.keys().map(libs);

When running test you can provide your entry-point file (test.ts) with main flag.
ng test --main=/path-to-your-file/test.ts"

Please remember to add also your new test.file in tsconfig.spec.json in files prop.
I hope this will work for your :)

[filipesilva]

On a new project that was using the versions below:

    "@angular-devkit/build-angular": "0.13.0-rc.0",
    "@angular/cli": "7.3.0-rc.0",

I ran these commands:

ng generate application another-app
ng test --watch=false

The --watch=false flag was passed correctly to both tests and they both finished:

$ ng test --watch=false
 11% building 9/9 modules 0 active29 01 2019 14:08:31.602:INFO [karma]: Karma v3.0.0 server started at http://0.0.0.0:9876/
29 01 2019 14:08:31.611:INFO [launcher]: Launching browser Chrome with unlimited concurrency
29 01 2019 14:08:31.618:INFO [launcher]: Starting browser Chrome
29 01 2019 14:08:36.394:INFO [Chrome 71.0.3578 (Windows 10 0.0.0)]: Connected on socket T2XU9h8fkA_NOHH0AAAA with id 21730359
Chrome 71.0.3578 (Windows 10 0.0.0): Executed 3 of 3 SUCCESS (0.112 secs / 0.161 secs)
TOTAL: 3 SUCCESS
TOTAL: 3 SUCCESS
 10% building 7/8 modules 1 active ...ects\another-app\src sync /\.spec\.ts$/29 01 2019 14:08:41.524:INFO [karma]: Karma v3.0.0 server started at http://0.0.0.0:9876/
29 01 2019 14:08:41.525:INFO [launcher]: Launching browser Chrome with unlimited concurrency
29 01 2019 14:08:41.529:INFO [launcher]: Starting browser Chrome
29 01 2019 14:08:45.873:INFO [Chrome 71.0.3578 (Windows 10 0.0.0)]: Connected on socket XT_MB1l0k5HjGBT6AAAB with id 375334
Chrome 71.0.3578 (Windows 10 0.0.0): Executed 3 of 3 SUCCESS (0.167 secs / 0.152 secs)
TOTAL: 3 SUCCESS
TOTAL: 3 SUCCESS

I'm pretty sure this was still broken in earlier 7.x versions, but it seems to be fixed in that one.

[Tomek6789]

Why each project in angular.json is opening browser for running tests?
Why we can not run all tests in ONE browser?

This is important whan you want to reduce test time execution.

[v-erena]

@Tomek6789 it looks like you can achieve that behaviour when using --watch. Then only one browser is opened and you see the test results for all tests. With --watch=false it does not work, though.

[Tomek6789]

@v-erena Nope that won't work. I want to run all tests from all projects on one browser and compile them onec.

If you run ng test <pojectName> you will run test for project. If you run ng test without projectName than it will execute for all projects, but still this execution will be for each project independent.

So ng test will run command from angular.json .projects["projectName"].architect.test with options provided.

But this is probably feature for angular.cli

[Itrulia]

One of the reasons why I switched to Jest (besides Jest has a nice CLI and doesn't require you to change source files to only execute selected tests).

[SWGeekPD]

if you have multiple apps than you can mention its name in the command as below.

yarn test web --code-coverage

here web is app in angular project.

[joostme]

@Tomek6789

We also wanted to execute all test of all our projects at once in one browser. Starting and stopping the browser for each library was incredibly time-consuming.

Our solution was to create a dummy project inside the angular.json. This will execute ALL tests in one single browser. Before, our tests ran around 20 minutes with 30 libraries because the browser took so long to start and stop. Now, our tests run around 1,5 minutes for everything.

You can see what we did in my commit here: joostme/ng-test-all@6e4a00a

[antipalindrome]

@joostme This is a great workaround for now, thanks! Hopefully Angular CLI fixes this issue soon though so we won't need to do all this to make it work, but I appreciate you sharing this, was a huge lifesaver for us.

[jlbates]

@joostme This is the work around I needed and the one that works best from everything I've read. Thanks my dude.

[angular-automatic-lock-bot]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}