ng e2e --watch support

[zackarychapple]

Currently e2e tests run once. It would be fantastic if the e2e suite could run with a watch flag like the unit tests.

[JohannesRudolph]

I was wondering the same thing, especially since --watch appears ng e2e --help. However, it doesn't appear to work for me (using 1.0.0-rc1).

[victornoel]

@JohannesHoppe it is because the current --watch option is related to the building of the app and not to the execution of the tests.

I agree we should have a way to re-run tests when file change though!

[filipesilva]

This isn't something we're looking at adding. It sounds good on paper but the only e2e suite that could benefit from it would be a tiny one.

[JohannesHoppe]

I don't get that argument. During development I always fdescribe / fit my test, no matter if it's Karma or Protractor. A watch mode would be super​ handy here.

[filipesilva]

I just timed how long it takes from the end of a build to the end of the e2e run in ng e2e on a new project, and it was 5 seconds total for 1 spec.

Is this an acceptable refresh rate for you? I'd expect most e2e specs to take longer than this.

I'm open to reopening this issue and mark it for community help. With a bit of effort in managing the serve rebuilds and protractor process it's doable.

I just honestly can't imagine this being truly useful. But there's no e2e watch mode in other setups that I know of, so maybe it's actually super cool and nobody every bothered doing it.

[JohannesHoppe]

I agree, you should save the file with care! I still think this feature would be cool! 😄

[josersleal]

I think that it would be useful to be able to run and re-test only the modified, or better, the relevant change affected parts, rather than re-run the entire thing which takes forever in dev world.
This would allow for a bdd flow closer to tdd flow... and if bdd is the daughter of tdd then it makes sense imo

[JohannesHoppe]

Try out the methods fdescribe (instead of describe) and/or fit (instead of it) for exactly that.

[rafinskipg]

We agree on that. Running e2e tests is slowwwwwwwwwwwwwwwwwwww and launching the suite each time is not how i imagined i would die.

👴

[dapriett]

+1 for watch support.

One thing that helped for me is to serve the app separately, then just run the test like so ng e2e -s false

[gviligvili]

@dapriett Wow, thanks, I've been looking for a way to do so.
Also asking for watch support :)

[barnash]

I tried to implement it for the past 2 hours. Unfortunately I couldn't see an easy way of doing it without changing protractor itself.

The thing is that the ng e2e command runs the protractor function in order to run the tests. protractor calls process.exit when it's done, so there is no way to run any code after it is finished (you can't run it in a loop, or watch files and stop it), when protractor decides to exit, it kills our process as well (since it's the same process).

I couldn't find a way to run protractor in a way it won't run process.exit, so I also though about another solution - running it in a separate process. It seemed a bit flaky and difficult for me, so for now, I'm not sure doing it internally in ng e2e as a flag will fit the rest of the design.

[filipesilva]

Heya @barnash, thanks for having a go at it. Thanks very good info that you've gathered about the current limitations. Perhaps some changes in Protractor are needed to support this after all.

@dapriett that's a good way to cut down on the serve time especially if you're already testing in the browser. Be ware of the live reloads though, they can interrupt the protractor tests.

[spy4x]

Based on @dapriett's message and npm watch tool I found workaround:

1. $ npm install --save-dev watch - Installs watch tool to watch file changes
2. Add "e2e:watch": "watch 'ng e2e -s false' src e2e --wait=1" to package.json scripts section
3. Run $ npm start to serve app
4. Run $ npm run e2e:watch
5. Profit :)

Just implemented that in one of my projects: https://github.com/spy4x/read-that-later/commit/4faf0cbb41b5e0500f277afda015528c35d977d7

[piotros]

Solution provided by @spy4x works as a charm.

I just want to add that Windows users have to use:
"e2e:watch": "watch \"ng e2e -s false\" src e2e --wait=1"
instead of:
"e2e:watch": "watch 'ng e2e -s false' src e2e --wait=1"

[glebmachine]

What is weird that developer should use workaround like that : )

[anodynos]

This is still unsolved :-) Kudos @spy4x, it worked great like this
"e2e:watch": "watch 'ng serve --environment=e2e & ng e2e -s false' src e2e --wait=1"

[intellix]

We've just started implementing e2e tests and I'm really surprised this doesn't exist :) How do people write them without getting them correct the first time?

I've got a registration form that I need to submit to verify that it gets to a summary page. Created a populateForm function that will loop through each of the fields populating them so I can do something like this in my test:

import { RegisterPage } from './register.po';

fdescribe('RegisterPage', () => {
  let page: RegisterPage;

  beforeEach(() => {
    page = new RegisterPage();
  });

  fit('should take user to summary step after completing form', () => {
    page.navigateTo();
    page.populateForm();
    expect(page.getSummary()).toBeDefined();
  });
});

My problem is that: I've never used protractor before in my life and getting this thing to work is gonna take probably 200 runs of ng e2e before I can verify that my populateForm functionality even works.

In my mind, I would just use fdescribe/fit and run e2e tests on watch mode. Every code change to my po.ts file would re-run the test and I would observe that I'm writing the thing correctly.

As it stands, a single run of ng e2e takes 55 seconds to first compile/serve and then there's the overhead of unzipping Chrome, starting it before it even gets to my test. It looks like it'll take me a good few hours of ripping out my hair just to figure out how to fill out a single form correctly.

Perhaps there's a way to run protractor manually in the Chrome console so I can just go to the register page and hammer it out until it works?

It's not that I want to run a 10 minute suite on every change, it's that I want to run a single test trial and error until it works without 1 minute overhead

@spy4x solution reduces overhead of the serve (55sec) but you still have to wait for Chrome to startup on each run. Would be nice if the overheard was the same as going from one it() to another

[spy4x]

@intellix I can't remember exactly how much time tests on my project took, but I think I made them about 11s for several pages. You can check what I did here: https://github.com/spy4x/read-that-later/tree/master/e2e
Beware, code is pretty old (Angular v2.4 is used there) and I didn't work on it for a long time, but still could be helpful (I hope).

[gkalpak]

FWIW, this is what I ended up using with cli@6.0.5 (where the --serve option has been replaced by --dev-server-target):

# Install dependencies:
npm install --save-dev concurrently watch 

// In package.json:
"scripts": {
  ...
  "e2e-watch": "concurrently --kill-others \"npm start\" \"npm run e2e-watch-no-serve\"",
  "e2e-watch-no-serve": "watch \"npm run e2e -- --dev-server-target=\" src e2e --wait=1"
}

[Juansasa]

@filipesilva
I think this feature is doable, we did something similar a few years ago with gulp.

The idea is to compile and deploy the source code on a code-server and watch for changes in non-spec files.

The tests runs on a separate server pointing to the deployed code-server and watching for changes in e2e-spec files and only rerun the tests only not trigger a complete rebuild as it is right now.

If you guys can sync the code-server rebuild and test rerun everything should work as expected.

[asmith2306]

Bumping. Would love to see watch mode made available for e2e tests.

[JanneHarju]

FWIW, this is what I ended up using with cli@6.0.5 (where the --serve option has been replaced by --dev-server-target):

# Install dependencies:
npm install --save-dev concurrently watch 

// In package.json:
"scripts": {
  ...
  "e2e-watch": "concurrently --kill-others \"npm start\" \"npm run e2e-watch-no-serve\"",
  "e2e-watch-no-serve": "watch \"npm run e2e -- --dev-server-target=\" src e2e --wait=1"
}

I refine this a bit because this solution will start test immediately after you run command and before ng serve is running so it will fail at first. OK you can save one file and then it will success, but better UX will be get if we wait that ng serve is up and running. Here is my solution for that. I combine wait-on into watch.

"e2e-watch": "concurrently --kill-others \"npm run start-project\" \"npm run e2e-watch-no-serve\"",
"e2e-watch-no-serve": "wait-on http-get://localhost:4200 && watch \"npm run e2e -- --dev-server-target=\" projects/project-e2e --wait=1",

remember to run npm install --save-dev wait-on

[tuzmusic]

@gkalpak and @spy4x I'm having trouble running your scripts. I modified it to fit my file structure (replacing src with app and e2e with e2e-tests but bash isn't finding many of the commands:

$ npm run e2e:watch

> recipes-angular@0.0.0 e2e:watch /Users/TuzMacbookPro2017/Development/code/AngularJS/recipe-angular-from-phonecat
> watch 'ng e2e -s false' app e2e-tests --wait=1

> Watching app
> Watching e2e-tests
sh: ng: command not found

And for the other script, after stopping my server in another terminal:

npm run e2e-watch

> recipes-angular@0.0.0 e2e-watch /Users/TuzMacbookPro2017/Development/code/AngularJS/recipe-angular-from-phonecat
> concurrently --kill-others "npm start" "npm run e2e-watch-no-serve"

[1] 
[1] > recipes-angular@0.0.0 e2e-watch-no-serve /Users/TuzMacbookPro2017/Development/code/AngularJS/recipe-angular-from-phonecat
[1] > watch "npm run e2e -- --dev-server-target=" app e2e-tests --wait=1
[1] 
[0] 
[0] > recipes-angular@0.0.0 prestart /Users/TuzMacbookPro2017/Development/code/AngularJS/recipe-angular-from-phonecat
[0] > npm install
[0] 
[1] > Watching app
[1] > Watching e2e-tests
[1] npm ERR! missing script: e2e
[1] 
[1] npm ERR! A complete log of this run can be found in:
[1] npm ERR!     /Users/TuzMacbookPro2017/.npm/_logs/2019-05-24T22_06_26_045Z-debug.log
[0] 
[0] > recipes-angular@0.0.0 postinstall /Users/TuzMacbookPro2017/Development/code/AngularJS/recipe-angular-from-phonecat
[0] > npm run copy-libs
[0] 
[0] 
[0] > recipes-angular@0.0.0 copy-libs /Users/TuzMacbookPro2017/Development/code/AngularJS/recipe-angular-from-phonecat
[0] > cpx 'node_modules/{angular,angular-*,bootstrap/dist}/**/*' app/lib -C
[0] 
[0] audited 3885 packages in 3.872s
[0] found 6 vulnerabilities (2 low, 1 moderate, 3 high)
[0]   run `npm audit fix` to fix them, or `npm audit` for details
[0] 
[0] > recipes-angular@0.0.0 start /Users/TuzMacbookPro2017/Development/code/AngularJS/recipe-angular-from-phonecat
[0] > watch-http-server ./app -p 8000 -c-1
[0] 
[0] Websocket Server Listening on Port: 8086
[0] Starting up http-server, serving ./app on: http://0.0.0.0:8000
[0] Hit CTRL-C to stop the server
[0] Scanned working directory. ready for changes..
[1] npm ERR! missing script: e2e
[1] 
[1] npm ERR! A complete log of this run can be found in:
[1] npm ERR!     /Users/TuzMacbookPro2017/.npm/_logs/2019-05-24T22_06_31_048Z-debug.log

Any ideas on how to get your awesome scripts to work?

[gkalpak]

@tuzmusic, based on the output and the code paths, you seem to be inside an AngularJS (1.x) project.
This repo (and the commands mentioned above) is related to Angular (2+), which is quite different.

[tuzmusic]

Ok thanks @gkalpak!

[glebmachine]

I'm migrated to cypress and its awesome, recommend!

Cypress itself: https://github.com/cypress-io/cypress
ng-builder to support in cli: https://www.npmjs.com/package/@nrwl/builders

[tuzmusic]

Thanks for the suggestion! I was just about to start using Cypress for testing my React apps as well!

[bmontgom]

This issue was opened almost three years ago, several Angular versions ago, and it still is not fixed? Any real reason or has the Angular team just not gotten around to it yet?

[ayyazzafar]

Based on @dapriett's message and npm watch tool I found workaround:

1. $ npm install --save-dev watch - Installs watch tool to watch file changes
2. Add "e2e:watch": "watch 'ng e2e -s false' src e2e --wait=1" to package.json scripts section
3. Run $ npm start to serve app
4. Run $ npm run e2e:watch
5. Profit :)

Just implemented that in one of my projects: spy4x/read-that-later@4faf0cb

In Angular 8+ it would be bit different like this:
watch 'ng e2e --devServerTarget' src e2e --wait=1

[red2678]

This ticket was opened about four years ago. From the comments here and the other issues opened that have been merged with this one, it is clear this would be beneficial to many users (small test suite or large). Will this ever be considered? If not, why not just close the issue then? If it is being considered, why not update us?

[doggy8088]

Here are my steps:

1. Install watch package locally.

   npm install --save-dev watch

2. Add the following snippet to scripts section in the package.json file.

   "e2e:watch": "watch 'ng e2e --dev-server-target= --webdriver-update=false' src e2e --wait=1"

3. Update WebDriver (only run once)

   npx webdriver-manager update

4. Execute npm start

5. Execute npm run e2e:watch

[IanGrainger]

It'd be great to have an option to start after the ng serve has finished restarting (or on e2e/src changes).

What I'd really love is a way to watch for the dev server restart, then restart the protractor tests immediately.

[PedroFerr]

No wonder React is strongly taking over Angular.... in all fronts!

[alan-agius4]

Closing as this is not in the Angular CLI roadmap, since Protractor is being deprecated angular/protractor#5502

[angular-automatic-lock-bot]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}