Can we speed up our Nightwatch tests by running them in PhantomJS ?

[nelsonic]

On a reasonably "mature" project, e.g: https://github.com/TheScienceMuseum/collectionsonline we are running several end-to-end test scenarios: https://github.com/TheScienceMuseum/collectionsonline/tree/master/test/client
Running all of these tests takes 3 Minutes on localhost and 4.28 minutes on Travis-CI e.g: https://travis-ci.org/TheScienceMuseum/collectionsonline/builds/172018586#L3263

Can we investigate using PhantomJS instead of Chrome to reduce the amount of time that it takes to run our tests?

Starting point: https://github.com/nightwatchjs/nightwatch/wiki/Running-tests-in-PhantomJS

[SimonLab]

Time for running Nightwatch tests on SMG:

[nelsonic]

A quick read of the Wiki page: https://github.com/nightwatchjs/nightwatch/wiki/Running-tests-in-PhantomJS
indicates that we need to use: https://github.com/detro/ghostdriver for this.
This is going to require a time-boxed spike...

[jedwards1211]

Just a tip: I've been using webdriver.io with the wdio-phantomjs-service to run integration tests for awhile, it's super easy. Probably a better bet than ghostdriver.

[jedwards1211]

@nelsonic don't know if it will be a big speedup though. They're both WebKit-based browsers. But the simplicity of test setup alone justifies it in my opinion

[nelsonic]

@jedwards1211 thanks very much for reminding us about http://webdriver.io
I think the reason I didn't consider using it was because it looks "unfinished" ...

But good to hear that you are having success with it.
Our primary objective is test execution speed. 🐌 >> 🐎

[jedwards1211]

Heh, fair point...I'm confused though, ghostdriver doesn't have any badges at all ;)

[nelsonic]

@jedwards1211 yeah, the situation with ghostdriver is way worse ... 😢
which is why we are considering switching to: dwyl/learn-karma#6 😇
but... we are willing to give Nightwach a "last chance" if we can get it to run faster ... 😉

I consider nightwatch to be our will be stoked if we can move on ...! 😜

[jedwards1211]

hahaha nice 😄 But you can't do integration tests with karma, can you?
I think webdriver made the mistake of showing the build badge for master in their README, which will show error from time to time since they are developing on master. (At the moment it's passing again). They're working on it actively and I'm sure they'll improve test coverage with time.

[DevS1993]

@nelsonic (i'm like 2 years late in this thread) Have your tried running your nightwatch tests in a HEADLESS manner? the test execution will be crazy fast.