BLT & Visual Regression Strategy

[justinlevi]

We currently have phpUnit and behat tests integrated into the BLT project. I'd love to see a default recommended visual regression strategy also included with BLT. Selfishly, and I think it would be good for Acquia too, I'd like to see Acquia Cloud and ACSF environments addressed first. I think having a starting point for integrating Visual Regression tests with BLT could be really helpful.

Pantheon has a good blog post about using Wraith
https://pantheon.io/docs/guides/visual-diff-with-wraith/

About a year ago I investigated Webdriverio and webdrivercss and casperjs but for some reason I feel like there were some conflicts/issues there...
https://github.com/webdriverio/webdrivercss

Integration w/ Browsershots might also be advantageous.

The motivation behind an "official" approach is that the entire space seems very unapproachable for most teams. Creating a spike, even if it's not the final approach, could create a launching point for some really interesting and useful conversations.

---

As I see it, the acceptance criteria for a feature like this would include the ability for the developer to easily run visual regression tests locally prior to committing a PR. During the automated build and deployment process it would be great to have these tests run as well.

I realize that there are quite a few different workflows and strategies for visual regression testing but having a baseline would be really helpful IMHO.

Here's an older thread I started over on the Drupal-VM github issue queue as well:
geerlingguy/drupal-vm#421

[grasmash]

Paging @jenter and @sarahjean :)

[justinlevi]

I am more than happy to work on this as well, would just like to get some thoughts before I dive in.

[sarahjean]

My main complaint with Wraith is the ruby dependency. I think it'd be better to look into a setup with PhantomCSS: https://www.phase2technology.com/blog/css-testing-with-phantomcss-phantomjs-casperjs-and-grunt/

[justinlevi]

Thoughts on BackstopJS?
https://github.com/garris/BackstopJS

[chrowe]

That looks like a good option to me.
Others are using it

• https://www.metaltoad.com/blog/visual-regression-testing-backstopjs
• https://www.youtube.com/watch?v=D63FWeYhISU

It says "BackstopJS may be just the thing if you develop custom WordPress, Drupal or other CMS templates. Tested on OSX."

[grasmash]

I think we should look at adding this to Cog rather than BLT.

Cog already manages dependencies with NPM. BLT does not, and it seems like a heavyweight addition. It could also conflict with node based requirements for project themes.

We should then more formally integrate Cog with BLT.

[grasmash]

I'm going to close this. Happy to support integration with Cog, but not willing to add any NPM dependencies to BLT itself at this time.