test: add End-2-End test suite

[yannickcr]

Summary

Same as algolia/instantsearch#4035 but for react-instantsearch.

This PR makes the following changes:

• Add algolia/instantsearch-e2e-tests as devDependency.
• Add 2 WebdriverIO configurations: one to run tests locally and one to run them on Sauce Labs.
• Add .env file to .gitignore so you can use it to store Sauce Labs credentials (see https://github.com/algolia/instantsearch-e2e-tests#credentials ).
• Add npm scripts to run End-2-End tests:
  • test:e2e: Alias for test:e2e:local.
  • test:e2e:local: Run the test suite on Chrome browser on your local machine.
  • test:e2e:saucelabs: Run the test suite on multiple browsers on the Sauce Labs service.
• Rename existing test:e2e to test:integration.
• Update CircleCI configuration
  • Rename test_e2e task to test_integration.
  • Add the new test_e2e task.

[Haroenv]

Looks promising overall & passes! All there’s to say is it seems some build steps happen multiple time, and that should be looked into

[tkrugg]

I think I asked this question already but I'm no longer sure what's the answer.
Is it not possible to hide this away in instantsearch-e2e-tests and just expose a command there?

[yannickcr]

See my answer: algolia/instantsearch#4035 (comment)
(it can be discussed of course 😉)

[Haroenv]

I also think we should hide this

[francoischalifour]

It would also make sense to me.

[samouss]

Agree.

[yannickcr]

Well... I'm not for this.

First of all there is very few advantages (if any?) to "hide" dependencies but the real problem is mainly that it will makes us call a subdependency from the parent project (react-instantsearch calls wdio directly), which is not recommenced (the package version we're targeting may not be at the node_modules top level, depending how npm/yarn deduplicated the dependencies).

[Haroenv]

can we also hide wdio call then?

[yannickcr]

What would be the objective of hiding these dependencies?

[francoischalifour]

It would remove the overall perceived complexity, as well as reduce the noise and cognitive overhead of managing these dependencies.

I find it surprising that we went all the way to packaging the E2E tests (e.g. see also wdio.saucelabs.conf.js which only exports the E2E package import), but are reluctant to hide the dependencies themself.

The E2E test suite would be more predictable if the wdio dependencies are packaged within it. Here, we are responsible for the wdio deps to work with a black-boxed test suite, which is quite contradictory.

[francoischalifour]

It would also make sense to me.