Integration Test Audit 1 - Feb 2020

[b4handjr]

I'll be going through the current automation plan for the integration tests, as well as the CI configuration and a few other things. Please add any thoughts here.

Works on #2221

[b4handjr]

Docs are being added - #2317

[b4handjr]

Changes:

• The tests can be run in parallel on CI
• Headless Firefox option should only be set for CI.

[b4handjr]

@jaredlockhart Anything else here?

[jaredlockhart]

• Can we move the tests dir into the app dir?
  • Flake8 and black everything (free if you move it into app)
• Should this use fill_overview fixture?
  • https://github.com/mozilla/experimenter/blob/master/tests/integration/test_create_experiment.py#L8-L17
  • etc repeated
• We added a class name to body element that should be used as the wait locator for all pages
  • https://github.com/mozilla/experimenter/blob/master/tests/integration/pages/base.py#L17-L22
• All CSS locators should be constants?
  • https://github.com/mozilla/experimenter/blob/master/tests/integration/pages/base.py#L30
• Use direct class/ids rather than structural queries
  • https://github.com/mozilla/experimenter/blob/master/tests/integration/pages/experiment_design.py#L22
  • Exception: this is fine https://github.com/mozilla/experimenter/blob/master/tests/integration/pages/experiment_timeline_and_population.py#L75

[b4handjr]

All CSS locators should be constants?

Some of these are chained off of other elements. Would you still want these to be added as a constant?
https://github.com/mozilla/experimenter/blob/master/app/tests/integration/pages/experiment_timeline_and_population.py#L71

[jaredlockhart]

Yeah I don't see why not?

[jaredlockhart]

Some pages define a page wait locator and don't use it:

https://github.com/mozilla/experimenter/blob/master/app/tests/integration/pages/experiment_detail.py#L13

https://github.com/mozilla/experimenter/blob/master/app/tests/integration/pages/experiment_objective_and_analysis.py#L18

https://github.com/mozilla/experimenter/blob/master/app/tests/integration/pages/home.py#L9

https://github.com/mozilla/experimenter/blob/master/app/tests/integration/pages/owned_experiments.py#L11

https://github.com/mozilla/experimenter/blob/master/app/tests/integration/pages/subscribed_experiments.py#L11

Some pages define a page wait locator and use the wrong locator:

https://github.com/mozilla/experimenter/blob/master/app/tests/integration/pages/experiment_design.py#L21-L27

Some pages don't define a page wait locator at all:

https://github.com/mozilla/experimenter/blob/master/app/tests/integration/pages/experiment_overview.py

Should be a class var:

https://github.com/mozilla/experimenter/blob/master/app/tests/integration/pages/experiment_timeline_and_population.py#L32

[b4handjr]

Should be a class var:

https://github.com/mozilla/experimenter/blob/master/app/tests/integration/pages/experiment_timeline_and_population.py#L32

I don't quite follow on this one.

[jaredlockhart]

Oh nevermind that I think I fixed it already in another PR

[b4handjr]

Closed in #2454