Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check aligned in TestLabelParam webdriver test #562

Merged
merged 3 commits into from
Sep 18, 2018

Conversation

lukebjerring
Copy link
Contributor

Description

Our integration tests should cover the full launch checklist.

This PR adds coverage for:

  • /results?label=stable&aligned
  • /interop?label=stable&aligned

Checks that the SHAs of the loaded test runs match.

Review Information

make go_firefox_test USE_FRAME_BUFFER=false FIREFOX_PATH=~/browsers/firefox/firefox FLAGS='-test.run=^TestLabel'

and

make go_firefox_test STAGING=true USE_FRAME_BUFFER=false FIREFOX_PATH=~/browsers/firefox/firefox FLAGS='-test.run=^TestLabel'

Copy link
Member

@Hexcles Hexcles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Hexcles
Copy link
Member

Hexcles commented Sep 18, 2018

Tests are failing:

e.g. Chrome: panic: no such element: no such element: Unable to locate element: {"method":"css selector","selector":"wpt-results"}

Firefox also fails a seemingly unrelated test: panic: no such element: Unable to locate element: input.query.

I think Firefox has started to (or will soon) ship native shadow DOM, so we'll need to drop the special case for Firefox when finding shadow elements, but I'm not sure if this is causing the failures.

@lukebjerring
Copy link
Contributor Author

At a guess, the selenium node package might have bumped their driver versions. FF62 seems to at least pass locally on my machine (both in docker + out of docker). Chrome passes fine outside docker, but not in docker. Guessing that's because my version of the driver on my box is older (ChromeDriver 2.40.565383).

@lukebjerring
Copy link
Contributor Author

FF seems happy - which means it's flaky. Which isn't newly caused by this PR. I'll merge as-is.

@lukebjerring lukebjerring merged commit 4b8b321 into master Sep 18, 2018
@lukebjerring lukebjerring deleted the integration-checklist branch September 18, 2018 20:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants