Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Karma tests disconnect when running tests on Safari #14848

Closed
Enriqe opened this issue Apr 24, 2018 · 10 comments · Fixed by #15510
Closed

Karma tests disconnect when running tests on Safari #14848

Enriqe opened this issue Apr 24, 2018 · 10 comments · Fixed by #15510
Assignees
@rsimha
Labels
P1: High Priority Related to: Flaky Tests WG: infra
Milestone
Sprint H2 April 2018

Comments

@Enriqe
Copy link
Contributor

Enriqe commented Apr 24, 2018

When running the tests on Travis for my PR #14679, I'm getting: Test disconnected because Safari 11.0.1 is not generating any test output for 2 minutes. https://travis-ci.org/ampproject/amphtml/jobs/369330113#L725 Note that it eventually reports a success in L728.

Have tried running it several times and the problems seems to persist.
@rsimha
Copy link
Contributor

rsimha commented Apr 24, 2018

I see several console errors being printed during Safari runs. Probably related? https://travis-ci.org/ampproject/amphtml/jobs/369330113#L826-L883

@rsimha
Copy link
Contributor

rsimha commented Apr 24, 2018

https://support.saucelabs.com/hc/en-us/articles/225104707-Karma-Tests-Disconnect-Particularly-When-Running-Tests-on-Safari contains several workarounds, but we already have them in place.

@rsimha
Copy link
Contributor

rsimha commented Apr 24, 2018 •
edited
Loading

Let's see if #14759 fixes this.

Specifically, this commit might improve things: karma-runner/karma@6c92019

@rsimha rsimha mentioned this issue Apr 24, 2018
Merged
@rsimha
Copy link
Contributor

rsimha commented Apr 24, 2018

Another PR that could potentially improve things is #13818

@rsimha rsimha mentioned this issue Apr 24, 2018
Merged
@rsimha
Copy link
Contributor

rsimha commented Apr 24, 2018

@Enriqe Seems like your tests were run on a version of master that's 6 days old. Can you rebase and re-run?

@Enriqe
Copy link
Contributor Author

Enriqe commented Apr 24, 2018

@rsimha sorry about that. Rebased and re-ran. Waiting for results.

@rsimha
Copy link
Contributor

rsimha commented Apr 24, 2018

Seems like this is a widespread problem on Safari, where even passing runs are idling for several minutes: https://travis-ci.org/ampproject/amphtml/jobs/370751738#L2053

@rsimha rsimha changed the title Karma tests disconnect when running tests on Safari 11.0.1 Karma tests disconnect when running tests on Safari Apr 24, 2018
@rsimha rsimha added this to the Sprint H2 April 2018 milestone Apr 24, 2018
This was referenced Apr 24, 2018
Merged
Closed
@Enriqe
Copy link
Contributor Author

Enriqe commented Apr 25, 2018

#14679 is now green. Thanks!

@rsimha rsimha mentioned this issue May 23, 2018
Merged
@rsimha rsimha mentioned this issue Jun 6, 2018
Merged
@rsimha
Copy link
Contributor

rsimha commented Jun 6, 2018

The hostname change to 127.0.0.1 is playing badly with some of our tests that use a hard-coded value for localhost.

@rsimha rsimha reopened this Jun 6, 2018
@rsimha rsimha mentioned this issue Jun 7, 2018
Closed
@cathyxz
Copy link
Contributor

cathyxz commented Jun 18, 2018

Seeing timeouts and disconnects here in https://travis-ci.org/ampproject/amphtml/jobs/393858309 and some recent failures on master this week.

@rsimha rsimha mentioned this issue Jun 22, 2018
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rsimha rsimha

Labels
P1: High Priority Related to: Flaky Tests WG: infra
Projects
None yet
Milestone
Sprint H2 April 2018
Development

Successfully merging a pull request may close this issue.

🐛 Re-enable Safari on Sauce labs
3 participants
@cathyxz @Enriqe @rsimha