-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove Travis stability checks (Chrome Dev + Firefox Nightly) #14033
Conversation
I'm sending this PR now because these stability checks create seemingly endless trouble, just today on a bunch of Chromium export PRs and also on #13951. Some of the problems it catches are real, but enough are just timeouts and the like that I find myself distrusting the checks and not wanting to ask anyone else to look into failures either. Disabling the checks will make it harder to verify that the Taskcluster equivalent is working, and at this point that is the main value of these checks. Perhaps we're at a point where we can flip things around to make the Taskcluster checks blocking and keep running Travis in a non-blocking way for a while to verify? |
We need to make the TC checks actually fail when there's a problem. |
I’m not sure what this PR is waiting on, but short of merging this, is there any reason not to go ahead and update .travis.yml to move the stability checks to being allow_failures cases? |
Sent #14096 instead. |
e110902
to
64fc92c
Compare
I've reopened this PR, but it looks like Chrome Dev infrastructure tests have begun failing. Probably caused by #14456. |
@jugglinmike, can any information be extracted by comparing the results in the period where this was an allowed failure? I've seen way more PRs blocked on flakiness lately, so I strongly suspect the new setup is just better at finding flakiness. If nobody suspects otherwise I think it's fine to not spend much time looking for cases where Travis failed but Taskcluster passed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Though we should probably wait for more feedback than just from me. 🙂
It hasn't been used in Travis CI since #14033.
It hasn't been used in Travis CI since #14033.
…nd, a=testonly Automatic update from web-platform-tests Remove the `./wpt check-stability` command (#15012) It hasn't been used in Travis CI since web-platform-tests/wpt#14033. -- wpt-commits: 12527c2a641a1f439a86762c60a47293b6c05272 wpt-pr: 15012
…nd, a=testonly Automatic update from web-platform-tests Remove the `./wpt check-stability` command (#15012) It hasn't been used in Travis CI since web-platform-tests/wpt#14033. -- wpt-commits: 12527c2a641a1f439a86762c60a47293b6c05272 wpt-pr: 15012
…nd, a=testonly Automatic update from web-platform-tests Remove the `./wpt check-stability` command (#15012) It hasn't been used in Travis CI since web-platform-tests/wpt#14033. -- wpt-commits: 12527c2a641a1f439a86762c60a47293b6c05272 wpt-pr: 15012
…nd, a=testonly Automatic update from web-platform-tests Remove the `./wpt check-stability` command (#15012) It hasn't been used in Travis CI since web-platform-tests/wpt#14033. -- wpt-commits: 12527c2a641a1f439a86762c60a47293b6c05272 wpt-pr: 15012
…nd, a=testonly Automatic update from web-platform-tests Remove the `./wpt check-stability` command (#15012) It hasn't been used in Travis CI since web-platform-tests/wpt#14033. -- wpt-commits: 12527c2a641a1f439a86762c60a47293b6c05272 wpt-pr: 15012 UltraBlame original commit: 4a3904dbedda6c0c6bea55b848482845f53f176c
…nd, a=testonly Automatic update from web-platform-tests Remove the `./wpt check-stability` command (#15012) It hasn't been used in Travis CI since web-platform-tests/wpt#14033. -- wpt-commits: 12527c2a641a1f439a86762c60a47293b6c05272 wpt-pr: 15012 UltraBlame original commit: e3f658283e19b091018735595068acc503d40449
…nd, a=testonly Automatic update from web-platform-tests Remove the `./wpt check-stability` command (#15012) It hasn't been used in Travis CI since web-platform-tests/wpt#14033. -- wpt-commits: 12527c2a641a1f439a86762c60a47293b6c05272 wpt-pr: 15012 UltraBlame original commit: 4a3904dbedda6c0c6bea55b848482845f53f176c
…nd, a=testonly Automatic update from web-platform-tests Remove the `./wpt check-stability` command (#15012) It hasn't been used in Travis CI since web-platform-tests/wpt#14033. -- wpt-commits: 12527c2a641a1f439a86762c60a47293b6c05272 wpt-pr: 15012 UltraBlame original commit: e3f658283e19b091018735595068acc503d40449
…nd, a=testonly Automatic update from web-platform-tests Remove the `./wpt check-stability` command (#15012) It hasn't been used in Travis CI since web-platform-tests/wpt#14033. -- wpt-commits: 12527c2a641a1f439a86762c60a47293b6c05272 wpt-pr: 15012 UltraBlame original commit: 4a3904dbedda6c0c6bea55b848482845f53f176c
…nd, a=testonly Automatic update from web-platform-tests Remove the `./wpt check-stability` command (#15012) It hasn't been used in Travis CI since web-platform-tests/wpt#14033. -- wpt-commits: 12527c2a641a1f439a86762c60a47293b6c05272 wpt-pr: 15012 UltraBlame original commit: e3f658283e19b091018735595068acc503d40449
Fixes #7660.