Travis CI fails on browser-stack tests:client

[johnjbarton]

So the post-commit tests (running off the master fork) fail on Travis.

The browser-stack tests:client only run on the master fork. That is a combination of the line

const TRAVIS_WITH_BS = !!process.env.BROWSER_STACK_ACCESS_KEY

https://github.com/karma-runner/karma/blob/master/test/client/karma.conf.js#L3
and the setting in the Travis config that marks the key as master-branch only.

But there is a copy that is marked for travis branch only, so pushing to that branch should trigger browserstack without walking on master.

The failures on Travis are timeouts; the browserstack viewer shows karma in the browser with just "Karma-Starting" on the page.

I will try to run with DEBUG logging and concurrency 1. Could this be a socket.io issue?

[devoto13]

In case we can't figure it out quickly and given the long history of problems we had with BrowserStack, maybe we should consider switching client tests to SauceLabs instead 🤔

[johnjbarton]

The issues with BrowserStack began with #3586, the socket-io update. Seems like the VM based browsers are failing.

We need to run the test:client exactly the same on pre and post commit for at least the one nodejs version. The will prevent us from committing changes that fail only one post commit.

[johnjbarton]

Ok I will try a saucelabs config tomorrow.

[karmarunnerbot]

🎉 This issue has been resolved in version 6.0.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

[XhmikosR]

After updating to v6.0.0 we are getting Safari failures ourselves https://github.com/twbs/bootstrap/runs/1707340240. Going back to v5.x works again.

Sounds like it's the same issue? Although, we are still using karma-browserstack-launcher 1.4.0 because the newer ones has other issues I couldn't solve.