Testing: Include IE11 "Canary" test in Travis build

[aduth]

There have been a number of occurrences[1][2][3][4][5][6][7][8] of catastrophic errors in Internet Explorer which would be immediately obvious on any attempt to load the block editor. In lieu of running the full set of end-to-end tests, we should explore a minimally viable solution to verify that the editor loads without errors and displays some expected elements on the screen.

Possible approach references:

• Travis supports a Windows build environment (reference)
• Microsoft offers a Internet Explorer WebDriver tool
• If Microsoft's tool is insufficient, we could explore using the Selenium InternetExplorerDriver
• If we cannot build our own environment for testing Internet Explorer, we could consider BrowserStack-based automation (possibly free?) or similar tooling (SauceLabs, etc)

[aduth]

Considering the future of browser support (eventually dropping Internet Explorer support), using something like BrowserStack to attempt minimal loading in a variety of browsers might be a more advisable approach.

Worth mentioning that Puppeteer support is expanding to add official support for Firefox, where we can broaden our end-to-end coverage to include this browser as well (a separate task).

[aduth]

The implementation is different than what is proposed here, but there was some improvements to the general broad goal of protecting against Internet Explorer-related issues in @tellthemachines's pull request at #18774 (simulating IE by running ESLint on the generated bundles in ES5 mode).

[tellthemachines]

Have there been any instances of this since I added the lint check? I haven't seen immediate errors on load for a while, but from time to time there are smaller issues like #21276. This wouldn't be caught if we were just checking for page load. I wonder if it would be more useful to have a few checks for basic interaction, e.g. can write text, can add blocks etc.

[aduth]

@tellthemachines I'm not personally aware of any "catastrophic failures" (white screens), but there's been a handful of smaller issues related to interaction (the one you mention, also #19979, #20598, and a few other minor ones). I agree that would be useful have testing for those interactions. At this point, since there's so little settled here as far as an approach, it would probably depend whether the solution we choose would have the means to script those sorts of interactions. The options in the original comment are all still viable, and still encompassing of what I understand to be the most promising options available. Writing some basic Selenium tests for Internet Explorer to run in Travis' Windows environment could be the most promising for what you're suggesting, perhaps?

[sainthkh]

I've created an experimental PR in #21528. I hope this solves the IE testing problem.

[gziolo]

WordPress is dropping support for IE 11 later this year so this issue is no longer relevant:

https://make.wordpress.org/core/2021/03/25/discussion-summary-dropping-support-for-ie11/

Thank you everyone for all the time spent on exploring this idea 🙇🏻