Consider using playwright?

[jtpio]

Opening a new issue to discuss potential ideas for the next iterations.

I've been trying out playwright lately and it's been really nice. It's also great that chromium, firefox and webkit are all supported by default.

Since the playwright api is very similar to puppeteer (and the same for the 3 browser), maybe it's worth considering it in the future?

The change would most likely be about updating imports, and maybe also about simplifying selectors. The galata cli could also be extended to configure browsers, so they could be set as matrix items on CI.

[willingc]

I've also been using Cypress as well.

[fcollonval]

I have been using Cypress and playwright (more recently). The latter is definitely better in term of webbrowser support (for instance Cypress does not support webkit - cypress-io/cypress#6422). Another advantage seen is a more pertinent selector when using the code generation feature of playwright than the one proposed by cypress when building a test case.

On the other hand the tools provided by Cypress are more advanced in term of UI for designing and debugging the tests - although playwright is progressing on that front with the introduction of the inspector in the latest version 1.9.0.

Another feature I like from playwright is the support of Python because I have never seen on the JS side a test framework as powerful as pytest.

[echarles]

For the jupyterlab benchmarks, we have been happy users of playwright

https://github.com/jupyterlab/benchmarks/blob/ae6dece2a5615b1965312699c37c6b518e41e17b/packages/run/src/index.ts#L8

[jtpio]

JupyterLab Classic also uses Playwright to do end to end testing with multiple browser tabs (opening notebooks in new tabs), and to be able to record videos: jupyterlab/retrolab#96

[bollwyvl]

Welp, I can't claim any experience with any of the above-mentioned tools. I have suffered through a number of only-works-for-this-library-in-this-language-in-this-browser tools in the past, which has soured me on JS-based approaches.

I'm not trying to sell anything, but when I do try to sell automated system testing to a multi-group project, i highlight these points:

• if the test can be described and agreed upon before a complex feature is even written, more stakeholders have agency
  • the report needs to come back in a way that closely matches the description
• the less the test description changes before the feature ships, while keeping the ability to rapidly agree to changes, increases engagement
• being able to robustly automate things other than browsers is important, e.g.
  • Verify that the last downloaded opens in excel without warnings
  • Start a virtual machine with MyApplication.ova
• once a team really hits its stride, The Next Test should often need to be a row in a (schema constrained) data file
• the installation profile of the toolchain needs to be highly predictable and portable
  • separating the system-under-test from the test environment from the instrumentation
  • it needs to work on the worst thing your project touches (e.g. a raspberry pi, special snowflake windows xp machine that runs microscope)

so after doing that... i usually end up back at robotframework with selenium, for all its warts. Turns out one can have playwright, as well, but i haven't investigated it.

[mbektas]

Thanks for the input everyone! I am experimenting with Playwright in the PR https://github.com/mbektasbbg/galata/pull/11 . Basic tests are working fine as before. I will try more advanced test cases.