Support for Playwright library

[VikashLoomba]

It looks like the Puppeteer team has moved on to developing Playwright (https://github.com/microsoft/playwright). Will Puppeteer-Extra be ported over or is it already compatible?

[XBeg9]

@berstend any updates here?

[brunogaspar]

The author is MIA unfortunately :\

Should we start a collaborative effort in porting this to Playwright perhaps?

I've been toying with Playwright for a few days and got some successes on some websites with Akamai but haven't dig super deep yet.

[XBeg9]

@brunogaspar agree. Lots of plugins don't work here anymore and it seems to be everything must be switched into playwright.

[VikashLoomba]

How do you guys feel about forking and starting on a playwright version? @XBeg9 @brunogaspar

[brunogaspar]

I think that's the proper way to go, at least until we get some feedback from @berstend

[brunogaspar]

I've created a fork here https://github.com/werxe/playwright-extra

Need to perform changes so it can then later be installed through NPM then going to start updating what i personally see it's needed for now.

You guys want to be added to the project?

[VikashLoomba]

Yup, I would love to help out!

…

On Apr 12, 2020, at 7:41 AM, Bruno Gaspar ***@***.***> wrote:  I've created a fork here https://github.com/werxe/playwright-extra Need to perform changes so it can then later be installed through NPM then going to start updating i personally see it's needed. You guys want to be added to the project? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[berstend]

(Apologies for the longer read and context)

I had a first look at Playwright and (being a Puppeteer fork) the API is mostly compatible. :-)

It looks like the Puppeteer team has moved on to developing Playwright (microsoft/playwright). Will Puppeteer-Extra be ported over or is it already compatible?

My understanding is that the lead switched over to MS but Puppeteer is still being actively developed by Google (they mention in their issue tracker that they're open for collaboration with Playwright though).

The main benefit of Playwright currently seems to be better browser support (firefox and especially webkit). This is not super relevant for users of our popular plugins, as e.g. stealth is optimized for chromium browsers (currently, at least).

---

Hard to tell what the future holds right now and if Playwright will completely absorb Puppeteer. Looking at the stats Puppeteer is still the behemoth in the headless space and Playwright a relatively small contender.

I'm pretty sure we can make puppeteer-extra compatible with both Puppeteer & Playwright, my concern is that eventually functionality might diverge significantly enough that maintenance of a single plugin system for both becomes painful.

On the other hand having completely separate packages (or even repos) just for Playwright doesn't make sense either as 99% of the code would be a duplicate of the Puppeteer version.

And then there's the naming question: Importing puppeteer-extra in a Playwright project isn't optimal, but publishing duplicate packages just to have a playwright- prefix isn't either. 😄 Renaming this project to something more generic is an option but nothing done lightly.

---

Is there any specific reason (except for hype 🤓) to favor Playwright over Puppeteer right now, in a project that would want to use puppeteer-extra as well? From my perspective the heavy duty headless crawling/automation space is still firmly in the hands of Chrome/Chromium. :-)

TL;DR - My stance is this:

• puppeteer-extra can be made compatible with Playwright if need be. In the event that Puppeteer dies and Playwright prevails we will obviously switch over and rename the project 😄
• Supporting both Puppeteer and Playwright comes with challenges (e.g. suddenly plugins have to expect to be used with FF and Webkit as well, this needs to be prepared and compatibility matrixes documented).
• Naming things is hard (especially if the underlying projects change or different ones are supported 😅)
• Making all that overloading and monkey-patching work with TypeScript reliably was tricky, adding more variety to the mix could be challenging

Interested to hear thoughts on this. :-)

[jozsi]

Is there any specific reason (except for hype 🤓) to favor Playwright over Puppeteer right now, in a project that would want to use puppeteer-extra as well? From my perspective the heavy duty headless crawling/automation is still firmly done with Chrome/Chromium. :-)

Yes, Playwright patches Chromium itself to fix stuff like the persistance of user-agent and viewport when opening a popup/new tab.

[berstend]

Yes, Playwright patches Chromium itself to fix stuff like the persistance of user-agent and viewport when opening a popup/new tab.

Ah true, I forgot about their custom browser patches. Hmmm. :-)
I'm curious if their custom binaries would work with pptr as well.

[jozsi]

Even if they do, there are some puppeteer workarounds that would compromise the experience, e.g. if you change the viewport size, it will reload the page - as far as I remember, that's not the case with playwright.

[berstend]

The majority of work to support Playwright is done, I basically ended up rewriting the whole project to support both Puppeteer & Playwright (+ whatever might come next) with a single underlying package (automation-extra) 😅

The progress can be tracked in #303 - the challenge is to stay backwards compatible with all existing plugins while supporting the new ones (the only way to do this is meticulous testing).

The end result will be small puppeteer-extra & playwright-extra packages that use automation-extra behind the scenes.

Plugins will be compatible with both (as well as chromium, firefox, webkit browsers), but the Plugins themselves need to be written/updated in a way to support all these things respectively (e.g. pptr has page.evaluateOnNewDocument, playwright has context.addInitScript - I'll be adding a shim to unify these APIs a little bit).

[berstend]

At long last: The release of playwright-extra is imminent, a beta version is available for testing now 😄

[berstend]

Condensed info on how to test drive the upcoming playwright-extra and new plugins can be found here: #454

[floppabro1337]

need the stealth on playwright-extra my g

[berstend]

playwright-extra with support for the stealth and recaptcha plugin is available (#664) 😄
https://github.com/berstend/puppeteer-extra/tree/master/packages/playwright-extra

Closing this issue 💪