[Bug] stealth plugin on Firefox throw TypeError: Cannot read properties of null

[woojoo666]

Steps to reproduce:

1. in a new node project, run PUPPETEER_PRODUCT=firefox npm install puppeteer puppeteer-extra puppeteer-extra-plugin-stealth
2. run the following script:

import puppeteer from 'puppeteer-extra';
import StealthPlugin from 'puppeteer-extra-plugin-stealth';

(async function test() {
	const browser = await puppeteer
		.use(StealthPlugin())
		.launch({ product: 'firefox', headless: true });
	const page = await browser.newPage();
	await browser.close();
})()

You should get error

TypeError: Cannot read properties of null (reading '1')
    at Plugin.onPageCreated (\path\to\project\node_modules\puppeteer-extra-plugin-stealth\evasions\user-agent-override\index.js:82:62)

The issue seems to be in user-agent-override/index.js line 82

      : (await page.browser().version()).match(/\/([\d|.]+)/)[1]

I tested it and for Firefox, seems like page.browser().version() returns Firefox instead of a version number. Will need to do some further research to see how to retrieve the version number for Firefox (aside from extracting it directly from the user-agent).

[woojoo666]

Also worth mentioning that Firefox has a privacy.resistFingerprinting flag that can be set, which already handles things the stealth plugin does, eg spoofing the user agent and navigator. So maybe the stealth plugin isn't needed on Firefox anyways? I haven't looked too deeply into the evasion techniques so I'm not too sure

[berstend]

We currently don't pay much attention to Firefox while developing puppeteer-extra as support is too rudimentary for any productive workloads.

The new automation-extra branch (#454) officially (or rather more specifically) supports using Firefox through the plugin framework, but realistically it's still unusable as a couple low level APIs are just not working with FF. :-/

The situation with Playwright + FF is better though as they're using custom browser patches to bruteforce better CDP support into FF lol.

I wrote more about this in the wiki: https://github.com/berstend/puppeteer-extra/wiki/Is-Puppeteer-Firefox-ready-yet%3F

I haven't checked recently if the situation improved but I pretty much doubt it.

Closing this for now as "WONTFIX" but happy to discuss this further.

[berstend]

Firefox has a privacy.resistFingerprinting flag that can be set

An issue that might come from that: Enabling it might make the browser "stick out" more, which we generally try to avoid in scraping

[woojoo666]

sorry just saw this message, but yeah that makes sense, in fact I actually ended up running into quite a few issues with Firefox so I went back to using a Chrome scraper anyways. I was looking into the issues with resistFingerprinting as well, and I think you're right that it can make you stand out more. I'll have to keep that in mind, thanks

[spencer17x]

I encountered the same problem in chromium.