Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(browser-pool): improve error handling when browser is not found #2050

Merged
merged 7 commits into from
Aug 30, 2023

Conversation

foxt451
Copy link
Collaborator

@foxt451 foxt451 commented Aug 25, 2023

Basically, just add a hint to the user that they might've forgotten to rebase their docker image/used the wrong template.
Closes #1459

…s if browser fails to launch on Apify platform
@foxt451 foxt451 self-assigned this Aug 25, 2023
@foxt451 foxt451 marked this pull request as draft August 25, 2023 11:37
@foxt451 foxt451 marked this pull request as ready for review August 25, 2023 13:22
@foxt451 foxt451 requested a review from B4nan August 25, 2023 13:22
Copy link
Member

@B4nan B4nan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you give me examples of how this looks in the logs?

private _notifyOfFailedLaunch() {
let debugMessage = `Failed to launch browser. ${this.launchOptions?.executablePath ? 'Check whether the provided executable path is correct.' : ''} `;
if (process.env.APIFY_IS_AT_HOME) {
debugMessage += 'Make sure you used the correct Actor template for Playwright: '
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its about using correct docker image, not actor template, i'd not mention the templates here at all

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think some new users and those who are not familiar with docker might not make the connection between the wrong docker image and the fact that they can fix the issue by instantiating with a new template. Maybe we could change it from Make sure to Try instantiating with that template ...? But for now I've removed the mention

@@ -74,7 +74,12 @@ export class PuppeteerPlugin extends BrowserPlugin<
}
} catch (error) {
await close();

if (process.env.APIFY_IS_AT_HOME) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it feels weird that for playwright we handle this outside of apify platform too, but here we dont. FYI puppeteer now has a similar CLI for installing the browsers https://pptr.dev/browsers-api/

not sure if its already mentioned in the error produced by puppetter?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really. But the bad thing about this command is that it installs executable into the folder where it was executed, and under a long directory hierarchy, like chromium/mac_arm/-xxxx.../../.../MacOS/Chromium, and you have to be aware of that executable location, to e.g. pass it into PUPPETEER_EXECUTABLE_PATH

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It mentions troubleshooting guide though most of the times

@foxt451
Copy link
Collaborator Author

foxt451 commented Aug 29, 2023

can you give me examples of how this looks in the logs?

Yes, the first screenshot is how it runs locally, without docker, if an executable is provided that is missing (playwright):
image
The second is when a wrong path for browsers on playwright is provided (via PLAYWRIGHT_BROWSERS_PATH=/ )
image
The third - when for playwright useChrome is true, but CRAWLEE_CHROME_EXECUTABLE_PATH=/ (locally)
image
The fourth - for puppeteer, wrong exec path, locally:
image
The next one for the command APIFY_IS_AT_HOME=true PLAYWRIGHT_BROWSERS_PATH=/empty apify run, PlaywrightCrawler:
image
The next one for Puppeteer, with this command APIFY_IS_AT_HOME=true PUPPETEER_EXECUTABLE_PATH=/empty apify run
image

@B4nan
Copy link
Member

B4nan commented Aug 29, 2023

The first screenshot looks like we retry that request? I'd say this should cause a hard failure.

We have the CriticalError class you can use, such errors will always cause the crawler to stop.

@B4nan B4nan requested a review from vladfrangu August 29, 2023 13:18
@foxt451
Copy link
Collaborator Author

foxt451 commented Aug 30, 2023

The first screenshot looks like we retry that request? I'd say this should cause a hard failure.

We have the CriticalError class you can use, such errors will always cause the crawler to stop.

I've updated it to throw CriticalError in the Puppeteer and Playwright plugins (not in BrowserPlugin, so as not to make assumptions about error handling for other possible implementations). It's a bit unpleasant though, that the error logging happens 3 times at different levels in such cases, but it still looks fine I guess:
image
image

@foxt451
Copy link
Collaborator Author

foxt451 commented Aug 30, 2023

Should now only print once, the last couple of commits disable logging of CriticalError where it is about to be rethrown anyway
image

@foxt451 foxt451 requested a review from B4nan August 30, 2023 13:47
@B4nan
Copy link
Member

B4nan commented Aug 30, 2023

Ok, that looks much better, let's see if it won't break any of the E2E tests: https://github.com/apify/crawlee/actions/runs/6026358796

@B4nan B4nan changed the title fix(browser-pool): provide hints to use the correct Docker base images if browser fails to launch on Apify platform fix(browser-pool): improve error handling when browser is not found Aug 30, 2023
@B4nan B4nan merged commit 282527f into master Aug 30, 2023
10 checks passed
@B4nan B4nan deleted the fix/browser-presence-validation branch August 30, 2023 15:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve error handling when browser is not installed (e.g. when using wrong docker image)
2 participants