Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upgrade Puppeteer to version 23.1.1 #18590

Merged
merged 1 commit into from
Aug 29, 2024
Merged

Upgrade Puppeteer to version 23.1.1 #18590

merged 1 commit into from
Aug 29, 2024

Conversation

timvandermeij
Copy link
Contributor

@timvandermeij timvandermeij commented Aug 9, 2024 •
edited
Loading

This major version contains three breaking changes that impact us:

  • The product option has been renamed to the more suitable browser.
  • The page.screenshot() API returns a Uint8Array instead of a Buffer, but since pngjs requires a Buffer object we need to do the conversion using Buffer.from() before passing data to pngjs.
  • The browser configuration should be set using a configuration file instead of environment variables. Note that as a bonus this allows us to remove the cross-env dependency since that was only used to set the Puppeteer environment variable equally for all operating systems.

For more information about the changes between the old and new Puppeteer versions refer to https://github.com/puppeteer/puppeteer/releases.

@timvandermeij
Copy link
Contributor Author

/botio test

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Linux m4)

Received

Command cmd_test from @timvandermeij received. Current queue size: 0

Live output at: http://54.241.84.105:8877/01125f9d6003662/output.txt

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Windows)

Received

Command cmd_test from @timvandermeij received. Current queue size: 0

Live output at: http://54.193.163.58:8877/a7956a157009551/output.txt

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Linux m4)

Failed

Full output at http://54.241.84.105:8877/01125f9d6003662/output.txt

Total script time: 30.20 mins

  • Unit tests: Passed
  • Integration Tests: Passed
  • Regression tests: FAILED
  different ref/snapshot: 280
  different first/second rendering: 2

Image differences available at: http://54.241.84.105:8877/01125f9d6003662/reftest-analyzer.html#web=eq.log

@Snuffleupagus
Copy link
Collaborator

Moreover, Puppeteer downloads/uses Firefox stable by default now instead of Firefox nightly. This is possible now that the WebDriver BiDi protocol implementation is finalized in Firefox stable, and this should also improve the overall stability of the tests (but it might cause some movement in the reference images).

Do we actually want to test in Firefox Release though, since by using Nightly we might be able to catch regressions before they reach users!?

/cc @calixteman

@calixteman
Copy link
Contributor

Moreover, Puppeteer downloads/uses Firefox stable by default now instead of Firefox nightly. This is possible now that the WebDriver BiDi protocol implementation is finalized in Firefox stable, and this should also improve the overall stability of the tests (but it might cause some movement in the reference images).

Do we actually want to test in Firefox Release though, since by using Nightly we might be able to catch regressions before they reach users!?

/cc @calixteman

I'm not sure it makes sense to test with release.
AFAIK, right now we test only with the first nightly of the cycle and then we always use the same during the cycle. So we're more or less testing with beta.
I'd a way prefer using nightly everyday in order to avoid any bad issues when releasing pdfjs in m-c (it's rare but it happened).

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/a7956a157009551/output.txt

Total script time: 44.42 mins

  • Unit tests: FAILED
  • Integration Tests: FAILED
  • Regression tests: FAILED
  different ref/snapshot: 261

Image differences available at: http://54.193.163.58:8877/a7956a157009551/reftest-analyzer.html#web=eq.log

@timvandermeij
Copy link
Contributor Author

Thank you for the feedback! I have pushed a new version of the patch, rebased on the current master, with an updated .puppeteerrc file that instructs Puppeteer to use Firefox Nightly again.

@timvandermeij
Copy link
Contributor Author

/botio test

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Linux m4)

Received

Command cmd_test from @timvandermeij received. Current queue size: 0

Live output at: http://54.241.84.105:8877/a91dfe255b3af42/output.txt

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Windows)

Received

Command cmd_test from @timvandermeij received. Current queue size: 1

Live output at: http://54.193.163.58:8877/5992bb86c9d526b/output.txt

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Linux m4)

Failed

Full output at http://54.241.84.105:8877/a91dfe255b3af42/output.txt

Total script time: 1.05 mins

  • Unit tests: FAILED
  • Integration Tests: FAILED
  • Regression tests: FAILED

Image differences available at: http://54.241.84.105:8877/a91dfe255b3af42/reftest-analyzer.html#web=eq.log

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/5992bb86c9d526b/output.txt

Total script time: 2.60 mins

  • Unit tests: FAILED
  • Integration Tests: FAILED
  • Regression tests: FAILED

Image differences available at: http://54.193.163.58:8877/5992bb86c9d526b/reftest-analyzer.html#web=eq.log

@timvandermeij
Copy link
Contributor Author

/botio test

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Linux m4)

Received

Command cmd_test from @timvandermeij received. Current queue size: 0

Live output at: http://54.241.84.105:8877/4d390aaa4ec9d87/output.txt

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Windows)

Received

Command cmd_test from @timvandermeij received. Current queue size: 0

Live output at: http://54.193.163.58:8877/efffe7052193f7c/output.txt

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Linux m4)

Failed

Full output at http://54.241.84.105:8877/4d390aaa4ec9d87/output.txt

Total script time: 0.78 mins

  • Unit tests: FAILED
  • Integration Tests: FAILED
  • Regression tests: FAILED

Image differences available at: http://54.241.84.105:8877/4d390aaa4ec9d87/reftest-analyzer.html#web=eq.log

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/efffe7052193f7c/output.txt

Total script time: 2.24 mins

  • Unit tests: FAILED
  • Integration Tests: FAILED
  • Regression tests: FAILED

Image differences available at: http://54.193.163.58:8877/efffe7052193f7c/reftest-analyzer.html#web=eq.log

@timvandermeij timvandermeij marked this pull request as draft August 10, 2024 17:30
@timvandermeij
Copy link
Contributor Author

/botio-linux unittest

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Linux m4)

Received

Command cmd_unittest from @timvandermeij received. Current queue size: 0

Live output at: http://54.241.84.105:8877/d9640a01116f79f/output.txt

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Linux m4)

Success

Full output at http://54.241.84.105:8877/d9640a01116f79f/output.txt

Total script time: 2.46 mins

  • Unit Tests: Passed

@timvandermeij
Copy link
Contributor Author

timvandermeij commented Aug 10, 2024 •
edited
Loading

I'm not sure why setting the version to nightly doesn't work on the bots because it works just fine locally and in the CI for e.g. the font tests which also run with Puppeteer. Using latest instead does seem to work as shown above and it's an alias for nightly (see https://github.com/puppeteer/puppeteer/blob/14717a7f3ec3d2012abbc4ce0e7ac00bf4375655/packages/browsers/src/browser-data/browser-data.ts#L67-L68), but given that it's a bit unsatisfying that nightly doesn't work on the bots I had a look at what's different between the CI and the bots and made mozilla/botio-files-pdfjs#46 to address it. I'm not sure if that will fix it, but even if it doesn't the cleanup and alignment with our GitHub Actions workflows can't hurt.

@timvandermeij timvandermeij mentioned this pull request Aug 15, 2024
Merged
@timvandermeij
Copy link
Contributor Author

timvandermeij commented Aug 19, 2024 •
edited
Loading

I found puppeteer/puppeteer@b2b93d8 which at least looks related (even though I can't reproduce the issue locally), so we'll wait until that's released and I'll update this PR to the newest version then.

@timvandermeij timvandermeij mentioned this pull request Aug 23, 2024
Open
@timvandermeij
Upgrade Puppeteer to version 23.1.1
54eead7 
This major version contains three breaking changes that impact us:

- The `product` option has been renamed to the more suitable `browser`.
- The `page.screenshot()` API returns a `Uint8Array` instead of a
  `Buffer`, but since `pngjs` requires a `Buffer` object we need to do
  the conversion using `Buffer.from()` before passing data to `pngjs`.
- The browser configuration should be set using a configuration file
  instead of environment variables. Note that as a bonus this allows us
  to remove the `cross-env` dependency since that was only used to set
  the Puppeteer environment variable equally for all operating systems.

For more information about the changes between the old and new Puppeteer
versions refer to https://github.com/puppeteer/puppeteer/releases.
@timvandermeij timvandermeij changed the title Upgrade Puppeteer to version 23.0.2 Upgrade Puppeteer to version 23.1.1 Aug 25, 2024
@timvandermeij
Copy link
Contributor Author

timvandermeij commented Aug 25, 2024 •
edited
Loading

I have finally been able to reproduce the bot issue seen above locally: it fails because the Puppeteer version on master populated the cache directory and this branch then thinks Firefox is already present while it should re-download it for this new major version. Purging the cache directory fixes the issue.

@calixteman Could you purge the Puppeteer cache directories (/home/ubuntu/.cache/puppeteer on the Linux bot and C:\Users\Administrator\.cache\puppeteer on the Windows bot)? I'll then trigger the tests again and his patch should be ready then.

@timvandermeij timvandermeij marked this pull request as ready for review August 25, 2024 10:54
@calixteman
Copy link
Contributor

/botio test

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Windows)

Received

Command cmd_test from @calixteman received. Current queue size: 0

Live output at: http://54.193.163.58:8877/a0a4587cf46a1a6/output.txt

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Linux m4)

Received

Command cmd_test from @calixteman received. Current queue size: 0

Live output at: http://54.241.84.105:8877/1fa333c2fc11bad/output.txt

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Linux m4)

Failed

Full output at http://54.241.84.105:8877/1fa333c2fc11bad/output.txt

Total script time: 31.28 mins

  • Unit tests: Passed
  • Integration Tests: Passed
  • Regression tests: FAILED
  different ref/snapshot: 16
  different first/second rendering: 2

Image differences available at: http://54.241.84.105:8877/1fa333c2fc11bad/reftest-analyzer.html#web=eq.log

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/a0a4587cf46a1a6/output.txt

Total script time: 46.26 mins

  • Unit tests: Passed
  • Integration Tests: FAILED
  • Regression tests: FAILED
  different ref/snapshot: 48

Image differences available at: http://54.193.163.58:8877/a0a4587cf46a1a6/reftest-analyzer.html#web=eq.log

calixteman
calixteman approved these changes Aug 26, 2024
View reviewed changes
Copy link
Contributor

@calixteman calixteman left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you.

@timvandermeij timvandermeij merged commit ea090fe into mozilla:master Aug 29, 2024
8 checks passed
@timvandermeij timvandermeij deleted the puppeteer-23 branch August 29, 2024 18:14
@YuvrajKaushal YuvrajKaushal mentioned this pull request Sep 30, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@calixteman calixteman calixteman approved these changes

Assignees
No one assigned
Labels
dependencies test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@timvandermeij @moz-tools-bot @Snuffleupagus @calixteman