[UIE-205] Puppeteer upgrade (version 22.7.1) #5117

cahrens · 2024-09-30T15:03:08Z

Jira Ticket: https://broadworkbench.atlassian.net/browse/UIE-205

This upgrades Puppeteer to the most recent version that I can where our tests that interact with iframes still work. Unfortunately, this doesn't address the ws security issue, but it gets us very close to a version that does (22.11.2 or possible 22.9.0). I will continue to see if I can find a workaround that allows us to bring in a newer version of Puppeteer, but we might as well upgrade to this version in the meantime.

Note that starting with version 20, Chrome for Testing is used instead of Chromium.

cahrens · 2024-09-30T15:26:31Z

integration-tests/jest-circus-environment.js

@@ -2,7 +2,7 @@ const PuppeteerEnvironment = require('jest-environment-puppeteer');
 const { parse } = require('path');
 const { maybeSaveScreenshot, savePageContent } = require('./utils/integration-utils');

-class JestCircusEnvironment extends PuppeteerEnvironment {
+class JestCircusEnvironment extends PuppeteerEnvironment.default {


Puppeteer changed how this is exported.

cahrens · 2024-09-30T15:27:20Z

integration-tests/jest-puppeteer.config.js

-    // Workaround for issue when accessing cross domain iframes in puppeteer while not headless:
-    // https://github.com/GoogleChrome/puppeteer/issues/4960
-    args: (({ headfull, devtools }) => {
-      const flag = '--disable-features=site-per-process';


None of this is needed anymore-- I ran the IA analysis test locally (which interacts with an iframe) with both HEADFUL and DEVTOOLS, and it passed.

cahrens · 2024-09-30T15:28:17Z

integration-tests/jest-puppeteer.config.js

-      }
-      return [];
-    })({ headfull: process.env.HEADLESS === 'false', devtools: process.env.DEVTOOLS === 'true' }),
+    protocolTimeout: 720_000, // 12 minutes


At some point, Puppeteer decreased their default protocolTimeout. I needed to increase this for the IA analysis tests, which wait for a very long time for Jupyter or RStudio to launch and be ready to interact with.

cahrens · 2024-09-30T15:28:33Z

integration-tests/package.json

@@ -8,21 +8,21 @@
    "test-flakes": "FLAKES=true yarn test-local --runInBand"
  },
  "devDependencies": {
-    "jest": "^27.4.3",
-    "jest-environment-puppeteer": "^6.0.3",
+    "jest": "^29.7.0",


I increased these all to their most recent version.

cahrens · 2024-09-30T15:31:30Z

integration-tests/package.json

  },
  "dependencies": {
    "@axe-core/puppeteer": "^4.6.0",
    "date-fns": "^2.24.0",
    "google-auth-library": "^7.9.2",
    "lodash": "^4.17.21",
    "p-retry": "^4.6.1",
-    "puppeteer": "^13.1.3",
+    "puppeteer": "22.7.1",


This is the last Puppeteer version that works with the IA iframes (using Chrome version 124.0.6367.78). All later versions I tried failed in the same way: broken computer icon in the iframe and a message about being unable to access dev leonardo.

@jdcanas just pinging you in case you might know, but it sounds like we eventually will need to rewrite our IA UI Integration tests to be better compatible with pupeteer?

cahrens · 2024-09-30T15:32:39Z

integration-tests/tests/billing-projects.js

@@ -40,7 +40,7 @@ const billingProjectsPage = (testPage, testUrl) => {

    assertChartValue: async (number, workspaceName, category, cost) => {
      // This checks the accessible text for chart values.
-      await testPage.waitForXPath(`(//*[@role="img"])[contains(@aria-label,"${number}. Workspace ${workspaceName}, ${category}: ${cost}.")]`);
+      await testPage.waitForSelector(`[role="img"][aria-label*="${number}. Workspace ${workspaceName}, ${category}: ${cost}."]`);


waitForXPath has been removed.

cahrens · 2024-09-30T15:33:02Z

integration-tests/tests/find-workflow.js

@@ -30,14 +30,14 @@ const testFindWorkflowFn = _.flow(
     without these lines it will redirect to dev-Terra even if started out at localhost:3000-Terra */
    if (testUrl === 'http://localhost:3000') {
      await findElement(page, clickable({ textContains: 'Yes' }));
-      const yesButtonHrefDetails = (await page.$x('//a[contains(text(), "Yes")]/@href'))[0];
+      const yesButtonHrefDetails = (await page.$$('xpath///a[contains(text(), "Yes")]/@href'))[0];


Rename of this method, and you have to specify 'xpath/' at the beginning if using an xpath.

cahrens · 2024-09-30T15:33:53Z

integration-tests/utils/integration-utils.js

 };

 const waitForNoModal = (page, { timeout = 30000 } = {}) => {
-  return page.waitForXPath('//*[contains(@class, "ReactModal__Overlay")]', { hidden: true, timeout });
+  return page.waitForSelector('.ReactModal__Overlay', { hidden: true, timeout });


Simplification

cahrens · 2024-09-30T15:34:15Z

integration-tests/utils/integration-utils.js

-  return page.waitForXPath(xpath, defaultToVisibleTrue(options));
-};
-
-const findErrorPopup = (page, options) => {


findErrorPopup was unused.

cahrens · 2024-09-30T15:34:24Z

integration-tests/utils/integration-utils.js

@@ -364,25 +360,19 @@ const findInDataTableRow = (page, entityName, text) => {
  return findElement(page, elementInDataTableRow(entityName, text));
 };

-const findButtonInDialogByAriaLabel = (page, ariaLabelText) => {


LizBaldo · 2024-09-30T16:12:06Z

integration-tests/package.json

  },
  "dependencies": {
    "@axe-core/puppeteer": "^4.6.0",
    "date-fns": "^2.24.0",
    "google-auth-library": "^7.9.2",
    "lodash": "^4.17.21",
    "p-retry": "^4.6.1",
-    "puppeteer": "^13.1.3",
+    "puppeteer": "22.7.1",


@jdcanas just pinging you in case you might know, but it sounds like we eventually will need to rewrite our IA UI Integration tests to be better compatible with pupeteer?

cahrens · 2024-09-30T18:01:05Z

@LizBaldo @jdcanas Regarding Liz's comment, no, there isn't a way to rewrite the test. The issue is that the iframe itself is showing no content (just an unhappy computer icon) because the content is getting blocked. I believe it is due to specific newer versions of Chrome, and hopefully there is some flag we can pass during test startup that will prevent the issue. I just haven't figured out yet what it is… all the things I saw suggested on Stack Overflow didn't work.

chromewebdata/:1 Refused to display 'https://leonardo.dsde-dev.broadinstitute.org/' in a frame because it set 'X-Frame-Options' to 'sameorigin'.

UPDATE: this @mlilynolting @jdcanas @LizBaldo this is caused by the third-party cookie issue. Apparently newer versions of Chrome for Testing are enforcing that third-party cookies are not enabled. Is there a plan yet for how to handle this problem? I feel like there should be an option that I can pass to Chrome to disable the enforcement, but I have yet to figure it out.

cahrens added 6 commits September 27, 2024 14:37

Simplifications from Puppeteer PR.

29b46de

Update to last version that uses Chromium.

6aff1c8

Upgrade to a version that uses Chrome for Testing.

1da7bf0

Another version that works.

e2616ec

Yet another version that works.

d482f3f

Last version that works with Leo iframe.

11db55c

broadbot temporarily deployed to pr-2 September 30, 2024 15:08 Inactive

cahrens commented Sep 30, 2024

View reviewed changes

cahrens changed the title ~~Puppeteer upgrade~~ [UIE-205] Puppeteer upgrade Sep 30, 2024

cahrens changed the title ~~[UIE-205] Puppeteer upgrade~~ [UIE-205] Puppeteer upgrade (version 22.7.1) Sep 30, 2024

cahrens marked this pull request as ready for review September 30, 2024 15:42

cahrens requested review from a team as code owners September 30, 2024 15:42

cahrens requested a review from aherbst-broad September 30, 2024 15:42

aherbst-broad approved these changes Sep 30, 2024

View reviewed changes

cahrens requested a review from msilva-broad September 30, 2024 15:57

LizBaldo approved these changes Sep 30, 2024

View reviewed changes

rjohanek approved these changes Sep 30, 2024

View reviewed changes

cahrens added this pull request to the merge queue Sep 30, 2024

Merged via the queue into dev with commit 18b4897 Sep 30, 2024
10 checks passed

cahrens deleted the puppeteer-upgrade branch September 30, 2024 18:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[UIE-205] Puppeteer upgrade (version 22.7.1) #5117

[UIE-205] Puppeteer upgrade (version 22.7.1) #5117

cahrens commented Sep 30, 2024 •

edited

Loading

cahrens Sep 30, 2024

cahrens Sep 30, 2024

cahrens Sep 30, 2024

cahrens Sep 30, 2024

cahrens Sep 30, 2024

LizBaldo Sep 30, 2024

cahrens Sep 30, 2024

cahrens Sep 30, 2024

cahrens Sep 30, 2024

cahrens Sep 30, 2024

cahrens Sep 30, 2024

LizBaldo Sep 30, 2024

cahrens commented Sep 30, 2024 •

edited

Loading

[UIE-205] Puppeteer upgrade (version 22.7.1) #5117

[UIE-205] Puppeteer upgrade (version 22.7.1) #5117

Conversation

cahrens commented Sep 30, 2024 • edited Loading

Jira Ticket: https://broadworkbench.atlassian.net/browse/UIE-205

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cahrens commented Sep 30, 2024 • edited Loading

cahrens commented Sep 30, 2024 •

edited

Loading

cahrens commented Sep 30, 2024 •

edited

Loading