feat: replace visual test driver to jest-image-snapshot and Firefox (Playwright)

[tujoworker]

Because jest-screenshot is not really maintained anymore, and it lacks Node.js v18 (and v16) support and does still not work with Node.js v16 on A1 chips – we need another approach.

jest-image-snapshot looks to be a good fit. It supports:

• Node.js v18
• jest watch CLI command to update outdated snapshots on the fly
• jest.retryTimes
• and it's maintained.

The only downside with jest-image-snapshot was; that it has not that many parameters to tweak for better results. So I went down the rabbit hole to find a better solution when it comes to the local/CI consistency – and I found it with Firefox via Playwright, made by Microsoft.

Firefox gave much more consistent results than Chromium.

So the PR also includes the switch from Puppeteer to Playwright. The latter supports even Webkit (Safari). So its a very interesting package to have in your tool-chain.

Details about the chromium / puppeteer rabbit hole:

I tried these versions of puppeteer: 8 to 15 + 19

Version 8, 9 and 10 did work fine. While version 10.2.0 has a change in its method of calculating page offsets. Here is it better explained.

This change leads to a negative side effect so snapshots differ suddenly always (yea, when scrolling) from when run on CI macOS vs local macOS. A typical comparison looks then as so:

I also see that "jest-image-snapshot" has not so many "parameters" to tweak. We basically have failureThreshold. Before we had several more.

[codesandbox-ci]

This pull request is automatically built and testable in CodeSandbox.

To see build info of the built libraries, click here or the icon next to each commit SHA.

Latest deployment of this branch, based on commit 0d6fbcb:

Sandbox	Source
eufemia-starter	Configuration

[langz]

Let's have a look/talk about this when I get back to work, would want to learn more 🙏 🧠

[tujoworker]

@langz Sure, I will tell you everything.

The PR description and code got changed very much since your comment.
I ensured to have clean commits, so you may primarily review these, commit by commit – because thousand files where changed 😅

And because there where so many deletions, you may filter them out:

[langz]

When running yarn test:screenshots:update a number of times locally on my machine(with threshold 0.01), it sometimes(randomly/every now and then) generates a few updated screenshots and errors:

Updated screenshots:

Errors:

 FAIL   browser: firefox  src/components/autocomplete/__tests__/Autocomplete.screenshot.test.ts (87.278 s)
  ● Autocomplete › have to match autocomplete opened list

    elementHandle.click: Element is not attached to the DOM
    =========================== logs ===========================
    attempting click action
      waiting for element to be visible, enabled and stable
      element is visible, enabled and stable
      scrolling into view if needed
      done scrolling
    ============================================================

      371 |
      372 |       case 'click': {
    > 373 |         await elementToSimulate.click()
          |                                 ^
      374 |         break
      375 |       }
      376 |

      at click (src/core/jest/jestSetupScreenshots.js:373:33)
      at makeScreenshot (src/core/jest/jestSetupScreenshots.js:97:50)
      at Object.<anonymous> (src/components/autocomplete/__tests__/Autocomplete.screenshot.test.ts:70:24)

[langz]

🖼️ 🥳

[langz]

Ref our discussion IRL, just before you left work today, about the errors appearing for the AutoComplete.

I ran yarn test:screenshots Autocomplete with threshold 0.01, and got 0 errors.
I ran yarn test:screenshots:update Autocomplete with threshold 0.01, and got 0 errors.

But still, when running yarn test:screenshots:update with threshold 0.01 I get the following error(same as reported earlier):

 FAIL   browser: firefox  src/components/autocomplete/__tests__/Autocomplete.screenshot.test.ts (75.534 s)
  ● Autocomplete › have to match autocomplete opened list

    elementHandle.click: Element is not attached to the DOM
    =========================== logs ===========================
    attempting click action
      waiting for element to be visible, enabled and stable
      element is visible, enabled and stable
      scrolling into view if needed
      done scrolling
    ============================================================

      371 |
      372 |       case 'click': {
    > 373 |         await elementToSimulate.click()
          |                                 ^
      374 |         break
      375 |       }
      376 |

      at click (src/core/jest/jestSetupScreenshots.js:373:33)
      at makeScreenshot (src/core/jest/jestSetupScreenshots.js:97:50)
      at Object.<anonymous> (src/components/autocomplete/__tests__/Autocomplete.screenshot.test.ts:70:24)

I'm not sure if this is critical, I've just reported what I've experienced/seen locally 😎
It might just be an issue on my machine, without A1 chip, and if so, then we can probably merge this PR, and fix/improve this for me when I get back next week 06.02.2023.

UPDATE: It seems to be random, I've now experience one "run" of yarn test:screenshots:update where I don't get any errors 🤔
UPDATE2: I ran it again, now I got the error again 🤡