bug: e2e tests failing when updating to 4.20.0

[danyball]

Prerequisites

• I have read the Contributing Guidelines.
• I agree to follow the Code of Conduct.
• I have searched for existing issues that already report this problem, without success.

Stencil Version

4.20.0

Current Behavior

We have around 700 e2e tests running with 4.19.2. After updating to 4.20.0 tests fails on our server (jenkins and gitlab ci tested). On my local Mac I can not see any errors.

• some of them "App did not load in allowed time. Please ensure the content loads a stencil application."
• some with random errors.
• the test script just stopped at a point - no more logging / gitlab stops those jobs after 1h

Expected Behavior

tests still running

System Info

Server:
- tested with several node versions, from 18 to LTS
- debian bookworm OS
- HeadlessChrome/121.0.6167.85 (with puppeteer 21.11.0 - also tested 21.0.0)
- on gitlab ci:
   - `- apt-get update && apt-get install -y libnss3 libdbus-1-3 libatk1.0-0 libx11-xcb1 libxcomposite1 libxcursor1 libxdamage1 libxi6 libxtst6 libxrandr2 libasound2 libatk-bridge2.0-0 libgtk-3-0 libcups2 libpango1.0-0 libxss1 fonts-liberation xdg-utils
`

local:
System: node 18.20.5 (LTS also working)
    Platform: darwin (24.1.0)
   CPU Model: Apple M1 Max (10 cpus)
    Compiler: /node_modules/@stencil/core/compiler/stencil.js
       Build: 1732216626
     Stencil: 4.22.3 🐤
  TypeScript: 5.5.4
      Rollup: 2.56.3
      Parse5: 7.1.2
      jQuery: 4.0.0-pre
      Terser: 5.31.1

Steps to Reproduce

• having many tests (do not find a reproducer)
• use specific stencil version
• run e2e tests on a server env

Code Reproduction URL

sorry no

Additional Information

https://discord.com/channels/520266681499779082/1304108899657318442/1304108899657318442

[christian-bromann]

Hey @danyball 👋 when loading a Stencil component using the Stencil testrunner in an e2e test case we verify that all components resolved their componentOnReady hook. If this has not happened before the config.testing.browserWaitUntil timeout is reached it fails with given error.

At this point there is not much I can do as this is very difficult to reproduce. If you have any ideas what may can cause this I would be curious to know.

[danyball]

Thanks Christian. Did you change something runtime timing relevant in 4.20?
Maybe something in

runtime: hydrate shadow dom first (#5911) (ccf1a89)
runtime: make isSameVnode return false on initial render in a hydration case (#5891) (82a7bb9)
?

Maybe the new if in "isSameVnode" could go wrong on slow machines or so?!

[christian-bromann]

@danyball potentially but I am not certain.