bug: Unit tests fail after update to 2.13.0

[Kwooda]

Prerequisites

• I have read the Contributing Guidelines.
• I agree to follow the Code of Conduct.
• I have searched for existing issues that already report this problem, without success.

Stencil Version

2.13.0

Current Behavior

Unit tests that ran successfully in versions prior to 2.13.0 are now failing intermittently.

Expected Behavior

Unit tests should run successfully as before.

Steps to Reproduce

I have created a repo: https://github.com/Kwooda/StencilUnitTests
This repo also exists to demonstrate a previously validated bug (#3276).
The readme describes the unit test failure under "Problem Statement #2"
Other unit tests in our suite fail similarly, failing to detect an event, or respond to mouse click or keypress, and the failures are intermittent. The one I included failed more consistently. If it doesn't fail on the first try, you may need to re-run it until it does. I'm guessing there must be an asynchronous race condition, and I don't know if the problem is in Stencil, Jest, or Puppeteer, but it is the e2e tests that fail.

Code Reproduction URL

https://github.com/Kwooda/StencilUnitTests

Additional Information

No response

[rwaskiewicz]

Hey @Kwooda 👋

Thanks for the repro! I'm not seeing "Problem Statement 2" in the README, would you mind updating it? Or adding it's content to the GitHub issue summary here? Ideally I'd love to be able to see a stacktrace/error message of what it is that's failing in your e2e tests

Can you also clarify your upgrade steps for me as well? Did you upgrade only Stencil, or a mix of Stencil + Jest + Puppeteer? If the latter, can you confirm for me the versions of Jest & Puppeteer you're coming from? (I believe you were coming from Stencil v2.6 based on this line in your README)

[Kwooda]

Sorry, I guess that didn't get pushed. Here's the text:

Problem Statement #2 - For: Unit tests fail after update to 2.13.0

In versions of StencilJS prior to 2.13.0, the unit test for the "price-story" component succeeded consistently.

After upgrading to StencilJS 2.13.0 or above, the unit test for "price-story" fails intermittently, but frequently,
when checking for a mouse click event on an icon.

[Kwooda]

Also, yesterday, I started with stencil core 2.6.0 and progressively installed 2.7, 2.8, 2.9, 2.10, 2.11, 2.12, until unit tests began to fail in 2.13.0. I only updated stencil. And in all cases, I was using the same versions of Jest and Puppeteer:

	"@types/jest": "^27.0.3",
	"@types/puppeteer": "5.4.5",
	"jest": "^27.4.5",
	"jest-cli": "^27.4.5",
	"puppeteer": "^10.0.0",
	"typescript": "4.6.2"

[rwaskiewicz]

Thanks @Kwooda,

I pulled down this reproduction and do see a test failing. I wanted to confirm two things with you:

• is it the price-story.e2e.ts or price-story.spec.ts suite that you're seeing a test fail in? I'm seeing the former, but you had mentioned the unit test (the latter) was failing for you - just want to make sure we're on the same page in that regard
  

• can you provide me a case (in a separate branch) where everything is working?

  • I want to make sure that we're on the same page with Jest/Puppeteer versions. Jest 27 didn't work with versions of Stencil prior to Stencil v2.13.0, so I'd be curious if there was an external jest configuration file/setup being used here when you were running Stencil versions 2.6 through 2.12 in your upgrade process
  • I took your existing reproduction and installed Stencil v2.12.0 (and Jest v26, as v27 doesn't work out of the box with this version). Running Jest without its cache npm t -- --no-cache also produces the same error in my screenshot above

[Kwooda]

I'll look into it and get back to you - the error is different in your screen shot than what I see locally. Also, I got pinged by the security team at work about uploading to a public repo, so I'll have to resolve that before I can update anything.

[rwaskiewicz]

Thanks, appreciate it!

[rwaskiewicz]

@Kwooda for what it's worth, if you're able to get the minimal reproduction case in the same state as the actual application (or confirm the reproduction has the same error message on your machine as what you're seeing locally as what you're seeing in the app) and provide that secondary reproduction I mentioned in #3287 (comment), that might expedite things quite a bit. Depending on the application size, we may not be able to allocate resources to diving into the application (as this often requires learning how an application is setup, sifting through details that are pertinent and those that aren't, etc) and is reserved for Stencil Enterprise customers. With the minimal reproduction cases, we probably have a better chance of figuring out the root cause in the context of our current exchange here on GitHub

[Kwooda]

Go ahead and close this out. I'm having difficulty recreating the issue consistently after updating Jest & puppeteer. I will open a new ticket if I run into any issue later on.