feat: decouple Jest from Stencil

[trekzavier]

Prerequisites

• I have read the Contributing Guidelines.
• I agree to follow the Code of Conduct.
• I have searched for existing issues that already include this feature request, without success.

Describe the Feature Request

Decoupling Jest from the dependency chain and just have consumers install jest (as a peer dep) and run the jest command directly on stencil files.

Describe the Use Case

There have been multiple Jest-related issues that pop up (#2942 being latest example). #2980 attempts to fix it, but it's likely more jest-related interoperability issues will occur in the future as the Jest codebase evolves.

Describe Preferred Solution

Removing jest and having consumers install and run it themselves would give the following benefits:

1. Consumers will have more flexibility to configure/run jest how they want
2. Consumers can opt out of jest in favor of more web-component friendly test runners (e.g. Web Test Runner)
3. Will save Stencil maintainers time that otherwise is spent constantly fixing jest issues 😛
4. Stencil-specific documentation and config around jest can be removed in favor of the official Jest documentation

Describe Alternatives

No response

Related Code

No response

Additional Information

No response

[splitinfinities]

so, this is interesting... There are a few things to consider around how best to get to a place like this for Stencil. Right now, it would take planning around how an integration would work in a more quality way that's external to Stencil.

Next, what we would need to ensure that if you DO have Jest (or other tools), the Stencil team wouldn't generally support any issues you may have with that external tool (especially as versioning can be exceptionally tricky). It may free up our capacity, however that may also lead to increased support tickets around many, many more tools that we would have highly reduced support and capacity for.

Next, Jest is highly coupled to Stencil's internal tooling, so the technical planning we would have to do is important to note here.

Finally, we'd need a better docs site to help us tell a story for people who want to use Jest, or Web Test Runner, or Cypress, etc., etc.

All that to say, we wouldn't be able to do this in the next Major release, but if there's enough community signal, I'll be happy to start putting a plan together around unbundling these for a Major release after a future v3.

So if you're landing on this issue, and you agree with it, please react to it! I sort issues by 👍 to help better understand what the community wants.

Edit: clarifying thoughts

[wbern]

I think I want to add that, Jest is becoming quite a mainstream tool. Maybe it's not the philosophy of StencilJs to externalize things since you want to give people an easy way to start developing, but given that you've got some really good things going on inside StencilJs, it would perhaps make it easier for everyone if you didn't bundle in Jest (and had to support that as well).

Hopefully this could give you time to improve other areas.

At the very least, it would be great if we could configure Jest (modularize it) more than we are able to now.

Edit: Oh, perhaps we do have good control over Jest.

https://stenciljs.com/docs/testing-config

The testing config setting specifies an object that corresponds to the jest configuration that should be used in your tests. Stencil provides a default configuration, which you likely won't need to edit, however it can be extended with the same configuration options as Jest. See the Configuring Jest Guide for configuration details.

[markcellus]

Oh, perhaps we do have good control over Jest.

https://stenciljs.com/docs/testing-config

Yeah but AFAICT that doesn't support all of Jest's options, unfortunately. And I believe it depends on what version of Jest you have installed.

There's a mismatching that currently can happen where, if the Jest version used in your project isn't the same as the Jest version Stencil expects, it causes issues like the ones mentioned in OP. This can be avoided if Jest wasn't coupled with Stencil's codebase.

[rwaskiewicz]

This issue has picked up quite a bit of 👍 's, so I want to open this up for some discussion.

if you're in favor of decoupling Jest & Stencil, what do you imagine your setup looking like? Specifically:

1. What libraries are you using for your unit tests? Specifically what test runners, assertion libraries, etc. are you using?

2. Similarly, what libraries are you using for your e2e tests, if any?

3. If you've attempted to use an unsupported version of Jest in the past, what was stopping you from adopting it in full?

4. If you've attempted to use any other testing library (e2e or unit), what blockers did you run into?

5. If Stencil required folks to setup the transpilation of TypeScript & Stencil components on their own (instead of doing this it itself), would you be comfortable doing so?

[johnjenkins]

I always think with these kinds of things, you risk not realising there might be a silent majority that’s happy with the status quo 😄.
I’m personally quite happy with the testing integrations stencil offers rn - love how easy it is to be up and running and everything is very configurable.
Areas I’ve found lacking are 1) MockDoc in general (but that isn’t jest’s fault) and 2) an inability to keep a session open between e2e tests

[wbern]

This issue has picked up quite a bit of 👍 's, so I want to open this up for some discussion.

if you're in favor of decoupling Jest & Stencil, what do you imagine your setup looking like? Specifically:

1. What libraries are you using for your unit tests? Specifically what test runners, assertion libraries, etc. are you using?

Jest, with "testing-library" occasionally. https://testing-library.com/docs/vue-testing-library/examples/

2. Similarly, what libraries are you using for your e2e tests, if any?

Cypress

3. If you've attempted to use an unsupported version of Jest in the past, what was stopping you from adopting it in full?

N/a

4. If you've attempted to use any other testing library (e2e or unit), what blockers did you run into?

N/a

5. If Stencil required folks to setup the transpilation of TypeScript & Stencil components on their own (instead of doing this it itself), would you be comfortable doing so?

Cypress feels very streamlined, same with jest. They don't even require configs for most basic needs.

What I want to add additionally, is that in my org, our appreciation for stencil does not come from its extended black box setup, but rather the wc support. We only use it for components, for the framework agnostic benefits. We know it helps us support web components and it does it really well (the wrapper for react for example is a godsend). Having to test things through stencil is usually a little awkward because we have test setups for our apps already, so extending to stencil would have made things more clear for us.

Puppeteer also feels like it's falling out of favor a bit, given how well cypress is progressing.

So yeah, most people in my org like stencil for its core features, but the part where it tries to be a bit like vue cli or create react app, is probably helpful to some, but ultimate we'd like to eject from things that we've already reached advanced levels of using ourselves.

Thanks for reading and opening up the discussion.

[markcellus]

Thanks. Our org doesn't use Stencil for E2E tests. We use webdriver IO.

But how are E2E tests a part of the discussion? I thought the ask is to just allow jest (for unit testing) to be replaced entirely or installed and ran on it's own to test stencil (web) components. 🤔

[rwaskiewicz]

But how are E2E tests a part of the discussion? I thought the ask is to just allow jest (for unit testing) to be replaced entirely or installed and ran on it's own to test stencil (web) components. 🤔

Great question - the reason I bring e2e tests into the discussion is to take a step back and consider Stencil-testing from a slightly higher level. The level of coupling that Puppeteer has with the Stencil codebase is similar to the level of coupling Jest + Stencil has. If we were move forward with some type of decoupling of Jest & Stencil, I'd like to take a more holistic approach & consider Puppeteer at the same time.

[markcellus]

The level of coupling that Puppeteer has with the Stencil codebase is similar to the level of coupling Jest + Stencil has.

If Puppeteer is coupled to Stencil in such a way, and that coupling is dependent on the coupling of Jest and impacting this issue, this is even more concerning than I thought. 😬

Stencil just compiles down to Web Components. So I'm having trouble understanding why all these testing dependencies are necessary. It's great that Stencil attempts to make testing easier, but there are already great alternatives for that.

I can't help but to think all of these mixing of dependencies is causing a level of complexity in the dependency graph that isn't needed.

I still don't quite understand why Stencil attempts to "mock" HTMLElements in tests, which leads to it's mock API frequently falling out of sync with the HTML Web specification (e.g. input elements not having the focus() method, missing animate() methods, no assignedElements() on slots, missing ES module support, etc).

If there are consumers who would rather Stencil do all of this mocking and test help for them, great. But can all that behind a flag or something? Then others like me can just use Stencil for its core web component functionality, without all the other unnecessary dependencies?

[rwaskiewicz]

Stencil just compiles down to Web Components. So I'm having trouble understanding why all these testing dependencies are necessary. It's great that Stencil attempts to make testing easier, but there are already great alternatives for that.

Today, Stencil takes a 'batteries included' approach for many things, testing being one of them. Although folks are required to install Jest & Puppeteer themselves, the configuration itself is handled by Stencil. A comment that's consistent throughout user interviews we've done is an appreciation for not having to "spend time configuring the tooling". In order to provide that functionality, Stencil (today) requires these dependencies. That doesn't mean this can't change in the future, it's just the way it was originally written & designed. When looking to change these types of things, we need to be mindful of both the folks that want the batteries included approach, and those that want to pick their own stack.

If there are consumers who would rather Stencil do all of this mocking and test help for them, great. But can all that behind a flag or something? Then others like me can just use Stencil for its core web component functionality, without all the other unnecessary dependencies?

Adding a feature flag is certainly an option/route we may take. If we were to do something like that I imagine it would be a flag for testing as a whole (rather than a separate flag for Jest + Puppeteer) to minimize the complexity of supporting folks who never opt out, opt out of Jest, opt out of Puppeteer, or opt out of both.

[markcellus]

So sounds like much of this is all, understandably, just legacy cruft. 👍

I'm in favor of keeping the "batteries included" approach as default and just allow a less bloated approach behind a flag. I can see that getting quite messy to support, but totally your call. To mitigate, it might make more sense to move the core component code into a seperate repo that this "batteries included" approach depends on. Then folks like me can just use that.

[johncrim]

We currently run jest directly (not through the stencil CLI), because it integrates nicely that way with our other jest tests (we use jest workspaces and subprojects for angular jest tests and stencil jest tests). This is pretty easy to do today using a jest config like the following:

#jest.config.js
const { pathsToModuleNameMapper } = require('ts-jest');
const { paths } = require('./libs/web-components/tsconfig.spec.json').compilerOptions;

/** @type {import('ts-jest/dist/types').InitialOptionsTsJest} */
module.exports = {
  displayName: 'WebComponents Tests',
  testRunner: 'jest-jasmine2',
  preset: "@stencil/core/testing",
  roots: ['<rootDir>/libs/web-components'],
  setupFilesAfterEnv: [
    '<rootDir>/my-setup.ts',
    'jest-extended/all'
  ],
  moduleNameMapper: pathsToModuleNameMapper(paths, { prefix: '<rootDir>' }),
  transformIgnorePatterns: [
    'node_modules/(?!.*\\.mjs$)'
  ],
  transform: {
    '^.+\\.(ts|tsx|jsx|css|mjs)$': '@stencil/core/testing/jestPreprocessor.js',
  },
  cacheDirectory: './.jest-cache'
};

I know that one can include pretty much any jest config in stencil config within the testing: {} property, but there can be issues with translation, so I think it would be good to document this approach in the docs and consider it supported. This approach provides a nice "escape hatch" for cases when the jest dependency or code in stencil is not in sync with what is being used; and it also is easier to use for debugging jest tests; and it can be used today.

We used this approach when we were using Jest 27 and it wasn't supported yet by stencil - we wrapped the jestPreprocessor.js with our own script which did the arguments fixup.

The above is independent of whether stencil core depends on jest; breaking dependencies is a separate issue:

I also really like the batteries-included approach for most projects, but I can see how it would be nice to not have jest dependencies in @stencil/core and provide a way that other test frameworks could be integrated (stencil with vitest anyone?). To this end, it would be nice to have a plugin model for the stencil CLI, and move the jest integration to a @stencil/testing-jest dependency. That way the batteries-included experience could remain, and there would be a clear migration path for existing projects, but also a way to replace (or mix) jest with other test frameworks.

[daotoad]

The batteries included approach is great, it was definitely a draw for me.

However, the approach is not free of downsides. For example the requirement for a very old version of pupeteer (v10 when v14 is current) is causing all sorts of security audit alerts from npm audit.

In an ideal world, the puppeteer integration would be managed through some sort of plugin interface and another library could be dropped in to provide the same services. Of course that's "just a simple matter of programming" 😬 and it may not be feasible to add any time in the foreseeable future.

Even so, just getting a dependency update would be very helpful.

[nachogarcia]

For us, it would be great to be able to use Cypress Component testing as we would like to use it even for the development phase.