bug: jest cli - no longer able to run individual unit tests

[Kwooda]

Stencil version:

 @stencil/core@2.14.0

Current behavior:
CLI no longer allows running individual unit tests. We updated from v2.6.0 to v2.14.0 and have lost the ability to run individual unit tests from the command line.

Expected behavior:
Running npx stencil test --e2e --spec component -- where component is the name of an individual unit test, should run just that test. Instead, it runs the entire test suite.

GitHub Reproduction Link:

Other information:

[rwaskiewicz]

Hey @Kwooda, this sound similar to #3275, but perhaps slightly different. Could you create a minimal reproduction repo so that I can make sure I understand the issue and add the correct test cases here?

[Kwooda]

Not sure how that's related. We have a bunch of unit tests, and would like to run just one of them. We used to be able to do that by just running npx stencil test --e2e --spec {testName} and it would run just that test. But after upgrading Stencil, any attempt to run a unit test runs all the unit tests. I'll try to find some time to set up a repo, but it seems easy enough to test by just trying to run a single unit test.

[rwaskiewicz]

Thanks, appreciate it! (marked as 'Awaiting Reply' for now while OP gets a reproduction case together for us)

[Kwooda]

I have created a repo to demonstrate the issue. Details are in the readme.
https://github.com/Kwooda/StencilUnitTests

[rwaskiewicz]

Thanks @Kwooda! I was able to use the reproduction case to determine this behavior started to occur in v2.13.0 - I'm going to label this issue so that it gets ingested into the team's internal backlog. From there we'll be able to prioritize it and look into it further. Thanks again!

[alicewriteswrongs]

Hey @Kwooda, sorry for the wait on this issue! We've diagnosed the issue and are working on a fix. In the meantime, with the reproduction you kindly provided we were able to come up with a workaround that should let you run a single test file while we come up with a more robust fix for the problem.

If you change the test script in package.json to omit the --coverage flag, so that it just looks like this:

  "test": "stencil test --spec --e2e",

then you should be able to do npm test my-component to run the specfile for that component. You could also try that out by doing npx stencil test --spec my-component.

The issue relates to how stencil test processes arguments before it passes them along to Jest. Basically, any flag-type argument (like --coverage) will currently be passed along to Jest, but arguments are being parsed incorrectly in the case that a test file pattern (like my-component in the example above) is passed as well.

So you should be able to continue using the --coverage flag alone, but until we come up with a fix it won't work in combination with a file pattern.

One suggestion could be to add a test.coverage script that looks like this:

  "test.coverage": "stencil test --spec --e2e --coverage",

and keep npm test more narrowly concerned with just running the tests.

Hopefully that unblocks you for the time being on running a single spec file from the command line, and we'll be sure to ping here when the more complete fix is in and released!

[Kwooda]

Thanks, Alice, that is very helpful! I tried your suggestion and it works, but I am having an issue where it runs just the test I specify, but then it stalls for a minute or two before returning to the command prompt. I am wondering if maybe there is a timeout I should look for or something? Thanks <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Virus-free. www.avg.com <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

…

On Fri, Apr 29, 2022 at 12:52 PM Alice ***@***.***> wrote: Hey @Kwooda <https://github.com/Kwooda>, sorry for the wait on this issue! We've diagnosed the issue and are working on a fix. In the meantime, with the reproduction you kindly provided we were able to come up with a workaround that should let you run a single test file while we come up with a more robust fix for the problem. If you change the test script in package.json to omit the --coverage flag, so that it just looks like this: "test": "stencil test --spec --e2e", then you should be able to do npm test my-component to run the specfile for that component. You could also try that out by doing npx stencil test --spec my-component. The issue relates to how stencil test processes arguments before it passes them along to Jest. Basically, any flag-type argument (like --coverage) will currently be passed along to Jest, but arguments are being parsed incorrectly in the case that a test file pattern (like my-component in the example above) is passed as well. So you should be able to continue using the --coverage flag alone, but until we come up with a fix it won't work in combination with a file pattern. One suggestion could be to add a test.coverage script that looks like this: "test.coverage": "stencil test --spec --e2e --coverage", and keep npm test more narrowly concerned with just running the tests. Hopefully that unblocks you for the time being on running a single spec file from the command line, and we'll be sure to ping here when the more complete fix is in and released! — Reply to this email directly, view it on GitHub <#3276 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2JD4VL2RZCK2KF4F25T4DVHQ4XFANCNFSM5QNW5U3Q> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[alicewriteswrongs]

Glad it can unblock your testing workflow!

As for Jest not exiting right away, there are potentially a few things to look at. I suspect that it's not a stencil issue per se because I'm not seeing that on a small component library, running stencil test --e2e --spec component-name. I usually see roughly a half second or so of time between the last thing that Jest prints to the console and the process actually exiting.

Jest does have some functionality built-in to help diagnose problems like these. The Jest documentation suggests trying out --detectOpenHandles to try to figure out what might be preventing a test from existing (note that for the same reason that --coverage won't work correctly with a file pattern right now --detectOpenHandles will also not work correctly with a file pattern). If that doesn't provide anything useful I'm not sure, my guess would be that if Jest isn't exiting right away it's most likely due to some async code that isn't exiting correctly, a timer, or something of that nature.

[theo-staizen]

I just updated to latest stencil and run into the --coverage bug as well, glad its already on your radar.

I tried your suggestion and it works, but I am having an issue where it runs just the test I specify, but then it stalls for a minute or two before returning to the command prompt. I am wondering if maybe there is a timeout I should look for or something?

@Kwooda I ran into that issue as well but it was only happening for tests that were using fake timers. It seems like Jest 27 updated their timer implementation (among a lot of other things).

To fix it I had to change jest.useFakeTimers() to jest.useFakeTimers('legacy');

Hope this helps

[rwaskiewicz]

I'm going to close this issue out. This has been fixed in #3444 and is a part of the 2.17.1 release. If the issue does not appear to be resolved, please feel free to open a new ticket. Thanks!

[nghiango]

Just realized that we can coverage one component with adding no-cache.

npx stencil test --spec --coverage --no-cache <component-name>

Noted: --no-cache need to be placed in the last argument to make it work. Otherwise, it doesn't work.

[Kwooda]

I just upgraded from Stencil 2.22.3 to 4.0.5 and this problem has returned.