test coverage is empty - v3.0.0

[pete-redmond-cko]

Is this a bug report?

yes, test coverage is always empty

After updating react-scripts to 3.0.0 test coverage is always empty, this worked before in v2.1.8

---

v2.1.8

v3.0.0

---

Did you try recovering your dependencies?

This happens after booting an app up using create react app using typescript create-react-app cra-coverage-example --typescript

Which terms did you search for in User Guide?

n/a

Environment

Environment Info:

  System:
    OS: macOS 10.14.4
    CPU: x64 Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz
  Binaries:
    Node: 11.12.0 - ~/.nvm/versions/node/v11.12.0/bin/node
    Yarn: 1.15.2 - /usr/local/bin/yarn
    npm: 6.7.0 - ~/.nvm/versions/node/v11.12.0/bin/npm
  Browsers:
    Chrome: 73.0.3683.103
    Safari: 12.1
  npmPackages:
    react: ^16.8.6 => 16.8.6
    react-dom: ^16.8.6 => 16.8.6
    react-scripts: 3.0.0 => 3.0.0
  npmGlobalPackages:
    create-react-app: Not Found

Steps to Reproduce

1. create-react-app cra-coverage-example --typescript
2. yarn test --coverage

Expected Behavior

test coverage would be output

Actual Behavior

test coverage is always empty

Reproducible Demo

https://github.com/pete-redmond-cko/cra-coverage-example or you can spin up an app using create react app v.3.0.0 and run yarn test --coverage

[bugzpodder]

If you turn off watch via:
yarn test --coverage --watchAll=false

It will work properly. Looks like a Jest bug. @stipsan

[esturcke]

Thanks @bugzpodder !

This also fixed a regression we had running tests/coverage in Docker causing us to hit up against the fs.inotify.max_user_watches limit. Added --watchAll=false and now our tests complete again.

[azeezat]

If you turn off watch via:
yarn test --coverage --watchAll=false

It will work properly. Looks like a Jest bug. @stipsan

This fixed the issue

[aenciso]

If you turn off watch via:
yarn test --coverage --watchAll=false

It will work properly. Looks like a Jest bug. @stipsan

It doesn't fix it for me though. Still getting empty results

[bugzpodder]

@aenciso can you show your entire command that you ran as well as the line in scripts section of package.json that shows actual command.

[aenciso]

@aenciso can you show your entire command that you ran as well as the line in scripts section of package.json that shows actual command.

From command line I run
yarn test --coverage --watchAll=false

[bugzpodder]

I cannot repo this, please provide more details or a minimal repro demonstrating this issue.

[omguhh]

I'm having the same issue as @aenciso - I think it's a jest bug, it's been brought up as an issue there as well after the latest release. jestjs/jest#8364

I can run all the tests fine and see coverage if I'm using this command
npm t -- --coverage --watchAll=false

but running npm t -- --coverage --watch returns an empty table of results, so I can't really 'watch' for changes when I'm updating a test :(

[mraak]

Jest needs more Jest tests to test the Jest.

[hugomn]

I have the same issue and can confirm that --watchAll=false temporarily solves the issue. But it's not a good solution because I cannot use watch at all, together with the coverage.

[bugzpodder]

Can you elaborate on your last statement

[hugomn]

@bugzpodder the only way to have coverage displayed is disabling watch. So when you are working on tests, trying to fix coverage issues, you'll need to re-run all tests every time. That's why I said it's only a temporary solution, but the bug is still there.

[bugzpodder]

As a workaround, you can temporarily modify the component file (or whichever file you are trying to add coverage for) and then the coverage will show up.

[aemc]

I thought I was the only one... glad I found this!

[bugzpodder]

@aemc @hugomn did modifying the original file help with the coverage issue under --watch mode? I am trying to gauge the extend of the bug.

[hugomn]

@bugzpodder the coverage doesn't show at all under watch mode. Even changing files.

[bugzpodder]

It works fine for me, I thought it might be typescript files but that worked as well. You'll need to provide a repo.

 PASS  src/App.test.tsx (5.164s)
  ✓ renders without crashing (15ms)

-----------|----------|----------|----------|----------|-------------------|
File       |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
-----------|----------|----------|----------|----------|-------------------|
All files  |    66.67 |      100 |      100 |    66.67 |                   |
 App.tsx   |      100 |      100 |      100 |      100 |                   |
 index.tsx |        0 |      100 |      100 |        0 |                 6 |
-----------|----------|----------|----------|----------|-------------------|
Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        9.245s
Ran all test suites related to changed files.

Watch Usage
 › Press a to run all tests.
 › Press f to run only failed tests.
 › Press q to quit watch mode.
 › Press p to filter by a filename regex pattern.
 › Press t to filter by a test name regex pattern.
 › Press Enter to trigger a test run.

[MrHus]

Unfortunately the npm test -- --coverage --watchAll=false does not work for me as well.

[bugzpodder]

@MrHus please provide a repo and outputs.

[tswardenski]

@bugzpodder I don't have a repo to share because I'm experiencing this for a work app and it isn't open source, however this does appear to be a Jest 24 watch issue to me. Here's the same output from both 2.1.8 and 3.0.1:

☁  app [init] ⚡  node -v && npm ls jest && yarn test --coverage
v11.4.0
app@1.0.0 /Users/tswardenski/app
└─┬ react-scripts@2.1.8
  └── jest@23.6.0

yarn run v1.16.0
$ react-scripts test --coverage
 PASS  src/lib/utils/__tests__/flatten.spec.js
 PASS  src/lib/utils/__tests__/mcUserInfo.spec.js
 PASS  src/lib/utils/__tests__/sendEvent.spec.js
 PASS  src/lib/i18n/actions/__tests__/index.spec.js
 PASS  src/lib/i18n/reducers/__tests__/i18n.spec.js
 PASS  src/lib/i18n/containers/__tests__/Intl.spec.js
--------------------------|----------|----------|----------|----------|-------------------|
File                      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
--------------------------|----------|----------|----------|----------|-------------------|
All files                 |    17.45 |     5.48 |     10.6 |     17.5 |                   |
 src                      |      100 |      100 |      100 |      100 |                   |
  setupTests.js           |      100 |      100 |      100 |      100 |                   |
 src/lib/api              |        0 |        0 |        0 |        0 |                   |
  Endpoint.js             |        0 |        0 |        0 |        0 |... 60,62,72,76,77 |
  api.js                  |        0 |        0 |        0 |        0 |... 46,50,51,52,53 |
 src/lib/components       |        0 |      100 |      100 |        0 |                   |
  index.js                |        0 |      100 |      100 |        0 |               1,2 |
 src/lib/components/Error |        0 |      100 |        0 |        0 |                   |
  Error.js                |        0 |      100 |        0 |        0 |    1,2,4,5,7,8,22 |
 src/lib/i18n             |       75 |    58.82 |      100 |    76.92 |                   |
  I18nLoader.js           |    77.78 |    58.82 |      100 |       80 |    18,20,41,45,62 |
  index.js                |        0 |      100 |      100 |        0 |                 1 |
 src/lib/i18n/actions     |      100 |        0 |      100 |      100 |                   |
  index.js                |      100 |        0 |      100 |      100 |                 4 |
 src/lib/i18n/constants   |      100 |      100 |      100 |      100 |                   |
  index.js                |      100 |      100 |      100 |      100 |                   |
 src/lib/i18n/containers  |    14.29 |        0 |        0 |    14.29 |                   |
  Intl.js                 |    14.29 |        0 |        0 |    14.29 |... 31,40,41,68,76 |
 src/lib/i18n/reducers    |      100 |      100 |      100 |      100 |                   |
  i18n.js                 |      100 |      100 |      100 |      100 |                   |
 src/lib/static           |    13.08 |        0 |        0 |    13.18 |                   |
  mc-lib.js               |    13.08 |        0 |        0 |    13.18 |... 1530,1535,1540 |
 src/lib/utils            |       90 |    88.24 |      100 |       90 |                   |
  flatten.js              |      100 |      100 |      100 |      100 |                   |
  index.js                |      100 |      100 |      100 |      100 |                   |
  mcUserInfo.js           |    77.78 |    71.43 |      100 |    77.78 |              9,13 |
  sendEvent.js            |      100 |      100 |      100 |      100 |                   |
--------------------------|----------|----------|----------|----------|-------------------|

Test Suites: 6 passed, 6 total
Tests:       13 passed, 13 total
Snapshots:   0 total
Time:        4.313s
Ran all test suites.
✨  Done in 6.34s.

☁  app [init] ⚡  node -v && npm ls jest
v11.4.0
app@1.0.0 /Users/tswardenski/app
└─┬ react-scripts@3.0.1
  └── jest@24.7.1
☁  app [init] ⚡  yarn test --coverage
No tests found related to files changed since last commit.
Press `a` to run all tests, or run Jest with `--watchAll`.
----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |        0 |        0 |        0 |        0 |                   |
----------|----------|----------|----------|----------|-------------------|

Watch Usage
 › Press a to run all tests.
 › Press f to run only failed tests.
 › Press q to quit watch mode.
 › Press p to filter by a filename regex pattern.
 › Press t to filter by a test name regex pattern.
 › Press Enter to trigger a test run.

and when pressing a to watch all:

 PASS  src/lib/utils/__tests__/sendEvent.spec.js
 PASS  src/lib/utils/__tests__/flatten.spec.js
 PASS  src/lib/utils/__tests__/mcUserInfo.spec.js
 PASS  src/lib/i18n/actions/__tests__/index.spec.js
 PASS  src/lib/i18n/reducers/__tests__/i18n.spec.js
 PASS  src/lib/i18n/containers/__tests__/Intl.spec.js
----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |        0 |        0 |        0 |        0 |                   |
----------|----------|----------|----------|----------|-------------------|

Test Suites: 6 passed, 6 total
Tests:       13 passed, 13 total
Snapshots:   0 total
Time:        3.167s
Ran all test suites.

Watch Usage: Press w to show more.

☁  app [init] ⚡  node -v && npm ls jest && yarn test --coverage --watchAll=false
v11.4.0
app@1.0.0 /Users/tswardenski/app
└─┬ react-scripts@3.0.1
  └── jest@24.7.1

yarn run v1.16.0
$ react-scripts test --coverage --watchAll=false
 PASS  src/lib/utils/__tests__/flatten.spec.js
 PASS  src/lib/i18n/actions/__tests__/index.spec.js
 PASS  src/lib/utils/__tests__/sendEvent.spec.js
 PASS  src/lib/utils/__tests__/mcUserInfo.spec.js
 PASS  src/lib/i18n/reducers/__tests__/i18n.spec.js
 PASS  src/lib/i18n/containers/__tests__/Intl.spec.js
------------------|----------|----------|----------|----------|-------------------|
File              |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
------------------|----------|----------|----------|----------|-------------------|
All files         |    17.43 |     5.48 |     10.6 |    17.59 |                   |
 api              |        0 |        0 |        0 |        0 |                   |
  Endpoint.js     |        0 |        0 |        0 |        0 |... 60,62,72,76,77 |
  api.js          |        0 |        0 |        0 |        0 |... 46,50,51,52,53 |
 components       |        0 |        0 |        0 |        0 |                   |
  index.js        |        0 |        0 |        0 |        0 |                   |
 components/Error |        0 |      100 |        0 |        0 |                   |
  Error.js        |        0 |      100 |        0 |        0 |            7,8,22 |
 i18n             |    77.78 |    58.82 |      100 |       80 |                   |
  I18nLoader.js   |    77.78 |    58.82 |      100 |       80 |    18,20,41,45,62 |
  index.js        |        0 |        0 |        0 |        0 |                   |
 i18n/actions     |      100 |        0 |      100 |      100 |                   |
  index.js        |      100 |        0 |      100 |      100 |                 4 |
 i18n/constants   |      100 |      100 |      100 |      100 |                   |
  index.js        |      100 |      100 |      100 |      100 |                   |
 i18n/containers  |    14.29 |        0 |        0 |    14.29 |                   |
  Intl.js         |    14.29 |        0 |        0 |    14.29 |... 31,40,41,68,76 |
 i18n/reducers    |      100 |      100 |      100 |      100 |                   |
  i18n.js         |      100 |      100 |      100 |      100 |                   |
 static           |       13 |        0 |        0 |    13.18 |                   |
  mc-lib.js       |       13 |        0 |        0 |    13.18 |... 1530,1535,1540 |
 utils            |       90 |    88.24 |      100 |       90 |                   |
  flatten.js      |      100 |      100 |      100 |      100 |                   |
  index.js        |        0 |        0 |        0 |        0 |                   |
  mcUserInfo.js   |    77.78 |    71.43 |      100 |    77.78 |              9,13 |
  sendEvent.js    |      100 |      100 |      100 |      100 |                   |
------------------|----------|----------|----------|----------|-------------------|

Test Suites: 6 passed, 6 total
Tests:       13 passed, 13 total
Snapshots:   0 total
Time:        3.953s, estimated 4s
Ran all test suites.
✨  Done in 5.60s.

// package.json
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject",
    "flow": "flow"
  }

[bugzpodder]

@tswardenski thanks for the repo. There seems to be two things here:

1. when you run yarn test --coverage, it doesn't show coverage details because no test files was detected and you got the message No tests found related to files changed since last commit.
   This is generally expected behavior, because no test is run you don't get coverage info.
2. After step 1., you press 'a' to run all tests, however still there are no coverage info showing up. This looks like a bug to me. The workaround is to run it with --watchAll=false

[tswardenski]

@bugzpodder I have made changes to both specfiles and JS files between commits and run the command and while it does run the test(s) for the uncommitted changes, the coverage report still displays an empty table.

[bugzpodder]

@tswardenski can you show me the output of yarn test --coverage in this case? Thanks.

[sketchbuch]

I have the same problem (v3.0.1). I can't run coverage at all. It used to work, this is my repo: https://github.com/sketchbuch/school-report

when I run > npm run test:c

it says no tests have changed. --watchAll or --watchAll=false does nothing. Pressing "a" runs the tests but produces no coverage or even runs coverage. The coverage folder is created but contains just a few files - mostly little images used in the report display.

[bugzpodder]

@sketchbuch For npm, you'll need an extra set of dashes to get watchAll flag to work, something like: npm run test:c -- --watchAll=false, which produces result:

-----------------------------------------|----------|----------|----------|----------|-------------------|
File                                     |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
-----------------------------------------|----------|----------|----------|----------|-------------------|
All files                                |    56.75 |    47.32 |    47.87 |    57.32 |                   |
 src                                     |        0 |        0 |        0 |        0 |                   |
  index.js                               |        0 |      100 |      100 |        0 |                10 |
  registerServiceWorker.js               |        0 |        0 |        0 |        0 |... 87,91,96,97,98 |
 src/js/actions                          |    98.16 |    81.82 |    98.68 |    98.16 |                   |
  appActions.js                          |      100 |      100 |      100 |      100 |                   |
  builderActions.js                      |      100 |      100 |      100 |      100 |                   |
  categoryActions.js                     |      100 |      100 |      100 |      100 |                   |
  classActions.js                        |      100 |      100 |      100 |      100 |                   |
  dataActions.js                         |      100 |      100 |      100 |      100 |                   |

Here I've changed one file, and the coverage also works correctly:

git diff
diff --git a/src/js/actions/appActions.js b/src/js/actions/appActions.js
index 225a476..a5513a5 100644
--- a/src/js/actions/appActions.js
+++ b/src/js/actions/appActions.js
@@ -1,4 +1,5 @@
 // @flow
+console.log('foo');
 
 import { APP_ERRORED, APP_LOADED } from '../constants/actionTypes';
 import type { ActionCreator } from '../types/action';

 PASS  src/js/actions/tests/appActions.test.js
  ● Console

    console.log src/js/actions/appActions.js:2
      foo

 PASS  src/js/components/App/App.test.js
  ● Console

    console.log src/js/actions/appActions.js:2
      foo

---------------|----------|----------|----------|----------|-------------------|
File           |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
---------------|----------|----------|----------|----------|-------------------|
All files      |      100 |      100 |      100 |      100 |                   |
 appActions.js |      100 |      100 |      100 |      100 |                   |
---------------|----------|----------|----------|----------|-------------------|

Test Suites: 2 passed, 2 total
Tests:       5 passed, 5 total
Snapshots:   0 total
Time:        3.256s, estimated 6s
Ran all test suites related to changed files.

Watch Usage
 › Press a to run all tests.
 › Press f to run only failed tests.
 › Press q to quit watch mode.
 › Press p to filter by a filename regex pattern.
 › Press t to filter by a test name regex pattern.
 › Press Enter to trigger a test run.

[sketchbuch]

@bugzpodder: Thanks! that got coverage working again 👍

[tswardenski]

@bugzpodder Here you go. v3 with Jest 24.7 running yarn test --coverage

 PASS  src/lib/i18n/containers/__tests__/Intl.spec.js
  <Intl />
    ✓ should render the container without crashing (41ms)
    ✓ should render the container without crashing (5ms)

----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |        0 |        0 |        0 |        0 |                   |
----------|----------|----------|----------|----------|-------------------|
Test Suites: 1 passed, 1 total
Tests:       2 passed, 2 total
Snapshots:   0 total
Time:        3.615s
Ran all test suites related to changed files.

Watch Usage
 › Press a to run all tests.
 › Press f to run only failed tests.
 › Press q to quit watch mode.
 › Press p to filter by a filename regex pattern.
 › Press t to filter by a test name regex pattern.
 › Press Enter to trigger a test run.

[bugzpodder]

@tswardenski in your example, I just want to double check that you modified the original source code src/lib/i18n/containers/Intl.js instead of src/lib/i18n/containers/tests/Intl.spec.js?
It's a known issue that if you just modify the test file Intl.spec.js, it isn't going to show the coverage of original source code.

[tswardenski]

Ah, yes. Editing the original source code does now include the changed file in the coverage output.

[amaankulshreshtha]

Is there a way to --watch and --watchAll=false at the same time, because the above combination doesn't work

[bugzpodder]

What doesn't work? Watch and watchAll=false does the exact opposite.

[amaankulshreshtha]

Is there a way, to put it in watch mode without having the error?

[bugzpodder]

I have no idea which error you are referring to. Please provide a repro on the error you are running into.

[amaankulshreshtha]

I was facing the same issue, where the coverage was not showing anything, despite setting the collectCoverageFrom property to the absolute path of the js file.
As per your answer above: #6888 (comment), this got resolved, but I have to run the test command every time I make any changes. So is there a solution where I don't have to manually trigger the tests.

[bugzpodder]

As you yourself suggested, if you don't want to manually trigger tests you need to run jest in watch mode. Hence yarn test --coverage or npm test -- --coverage should show the coverage of a source file provided that you made some modifications to it and it gets picked up by the watch mode.

[madeleinemf]

I am still getting no coverage when running npm test -- --coverage (react-scripts v3.0.1). The workaround of running with --watchAll=false does work, but is a little annoying.

[davidnguyen11]

try this one

npm run test -- --coverage --watchAll=false

[bugzpodder]

At this point, I haven't seen a legit bug except for in a minor case (pressing 'a' in watch mode doesn't retroactive show coverage). In all cases the feature is working as expected. If you disagree, please file a new issue with a clean repro so we can investigate. Thanks!

[mwibutsa]

Thank you --watchAll worked for me