Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test coverage not working #8

Closed
jedateach opened this issue Mar 9, 2020 · 4 comments · Fixed by #9
Closed

Test coverage not working #8

jedateach opened this issue Mar 9, 2020 · 4 comments · Fixed by #9

Comments

@jedateach
Copy link

jedateach commented Mar 9, 2020

By instrumenting test coverage, we can confidently change code with the knowledge that the area is covered by tests. If an area isn't covered, then we'll be inclined to introduce tests before making changes. Once coverage is working, we'll be able to hook into tooling like code climate and coveralls to automatically reject PRs without sufficient tests.

Istanbul is the main tool js projects use to instrument code, such that running the instrumented code will report which lines of code have been run, and how many times.

What is tricky however is getting it to work with our current stack: Typescript + Rollup + Testem

Because I'm running tests in Testem, the coverage report needs to be sent via AJAX, as demonstrated in the Testem GitHub example.

I managed to get the three tools working together, with some level of coverage reporting given, but the line numbers are incorrect. This far from ideal.

These screenshots show that things clearly don't line up:
Screen Shot 2020-03-09 at 7 46 33 PM

A number of sources cite source maps as the problem:

I tried stripping my entire project down to just some bare essentials, and found that when I have either comments or module imports, things get out of alignment

Without comments:
Screen Shot 2020-03-09 at 7 59 32 PM

With comments:
Screen Shot 2020-03-09 at 7 50 47 PM

Any help would be appreciated

@jedateach jedateach added the help wanted Extra attention is needed label Mar 9, 2020
@jedateach jedateach changed the title Test coverage reporting Test coverage not working Mar 9, 2020
@jedateach
Copy link
Author

jedateach commented Mar 13, 2020

I've since realised that Typescript can be instrumented by Istanbul / nyc, but I ran into issues using that approach too. 😞

@jedateach
Copy link
Author

Solution to my typescript instrumentation problems may be arriving soon: istanbuljs/nyc#1292

@jedateach jedateach removed the help wanted Extra attention is needed label Apr 20, 2020
@jedateach
Copy link
Author

A few ideas have been given to do this:
istanbuljs/istanbuljs#557

However failing that, and make a compromise - an alternative strategy may be to just measure the coverage of the js that is produced by typescript. Apparanelty this is what the typescript team does.

@jedateach
Copy link
Author

Great news: a way forward has been found by nearly recreating the build from scratch.
It was easier to figure this all out using project dedicated to getting the build how I wanted to work: https://github.com/jedateach/typescript-boilerplate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant