Deprecate in favor of istanbul report

[jamestalmage]

It looks like this is probably an unnecessary module, and that istanbul report json will do the same thing:

Usage: istanbul report <options> [ <format> ... ]

Options are:

      --config <path-to-config>
              the configuration file to use, defaults to .istanbul.yml

      --root <input-directory>
              The input root directory for finding coverage files

      --dir <report-directory>
              The output directory where files will be written. This defaults
              to ./coverage/

      --include <glob>
              The fileset pattern to select one or more coverage files,
              defaults to **/coverage*.json

      --verbose, -v
              verbose mode


<format> is one of 
      clover  XML coverage report that can be consumed by the clover tool
      cobertura
              XML coverage report that can be consumed by the cobertura tool
      html    Navigable HTML coverage report for every file and directory
      json    prints the coverage object as JSON to a file
      json-summary
              prints a summary coverage object as JSON to a file
      lcov    combined lcovonly and html report that generates an lcov.info
              file as well as HTML
      lcovonly
              lcov coverage report that can be consumed by the lcov tool
      none    Does nothing. Useful to override default behavior and suppress
              reporting entirely
      teamcity
              report with system messages that can be interpreted with TeamCity
      text    text report that prints a coverage line for every file, typically
              to console
      text-lcov
              lcov coverage report that can be consumed by the lcov tool
      text-summary
              text report that prints a coverage summary across all files,
              typically to console

[erindru]

The one thing istanbul report json doesnt seem to do is allow you to specify a base directory to combine files generated by karma. Unless i'm missing something?

[jamestalmage]

You should be able to do istanbul report --dir /path/to/coverage/dir --include **/*coverage.json json

I'm not entirely sure how karma lays out the files. As I recall there was a subdirectory for each browser, with a coverage.json in the folder. You should be able to figure out a pattern that works.

[jamestalmage]

Note that karma has an issue that may make it difficult to combine files generated by Karma with non Karma sources.

[erindru]

That defines the path to the coverage files, not a basePath to prepend to all the files within the coverage files. With karma-coverage, you can specify a basePath to your source, but when it outputs the reports, they dont include this basePath.

Therefore, when instanbul report reads the coverage files and tries to resolve the paths to the source files, it fails.

instanbul-combine fixes this issue by allowing a base option to be defined

[jamestalmage]

Right, per the issue I reference above, karma is doing it wrong. The Istanbul spec calls for paths to be absolute. For some reason, karma-coverage takes conformant output from the Istanbul tooling and replaces the absolute paths with relative ones. istanbul-combine does work around this, but it will have limited utility if/when that is fixed.

I wrote this tool to combine reports from karma and non-karma tests. But before I understood how karma was breaking the coverage reports. This whole library is basically a bandaid for a karma bug.

Once my PR gets merged into karma-coverage, I think I'll be able to safely deprecate this library and close this issue.

[jamestalmage]

OK,
So I made some more noise, and my PR just got merged.

@erindru
They plan to cut a release of karma-coverage sometime later today/tomorrow. Once that happens, can you try using the istanbul command line and verifying this library is no longer required for you? If it is, there may be another issue we need to look into it.

[erindru]

Yep, the 0.5.0 version of karma-runner fixes this issue, allowing istanbul report to work correctly. Thanks for pushing this through!

[yomed]

FWIW I wrote a tiny grunt plugin https://github.com/yomed/grunt-istanbul-combine because I couldn't seem to get grunt-istanbul to combine reports. Could be my own error, but this package did turn out to be useful...

[jamestalmage]

Cool.
I'm interested as to what you couldn't get to work with straight istanbul. What are you using as your test runner? Karma?

You should do it this way:

1. Run your tests with coverage (if using Karma - see the issue discussed above).
2. Run 'istanbul report' with all the json files as input, and output a single 'json' report
3. Run whatever report you want with the output from step 2

If that doesn't work for you, we should file an issue with istanbul or grunt-istanbul.

Either way. That's a lot of steps, so there's probably still utility for something like your plugin (even if this module is ultimately deprecated).

[yomed]

I have a big pipeline with separated unit tests and functional tests, so the unit coverage is measured with just mocha+istanbul, and the functional coverage needs karma in there. So all of the coverage is already handled, and I literally just need to merge two reports together.

I tried using grunt-istanbul for the makeReport task by itself, but maybe it wasn't meant for that? Sorry that's not too useful of a description. When I have time I'll go back and try to debug further, and possibly bring it up with istanbul or grunt-istanbul.

[jamestalmage]

What version of karma are you running? (istanbul-combine is definitely required if you are using < 0.5.0)

[sirmaenterprise]

For some reason istanbul report fails with unknown error when trying to comobine two reports. Howver istanbul-combine works perfectly.

[jamestalmage]

@SirmaITT Can you provide an example of how you are trying to use istanbul report?

[sirmaenterprise]

I have two reports - one made by karma and one made by protractor (i manually instrument the code before running the protractor tests).
If I run "istanbul report" on each file individually everything works fine. However if I run it on both files (using the below command), an error occurs "'seen' property not found".

node ./node_modules/.bin/istanbul report --include=reports/*/coverage.json -d overall_coverage lcov

[jamestalmage]

Are you using all the latest versions (karma, karma-coverage plugin, and istanbul?)

[sirmaenterprise]

I use "istanbul": "gotwarlost/istanbul.git#source-map" (the source-map branch)
karma is the latest version
For karma-coverage I use "karma-coverage": "douglasduteil/karma-coverage#next",

[jamestalmage]

Try with standard istanbul and see if you can get it to work.
If so, we probably need to let whoever is maintaining that branch know.

Also, just for my curiosity, does the souremap branch make a big improvement?

[sirmaenterprise]

Big improvement in which direction? I have the coverage + source maps working.

[jamestalmage]

Why are you using coverage + source maps?
I am assuming so that stack trace line numbers aren't garbled. I'm asking if that is working out well.

[sirmaenterprise]

I'm using it to get a nice report where i can see the coverage on my original (non-transpiled) code.

[jamestalmage]

Ohhh.
It takes sourcemaps IN, gotcha.

It didn't do that already?

[sirmaenterprise]

No. But you can track the progress here - gotwarlost/istanbul#212

[richardm]

Tried a dozen variations of this, with no luck:

report -v --include coverage/*.json --dir coverage-output json

I have a coverage directory containing two .json files which were generated with the latest version of karma. I'm getting the same error as here: gotwarlost/istanbul#403

My json files have absolute paths, not relative.

I posted there as well, but this project seems more active, and I'm wondering if anyone else has come across that error and found a workaround.

[yomed]

@jamestalmage FYI I've since moved to using istanbul directly for the reporting as you suggested, but my karma reports mysteriously have relative paths instead of absolute paths (despite being on the latest versions of karma grunt-karma and karma-coverage). I've hacked around it, but I think there is still some value in this plugin since it fixes that issue.

[tp]

@jamestalmage Does this work with plain "istanbul" for you already with istanbul v0.4?

I tried upgrading to istanbul 1.1-alpha but then had issues with the format being reported as wrong. (Maybe due to my karma setup still using istanbul 0.4 for reporting), but maybe it's just the bug that @richardm linked.

Anyway, since I integrated this project, I was finally able to get combined coverage across ava and karma tests :-) So I feel there is definitely in this at the moment!

[bedorlan]

istanbul report works as expected

[southpolesteve]

FYI with latest Istanbul, I no longer needed this library. Was successfully able to merge coverage reports from karma and jest.