Better integration with other pre-processors

[schulzch]

When integrating Brunch, Karma and Istanbul precious information from source maps is lost. I think its a good idea to look for source maps during instrumentation (app.js.map for app.js) and display sources instead of intermediate code.

We use Brunch to compile the application (bare vendor.js and requirejs app.js) and Karma to compile tests, instrument app.js using Istanbul and run tests inside Chrome/Firefox/etc.. Unfortunately the coverage report and source view (developer tools) display a single app.js instead of source files. Random guessing what the source file was is a real pain in the ***, not to mention virtually useless statistics.

This possibly relates to:

• Issue with generating lcov/html reports for traceur-compiled files Issue with generating lcov/html reports for traceur-compiled files #162
• Support source maps for html report Support source maps for html report #59
• Support for CoffeeScript, TypeScript
• Enterprisy tool chains

[gotwarlost]

Is there a sample project and a build script that will help me understand all the moving parts without having to learn a bunch of tools? I realize that you are asking for something more abstract but I like to get my hands dirty with your use-case before trying to come up with a solution.

[justindujardin]

@gotwarlost see pow-core for an example, and an example report.

It would be much nicer if istanbul could look at my source maps and generate reports that include the original TypeScript files instead of one composite javascript file.

Clone the repo, set coverageDebug = true at the top of karma.config.js, and npm install && npm install bower && ./node_modules/.bin/bower install. Then npm test whenever you want to generate a new report, which will show up in .coverage

+1 for this feature.

[jessaustin]

@justindujardin I agree this would be a nice enhancement for istanbul. This is just a quick note to make installation of pow-core easier. Add a postinstall property to the scripts property of package.json:

  "scripts": {
    "postinstall": "bower install"
  }

Now a simple npm install is all you need!

[chinchang]

+1
Right now I need to run tests on development environment where there is no concatenation and minification so that I get coverage for source files.

[schulzch]

@gotwarlost pow-core matches my use case more or less (its using TypeScript instead of CoffeeScript and Grunt instead of Brunch).

One way to implement this feature would be by using Mozilla's Source Map parser:

1. Parse source map in instrumenter.js:instrumentSync()
2. Annotate each uninstrumented AST node with a source line and source column
3. Modify this.sourceMap in instrumenter.js:instrumentASTSync() so that instrumented code points to source code instead of uninstrumented code

Viable approach?

[vvakame]

👍

[vvakame]

@gotwarlost I know good document for SourceMap.
I think gulp ( and browserify ) is defacto of streaming processing tools. I think that it is good when you reference the gulp of policy.

https://github.com/floridoo/gulp-sourcemaps/wiki/Plugins-with-gulp-sourcemaps-support
http://thlorenz.com/blog/browserify-sourcemaps
https://github.com/azu/multi-stage-sourcemap
https://github.com/twada/battlefield-sourcemaps

in karma-sourcemap.
https://github.com/demerzel3/karma-sourcemap-loader/blob/2c9ee367aa9fcf7cf65bf8c224d5c89b7f151d12/index.js#L9

I want to use SourceMap with TypeScript and karma ( karma-sourcemap -> karma-coverage (it uses istanbul!) ).

---

this is very useful document if you can read japanese...
http://efcl.info/2014/09/03/multi-stage-sourcemap/

[fkling]

👍 Since transpilers became increasingly popular this year, having support for source maps would be tremendously useful.

[lo1tuma]

👍

[arielschiavoni]

+1

[dayyan]

👍

[GoulwenBZH]

+1

[simonzack]

👍, this really makes it hard with 6to5 or traceur. I cannot switch to istanbul due to this issue.

[jvilk]

👍 I completely agree. It seems like this could be a post-processing step, since Istanbul already has the relevant JS source location information for covered / non-covered statements. I believe you can map those back to the original source code via the source maps.

[jtangelder]

👍

[Iorg]

+1

[danvk]

I wound up writing a script to apply a source map to LCOV data to get coverage data for a recent project. Built-in support from Istanbul would be fantastic.

[gotwarlost]

@danvk - this is great! This gives me a good starting point. Thanks!

[brunodunbar]

This feature is gonna be very helpful, especially when running tests on minified/uglified code.

[gotwarlost]

Working on it in the source-map branch... Code is still very raw and not ready for use.

[mysterycommand]

👍

[tiriana]

+1 Using source maps would increase built code coverage significantly. Are there any chances for this to happen? [typescript => js]

[wojciak]

+1 (ts -> js)

[topaxi]

+1 https://github.com/douglasduteil/isparta is doing something like this already for babel, but sadly isn't an option for me anymore

[PinkaminaDianePie]

+1 this feature more important than any other

[rattrayalex]

+1 TS

Any word on progress?

[acromarco]

+1 (ts -> js)

[trodrigues]

@D34THWINGS Have a look at this project https://github.com/contentful/contentful-space-sync

It pretty much does the same stuff that @gotwarlost did on his demo project, essentially:

• Make sure your babel config has sourcemaps enabled: https://github.com/contentful/contentful-space-sync/blob/master/.babelrc#L5
• Run the istanbul cli script via babel-node so it reads the same configuration file https://github.com/contentful/contentful-space-sync/blob/master/package.json#L13

That's about it. Initially I posted here because of a babel plugin I was using, and I wasn't sure where the actual problem was, but it turned out to be mostly a matter of setting things up properly.

[shaharmor]

Is there any complete guide on how get this working with typescript/webpack/karma?

[joshma]

Is there planned support for sourceMappingURL? Bottom of my file looks like:

//# sourceMappingURL=__maps__/polyfills.js.map

and I'm getting this error:

Error returning source map for /Users/joshma/my_project/js-build/polyfills.js
"coffee/polyfills.coffee" is not in the SourceMap.
/Users/joshma/my_project/node_modules/istanbul/node_modules/istanbul-api/node_modules/istanbul-lib-source-maps/lib/map-store.js:133
            if (this.verbose) {

(And I'm using (gulp-sourcemaps](https://www.npmjs.com/package/gulp-sourcemaps), if that's relevant)

[donaldpipowitch]

Can someone help me and explain what is the current state of this issue? This issue is still open, but it seems some people had a working example in mid 2015.

To be specific: Can I get code coverage when I use Karma, TypeScript and Webpack in a similar setup like this one?

[tilgovi]

I got working code coverage, but it breaks source maps. The source maps work fine in the coverage reporter, but since the source map pass happens on the coverage output rather than during the code instrumentation, source maps are broken during the test run if you need to interactively debug the tests.

So, all my setups so far run test suites with an optional environment variable to enable coverage. I turn it on in CI and leave it off locally.

[donaldpipowitch]

Thank you. Could you please share your config?

[tilgovi]

Here's an example with browserify and browserify-istanbul:

• https://github.com/tilgovi/dom-seek/blob/master/package.json
• https://github.com/tilgovi/dom-seek/blob/master/karma.conf.js

You'll notice in the karma configuration that I only add istanbul to the pipeline if coverage is requested. This is because, as I said, with this setup the coverage report takes the intermediate source map into consideration so that line and branch coverage is accurate but the instrumentation does not so debugging in the browser is not possible at the same time coverage is enabled.

[donaldpipowitch]

Thank you.

[Izhaki]

For anyone interested in mapped coverage reports involving:

• Typescript
• Jasmine / Mocha
• tsc
• npm scripts

This may be of help:

Output

Jasmine

See https://github.com/Izhaki/Typescript-Jasmine-Istanbul-Boilerplate.

package.json (the relevant stuff):

{
  "scripts": {
    "postinstall": "typings install dt~jasmine --save --global",
    "test": "ts-node node_modules/.bin/jasmine JASMINE_CONFIG_PATH=jasmine.json",
    "test:coverage": "ts-node node_modules/istanbul/lib/cli.js cover -e .ts  -x \"*.d.ts\" -x \"*.spec.ts\" node_modules/.bin/jasmine -- JASMINE_CONFIG_PATH=jasmine.json"
  },
  "devDependencies": {
    "istanbul": "^1.1.0-alpha.1",
    "jasmine": "^2.4.1",
    "ts-node": "^0.9.3",
    "typescript": "^1.8.10",
    "typings": "^1.3.1"
  },
}

Mocha

See https://github.com/Izhaki/Typescript-Mocha-Istanbul-Boilerplate.

package.json (the relevant stuff):

{
  "scripts": {
    "postinstall": "typings install dt~mocha dt~require dt~chai --save --global",
    "test": "mocha --compilers ts:ts-node/register src/**/*.spec.ts",
    "test:coverage": "ts-node node_modules/istanbul/lib/cli.js cover -e .ts  -x \"*.d.ts\" -x \"*.spec.ts\" _mocha -- --compilers ts:ts-node/register -R spec src/**/*.spec.ts"
  },
  "devDependencies": {
    "chai": "^3.5.0",
    "istanbul": "^1.1.0-alpha.1",
    "mocha": "^2.5.3",
    "ts-node": "^0.9.3",
    "typescript": "^1.8.10",
    "typings": "^1.3.1"
  },
}

[nojvek]

What is the current status of source mapping with Istanbul?

For our project remap-istanbul failed horribly so we ended up writing our own script to merge and map the coverage json files.

I'd love to contribute back to istanbul so "istanbul report" can have a nice --sourcemap option.

[faergeek]

@nojvek I use nyc + babel-plugin-istanbul for that, it works great with ava and babel.

[Glavin001]

I second nyc!

I found Izhaki/Typescript-Mocha-Istanbul-Boilerplate#2 (comment) which recommended reading http://rundef.com/typescript-code-coverage-istanbul-nyc

My final package.json which is working well for our project is:

{
  // ...
  "scripts": {
    "test:unit": "cross-env TS_NODE_PROJECT=test/unit mocha --compilers ts:ts-node/register --recursive test/unit",
    "test:coverage": "nyc -e '.ts' --r html -r lcov -r text npm run test:unit"
  },
  // ...
  "devDependencies": {
    "cross-env": "^3.1.4",
    "istanbul": "^1.1.0-alpha.1",
    "mocha": "^3.2.0",
    "nyc": "^10.1.2",
    "ts-node": "^2.0.0" 
  },
  // ...
}

Hope this helps!

[mmc41]

@Izhaki Could not get mocha support working. First I got the error "Unable to resolve file [.ts]", then if I removed the (undocumented?) "-e .ts" option, the typescript compiler began wrongly to process source files under node_module which gave redeclaration errors.

[mmc41]

@gotwarlost Would it be possible to have build-in support in istanbul for source maps (e.g. typescript etc). I tried various workarounds as suggested here but they either don't work with newest versions or they introduce other problems like making mocha not report correct source line on error or they are not portable.

[Izhaki]

Can someone please enlighten me as to the actual relationship between this project (istanbul) and nyc?

If one is to replace the other, it seems to me very reasonable to got the nyc path.

But if the two are developed in parallel (and thus somewhat competing), it is perhaps unwise to merge the nyc PR on the sample project, since it uses a 3rd party solution? It will be rather odd if this repository gets an update that solve the issue discussed without nyc.

What do people think?

[mummybot]

From Istanbul's website the relationship appears to be nyc IS the CLI for Istanbul:

Istanbul instruments your ES5 and ES2015+ JavaScript code with line counters, so that you can track how well your unit-tests exercise your codebase.
The nyc command-line-client for Istanbul works well with most JavaScript testing frameworks: tap, mocha, AVA, etc.

[Izhaki]

I've seen that, but still unclear as to the relationship between https://github.com/gotwarlost/istanbul and https://istanbul.js.org/. There's no mentioning of the latter in the former.

[BrainCrumbz]

TBH, @Izhaki is right, I mean, on https://istanbul.js.org/ web page the GitHub link does not point here, but to https://github.com/istanbuljs

[mummybot]

I agree the relationship between the projects is unclear. That would be a question for @davglass, given he has many contributions to both projects, or to @gotwarlost who wrote the original Istanbul.

Jest (Facebook's test framework) which bakes in coverage pulls in istanbul-api from https://github.com/istanbuljs. I don't think you can go wrong using any of the Istanbul.js.org repos. Looking at the number of commits to the base Istanbul recently I assume that the project is relatively mature.

[mmc41]

I also wondered about the project relationship. I have moved on to nyc - works well with typescript, sourcemaps etc.

[morsh]

FYI - This happened to me on a windows machine.
I manages to fix it by cleaning up the Temp folder: C:\Users\USERNAME\AppData\Local\Temp\jest. (delete jest temporary folder)