Add sourcemap support

[fdintino]

This pull request is based on #346, but somewhat cleaned up (and it looks like that PR might be abandoned).

[fdintino]

What's the status of this PR? Any chance it could be merged in at some point?

[jezdez]

@fdintino This needs tests and docs at least. Are there other compressor supporting source maps that could be added at the same time?

[fdintino]

I've also written support for sourcemaps for uglifyjs and cssclean.

I agree on the tests, though I wasn't sure what the best way to go about that is. My first thought was to get node and java packages installed with the .travis.yml and bootstrap the different test cases with package.json files installed into temp directories (there is even an npm package to get closure.jar, so I could even use it for that). I would of course check if node or java is in the PATH and if not raise a SkipTest

[aehlke]

Thanks for this. I haven't had capacity for working more on my old PR - appreciate you picking this up!

[Spacerat]

Python 2.7's json.dumps doesn't allow strings as arguments for "indent" so I had to change this to indent=4 locally

[fdintino]

Thanks. We use python 2.7 so I'm not sure how this initially escaped my notice, but I discovered the same thing and just haven't pushed out the change (I'm still waiting for my unit test branch to get merged in before I rebase this pull request off of it).

[Spacerat]

Just a thought, do you think it's enough to add source map support at the compression stage but not the compilation stage? Without source maps for compilers, what do you do about coffeescript or ES6/jsx files?

[fdintino]

Provided that the compiler stage adds the sourceMappingURL comment at the bottom of the file, then the cleancss and closure compressors will merge the original input source maps into the final one. We use the sass compiler with sassc and the --sourcemap flag¹ and it works great with the cleancss compressor (if you want, inspect an element in Chrome on http://www.theatlantic.com/ to see it at work). I haven't tested that it works with closure (though I plan on writing unit tests for that once #531 is merged) but in theory it should, as I explicitly parse and pass along the input source maps in the closure arguments. UglifyJS only supports a single input source map via the cli for now but the support already exists in the API for multiple input source maps so you could customize the uglifyjs bin script to add this feature for yourself until the cli ticket is fixed. Or use the closure compressor, though it's about 10 times slower than uglify. None of the other bundled compressors support input source maps, so it would have to be between the above tools if you want this feature. The only other tool I'm aware of that can minify and concatenate input source maps is gulp. And I did write a gulp compressor before we switched to cleancss. But it's a terrible mess of dynamically building a Gulpfile.js in a temp directory and executing it with that cwd.

¹ Annoyingly, sassc (i.e. libsass), node-sass, and ruby sass all have different command-line flags for enabling source map output

[fdintino]

As you can tell, I've thought a lot about this :-)

So the short answer I suppose is that you can add the appropriate flags in your %(COMPILER)s_ARGUMENTS pipeline setting to enable sourcemaps for the time being. I think it would make sense for the OUTPUT_SOURCEMAPS setting to automatically add these flags to compilers that support source maps when True. But to be honest, given that it works well enough for our specific purposes, I'm not very motivated to improve upon this pull request until I get a sense that the maintainers (and I suppose specifically @cyberdelia) are ever going to merge them or even give feedback.

[fdintino]

This doesn't yet include the unit tests for source maps, but it's freshly rebased off of master (resolving the conflicts) and a few minor fixes are included, in case anyone wants to use this branch while I add tests and documentation.

[davidt]

This comment is now obsolete.

[tobz]

@cyberdelia @davidt

Is there any hope of ever merging this? It seems like a lot of the more meaningful, architectural PRs are sort of just being ignored out-of-hand, and I've yet to see any explanation as to why.

[fdintino]

@tobz This PR isn't being ignored, it's on me actually to finish it. I have some work I haven't pushed out yet that allows for sourcemaps for the post-processors that support it, but getting those sourcemaps to concatenate in the compressor step has proved challenging, specifically for javascript. uglifyjs does not support multiple input sourcemaps, nor does closure compiler. One option is to concatenate the sourcemaps in python (the source map spec is not all that complicated) and then pass a single input sourcemap to uglifyjs, but when I attempted that the performance isn't great. I'll work a bit more on it and perhaps leave multilevel sourcemap support for a future pull request.

[Spacerat]

@fdintino Adding my two cents on what worked for me: I ended up going with the 'concatenate sourcemaps then feed them to uglifyjs' option, and I used your source-map-concat-cli package to do the compression. Then I wrote a custom compressor to tie it all together. I never cared too much to test the performance since everything is only compiled once for Production anyway, although it seems to run fast enough in development.