Add some CLI testing

[addaleax]

Add regression tests for #209 and #190. Feel very free to point out things you’d like to be changed! 😄
And btw, these tests should work on CI systems, but fail for me locally when I use the node/npm binaries that are not shipped with their tarballs (make install from the Node.js git source). istanbuljs/spawn-wrap#19 should address that, though.

[bcoe]

@addaleax awesome \o/ thanks for biting the bullet and starting to write some integration tests for the CLI.

A few thoughts:

• let's pull the cli integration tests into their own test file, perhaps called nyc-bin.js.
• let's start displaying coverage information for bin/nyc.js.
• let's also pull the fixtures into their own sub-folder, probably called cli.

Really appreciate your contributions \o/

[addaleax]

let's pull the cli integration tests into their own test file, perhaps called nyc-bin.js.

Sounds like a good idea. Two things:

1. This would require re-writing build-tests.js to handle both files, or make it take an argument indicating which source file to split up, right?
2. There are already some tests which spawn the nyc cli – Should these be moved, too?

[bcoe]

@addaleax correct me if I'm wrong, but I don't think the build step for source-map-cache.js is as complex as the nyc-test.js build process? I think we can probably copy the approach used for source-map-cache.js.

I would say: if the tests are specifically exercising aspects of the cli: how it parses arguments, its various commands, its exit codes; let's pull them into this new file.

The build step of the tests is designed to test the nitty-gritty functionality if index.js in a few different modes of operation, cached, uncached, etc.

[addaleax]

@bcoe No, you’re right, that’s why I’m asking :-) I’ll try that then, shouldn’t take too long.

[addaleax]

Btw, are you fine with the added symlinks? I don’t quite remember how current git implementations on Windows handle them (… is that why AppVeyor won’t start?) Removed the symlinks while rebasing

[addaleax]

@bcoe Since all CI tests run on Node.js >= 0.12, is using child_process.spawnSync okay? That would definitely make the tests more concise.

[jamestalmage]

I don't think the build step for source-map-cache.js is as complex as the nyc-test.js build process? I think we can probably copy the approach used for source-map-cache.js.

Correct. nyc-test.js is split into multiple files so you get a clean process for each test. This ended up being necessary because of the way nyc mucks with the currently running process (and because it doesn't behave well when you invoke it multiple times on the same process).

You probably still need a build step to create bin/nyc.covered (same as how source-map-cache.covered.js is created).

You should do something similar to this to determine if the covered file exists or not.

---

@addaleax If the above stuff regarding coverage is confusing, feel free to just focus on writing some good tests and I can add coverage in another PR when you are done.

[addaleax]

@jamestalmage It’s not confusing by itself – like, I know the purpose and how it works in theory – but you obviously have the advantage of practical experience with the necessary steps. So, if you have the time, feel free to do that 👍

[addaleax]

@bcoe The last commit adds the test cases I talked about… Travis should become green when including the spawn-wrap PR (Edit: it does 🎉)

[bcoe]

@addaleax looks like you might have already, but if you rebase these tests should now start passing.

[addaleax]

@bcoe yup, just did after you merged #226 😄 but Travis is green here too, so that should be fine… is there anything else that should definitely go into this PR?

[bcoe]

@addaleax this rocks and you rock.