Usage with .NET Core

[jonjomckay]

I'm having a hard time figuring out how to run my unit tests and generate the coverage.xml report for my assemblies... I spotted you added some more documentation this morning (thanks!), so now I'm running from inside my unit test project directory:

$ dotnet build
$ dotnet run --project ~/Downloads/altcover.1.4.150-beta/tools/netcoreapp2.0/AltCover/ -- /i:bin/Debug/netcoreapp2.0
$ cp -R __Instrumented/* bin/Debug/netcoreapp2.0
$ dotnet test --no-build

The tests run successfully (and I think they're instrumented), but my coverage.xml file is the same before and after (i.e. it says no code points were hit). I've tried a bunch of different command combinations (e.g. dotnet run --project ~/Downloads/altcover.1.4.150-beta/tools/netcoreapp2.0/AltCover/ -- /o:bin/Debug/netcoreapp2.0 -- dotnet test --no-build) but no dice :(

Any pointers, or plain commands that you've been using?

P.S. Thanks for being the guy to finally bring code coverage to .NET Core!

[SteveGilham]

Thank you for kicking the tyres on this one.

On your first point, about the lack of "how to", I've belatedly started to extract the practical examples onto the project wiki.

As to the "it doesn't seem to work" with the 1.4.150 build; this is the show-stopper bug that emerges just when a project is winding down to completion, and is the ticket I'm working on at the moment, being something I discovered too, when adding a new self-test build step so I could get a report on how much of the code was being tested under .net core.

The program works by registering a ProcessExit handler, which, should allow a couple of seconds extra processing for the application to tidy itself up. The AltCover recorder assembly uses this grace period to dump the coverage to file; and my experience using this approach in real-world unit testing in the day job back in the 2010-11 time-frame was that this approach was adequate for getting test coverage.

In practice it seems that running under dotnet test isn't allowing enough time for the process to complete. Removing an obvious inefficiency in the XML rewriting (commit ac442eb) improved the process to the point at which the file rewrite would usually have started before the process was killed; and then by adding the extra parameter to exclude the unit tests themselves from the XML (commit 1277986) means that the current drop (1.4.170-beta) can reliably self test -- but that the tool as it stands won't scale much above 1kloc for covered code.

I think the resolution has to involve a companion process that will do the work that the ProcessExit handler doesn't get time for, rather than attempting to do it all autonomously in-process.

[SteveGilham]

Experimentally (by putting Debug.WriteLine traces into the flushing process), the ProcessExit stage is allowed no more than 100ms in the .net core environment (as in, no larger elapsed times are recorded from runs made using dotnet test); while on framework, the flush process for the self-test took 400ms before the efficiencies made in build 170. Those brought the elapsed down to about 80ms. (All times recorded on my development machine.)

[FastNinja]

I am also keen to know - what is the exact command can I ran to get coverage while running xUnit unit test using dot net core on Linux...

[SteveGilham]

So I've added an XUnit demo that's run successfully on the linux build machine at travis-ci.

The steps are

1. purge the $(projectDir)/bin directory to remove any prior instrumentation
2. build with dotnet "build" $(Project) --configuration Debug
3. instrument with `dotnet run --project altcover.core.fsproj -- -x "/path/to/report.xml" -o "/some/clean/scratch/directory" -i "$(ProjectDir)/bin/Debug/netcoreapp2.0"
4. Copy/move all the files from "/some/clean/scratch/directory" to "$(ProjectDir)/bin/Debug/netcoreapp2.0"
5. Test with dotnet test --no-build --configuration Debug $(Project)

Hope this helps.

[SteveGilham]

And with the latest drop (v1.5.200-rc), step 5 above can be (and would preferably be)

dotnet run --project <path to>/altcover.runner.core.fsproj -- -x "dotnet" -r "$(ProjectDir)/bin/Debug/netcoreapp2.0" -w "$(ProjectDir)" -- test --no-build --configuration Debug $(Project)

[SteveGilham]

For working examples see a sample travis build. Look for Target: ReleaseXUnitFSharpTypesDotNet and ReleaseXUnitFSharpTypesDotNetRunner

This uses the "solution builds to one place, project another" shortcut, rather than a manual copy.

[SteveGilham]

And with the latest drop (v1.6.212-beta), step 5 above can be (and would preferably be)

dotnet run --project /altcover.core.fsproj -- runner -x "dotnet" -r "$(ProjectDir)/bin/Debug/netcoreapp2.0" -w "$(ProjectDir)" -- test --no-build --configuration Debug $(Project)

See the corresponding travis build.