Added GoogleTest style output for unit tests

[samsinsane]

No description provided.

[TurkeyMan]

I like it... but I'm not sure if that's the practical part of me, or the "ooh, shiny thing" part of me ;)
Like, I'm not actually sure it's objectively superior to emit 200 lines of text when running the tests...

Anyway, I'm not saying don't merge this... it's cute. Just that there's a part of me that wonders if it's sensible :P

[tvandijck]

Well, I certainly like it because it's shiny, but there is more in this commit then just shiny output, it also does 'duplicate name detection', which already caught a few issues, and it's sorting the tests for a more deterministic run order. So for that reason I think we can take the shiny too ;)

[samsinsane]

I hear what you're saying, it's very much shiny. However, there is a benefit to this; you can see the timings. This allows us to look into unit tests that are "taking too long", or for users like me, you can see that the HTTP and Nuget tests take ~20 seconds to run and that you haven't broken anything. Other than that, all other fixes and changes could be done separately.

I do agree that emitting two lines for each of the 1783 unit tests, as well as three lines (one empty) for the 177 test suites, and then another 3 lines for the test action itself is a bit intense. An extra 4100 lines isn't something we should ignore, and I'm absolutely happy to discuss changes. Though, it seems like you two like the shiny enough to accept the extra 4100 lines of logging and it does only happen when testing, end users will never see it.

[tvandijck]

4100 lines is nothing... I'm looking through logs at least 100x that size at the moment..
What I do care about however, is that if a test fails, that I don't have to scroll up 4000 lines, because it was the first one that failed, and the rest all succeeded..

So it would be preferable to have a summary at the end of all failed tests (with callstack etc).
whether you simply rerun the failed tests at the end, or save the output from those tests, I don't really care... I just don't want to have to search for fails.

[samsinsane]

So it would be preferable to have a summary at the end of all failed tests (with callstack etc).
whether you simply rerun the failed tests at the end, or save the output from those tests, I don't really care... I just don't want to have to search for fails.

It does list the failed tests at the end, it doesn't emit the error message but that should be an easy fix. Would you like me to do that?

[tvandijck]

if you could, and then also rebase the thing so we can merge it.

[samsinsane]

if you could, and then also rebase the thing so we can merge it.

Done. This is what the output looks like with errors: (the HTTP tests decided now was the time to break and take ages)

[==========] 1783 tests from 177 test suites ran. (210610 ms total)
[  PASSED  ] 1781 tests.
[  FAILED  ] 2 tests, listed below:
[  FAILED  ] premake_http.http_get
HTTP response code said error
The requested URL returned error: 503 Service Unavailable

stack traceback:
[  FAILED  ] vstudio_vs2010_project_refs.referencesAreRelative_onDifferentProjectLocationWithCLR
...-core/modules/vstudio/tests/vc2010/test_project_refs.lua:87: (2) expected:
        <ProjectReference Include="MyProject.vcxprojs">
...but was:
        <ProjectReference Include="MyProject.vcxproj">
fulltext:
<ItemGroup>
        <ProjectReference Include="MyProject.vcxproj">
                <Project>{00112233-4455-6677-8888-99AABBCCDDEE}</Project>
                <Private>true</Private>
                <ReferenceOutputAssembly>true</ReferenceOutputAssembly>
                <CopyLocalSatelliteAssemblies>false</CopyLocalSatelliteAssemblies>
                <LinkLibraryDependencies>true</LinkLibraryDependencies>
                <UseLibraryDependencyInputs>false</UseLibraryDependencyInputs>
        </ProjectReference>
</ItemGroup>

stack traceback:
        ...-core/modules/vstudio/tests/vc2010/test_project_refs.lua:87: in function 'testFunction'

 2 FAILED TESTS