Failing to list tests when using value-parameterized tests

[WillStavely]

Hi,

My google tests contain some value-parameterized tests. These run the tests over the values contained in an std::vector<std::pair<SomeType,SomeType>> where SomeType is a type I've defined.

The GoogleTestAdapter seems to have trouble listing the tests. Here's the output:

------ Discover test started ------
Google Test Adapter: Test discovery starting...
ERROR: Could not list test cases of executable 'C:\src\Win32\Debug\UnitTestRunner.exe': executing process failed with return code 3
Command executed: 'C:\src\Win32\Debug\UnitTestRunner.exe --gtest_list_tests', working directory: 'C:\src\Win32\Debug'
Command produced no output
Found 0 tests in executable C:\src\Win32\Debug\UnitTestRunner.exe
Test discovery completed, overall duration: 00:00:02.6580860
ERROR:
================
The following errors and warnings occured during test discovery (enable debug mode for more information):
ERROR: Could not list test cases of executable 'C:\src\Win32\Debug\UnitTestRunner.exe': executing process failed with return code 3
Command executed: 'C:\src\Win32\Debug\UnitTestRunner.exe --gtest_list_tests', working directory: 'C:\src\Win32\Debug'
Command produced no output
========== Discover test finished: 0 found (0:00:02.80028) ==========

If I run the test executable with --gtest_list_tests myself, it runs successfully with the following output:

C:\src\Win32\Debug>UnitTestRunner.exe --gtest_list_tests
SomeOtherTestFixture.
Test1
Test2
LotCalibrationParameterizedTests/LotCalibrationTestHarness.
RunCalibrationTests/0 # GetParam() = (40-byte object <14-96 D9-01 98-25 60-04
00-CD CD-CD CD-CD CD-CD CD-CD CD-CD CD-CD CD-CD 00-00 00-00 0F-00 00-00 48-C7 5
F-04 FE-FF FF-FF>, 40-byte object <14-96 D9-01 C8-6D 60-04 00-CD CD-CD CD-CD CD-
CD CD-CD CD-CD CD-CD CD-CD 00-00 00-00 0F-00 00-00...
RunCalibrationTests/1 # GetParam() = (40-byte object <14-96 D9-01 10-2B 60-04
00-CD CD-CD CD-CD CD-CD CD-CD CD-CD CD-CD CD-CD 00-00 00-00 0F-00 00-00 48-C7 5
F-04 FE-FF FF-FF>, 40-byte object <14-96 D9-01 F8-6F 60-04 00-CD CD-CD CD-CD CD-
CD CD-CD CD-CD CD-CD CD-CD 00-00 00-00 0F-00 00-00...
C:\src\Win32\Debug>

If I comment out the parameterized tests from the code, the GoogleTestAdapter behaves correctly and I can run the tests.

Thanks,
Will

[csoltenborn]

Thanks for your bug report! This is a bit strange, since the error indicates that the test executable could not be run (but no parsing of the output should have happened yet), so I do not see how this could be related to the tests contained in that executable. Are you using the old or the new test execution framework (see options), and can you reproduce the problem with the other framework, too?

Currently, my guess would be that there's a difference between execution on the console and within the adapter (e.g., something's missing on the PATH for the latter). Can you please double-check whether this is the case? For instance, can you run your executable via VisualStudio? I.e., make your test project the StartUp project of your solution, add the --gtest_list_tests option via the project's properties, and run/debug your project.

If this all doesn't help, I fear we need some sample test code to reproduce the problem over here (of course, no actual test logic required). Could you then please try to strip down your code and provide a minimal test file having that problem?

[WillStavely]

Thanks for the quick response. Running the executable from VisualStudio doesn't seem to have any issues. I don't have the time at the moment to create a minimal version of the code, but I will take a look at this in the next few days.

[csoltenborn]

I noticed only now that the output of your executable with option --gtest_list_tests looks strange; the test names should be indented by two spaces:

SomeOtherTestFixture.
  Test1

Is this an issue of GitHub formatting or something, or is this indeed the output the executable produces? If the latter, which gtest version is this?

And just to be sure: Have you checked whether the test executable's return code is indeed 0 when run with option --gtest_list_tests? You've written above that you've successfully run it - just want to make sure we're on the same line.

Also to be on the safe side: Have you tried the old execution framework? You didn't mention that in your last post...

[csoltenborn]

Closed due to inactivity - feel free to re-open if you still have problems...