Test Discovery did not find any Tests

[Jens-Ehrlich]

After building a project, the test discovery is started. No tests are found.

Example output of the discovery process:

------ Discover test started ------
15:20:27.921 - Visual Studio Version: VS2015
15:20:27.923 - Google Test Adapter: Test discovery starting...
15:20:27.929 - Solution settings: AdditionalTestExecutionParam: '--private --project-dir $(ExecutableDir) --disable-dbg-report', BatchForTestSetup: '', BatchForTestTeardown: '', BreakOnFailure: False, CatchExceptions: False, DebugMode: True, KillProcessesOnCancel: True, MaxNrOfThreads: 8, NrOfTestRepetitions: 1, ParallelTestExecution: False, ParseSymbolInformation: True, PathExtension: '$(ExecutableDir)..\dll;$(ExecutableDir)......\bif\libglob\sinstall\dll;$(ExecutableDir)......\external\qt\lib', PrintTestOutput: True, RunDisabledTests: False, ShowReleaseNotes: True, ShuffleTests: False, ShuffleTestsSeed: 0, TestDiscoveryRegex: '', TestDiscoveryTimeoutInSeconds: 120, TestNameSeparator: '', TimestampOutput: True, TraitsRegexesAfter: {}, TraitsRegexesBefore: {}, UseNewTestExecutionFramework: True, WorkingDir: '$(ExecuteableDir)'
15:20:27.933 - No settings configured for test executable 'F:\R163\mb2cpp.bin\debug\mb2_lib_intf_ImportAdapter_IntegrationTests\mb2_lib_intf_ImportAdapter_IntegrationTests.exe'; running with solution settings: AdditionalTestExecutionParam: '--private --project-dir $(ExecutableDir) --disable-dbg-report', BatchForTestSetup: '', BatchForTestTeardown: '', BreakOnFailure: False, CatchExceptions: False, DebugMode: True, KillProcessesOnCancel: True, MaxNrOfThreads: 8, NrOfTestRepetitions: 1, ParallelTestExecution: False, ParseSymbolInformation: True, PathExtension: '$(ExecutableDir)..\dll;$(ExecutableDir)......\bif\libglob\sinstall\dll;$(ExecutableDir)......\external\qt\lib', PrintTestOutput: True, RunDisabledTests: False, ShowReleaseNotes: True, ShuffleTests: False, ShuffleTestsSeed: 0, TestDiscoveryRegex: '', TestDiscoveryTimeoutInSeconds: 120, TestNameSeparator: '', TimestampOutput: True, TraitsRegexesAfter: {}, TraitsRegexesBefore: {}, UseNewTestExecutionFramework: True, WorkingDir: '$(ExecuteableDir)'
15:20:27.941 - File does not seem to be Google Test executable: 'F:\R163\mb2cpp.bin\debug\mb2_lib_intf_ImportAdapter_IntegrationTests\mb2_lib_intf_ImportAdapter_IntegrationTests.exe'
15:20:27.942 - Test discovery completed, overall duration: 00:00:00.0478373
========== Discover test finished: 0 found (0:00:00,6630663) ==========

I am able to start the tests from command line using the extended path. --gtest_list_tests does work too.

This problem did not occur in version 0.10.2. We first encountered it in version 0.11. I am using version 0.10.2 again and everything seems to work again.

Any help or advise is appreciated.

[csoltenborn]

Apparently, the new feature to automatically recognize test executables doesn't work in your case. The fix is to either provide a custom regex (for now), or to make sure there's an according .is_google_test file (see trouble shooting section of the docs).

Could you attach the executable (or send it to me by private mail)?

[Jens-Ehrlich]

Hi, thanks for your response. I have create a minimal example consisting of 2 tests. I hope this is what you expected. https://drive.google.com/open?id=0B_kwzV99gRDqQU5fdUQtRkI3bEk Using a custom regex does work for now. The regex is ".exe". Best Regards Jens Ehrlich

…

On 26 September 2017 at 20:30, Christian Soltenborn < ***@***.***> wrote: Apparently, the new feature to automatically recognize test executables doesn't work in your case. The fix is to either provide a custom regex (for now), or to make sure there's an according .is_google_test file (see trouble shooting section of the docs). Could you attach the executable (or send it to me by private mail)? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#169 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGrOMDDVrI-AaepBs4kg2xW9PiiNE0drks5smUKygaJpZM4PkRna> .

[Isameru]

It seems that the project name must contain phrase "test" in its name to make the discovery work.
Well... that's fine... usually.

[csoltenborn]

@Jens-Ehrlich Thanks, that's exactly what I was looking for. However, the guys at Microsoft have already discovered and fixed this bug (GTA code doesn't look for dynamically linked gtest).

@Isameru No, that's not the case here. In opposite, 0.11 is the first version which does not require any naming scheme (or appropriate configuration) if gtest is compiled into the executable (as long as this bug isn't fixed).

[Isameru]

Right. I'm using version 0.10.1.0.

[csoltenborn]

Version 0.11.1 will check for an import of DLL gtest.dll (and should thus work fine with your example executable). A more sophisticated investigation of the linked DLLs will probably be implemented in a later version.

[jmecosta]

@csoltenborn is there a dedicated ticket for the problem you reported. 0.11.1 does not work when tests use gtest build as dynamic dll.

[17.10.2017 11.55.38 Informational] Google Test Adapter: Test discovery starting...
[17.10.2017 11.55.38 Informational] Solution settings: AdditionalTestExecutionParam: '', BatchForTestSetup: '', BatchForTestTeardown: '', BreakOnFailure: False, CatchExceptions: True, DebugMode: True, KillProcessesOnCancel: True, MaxNrOfThreads: 8, NrOfTestRepetitions: 1, ParallelTestExecution: False, ParseSymbolInformation: True, PathExtension: '', PrintTestOutput: True, RunDisabledTests: False, ShowReleaseNotes: False, ShuffleTests: False, ShuffleTestsSeed: 0, TestDiscoveryRegex: '', TestDiscoveryTimeoutInSeconds: 30, TestNameSeparator: '', TimestampOutput: False, TraitsRegexesAfter: {}, TraitsRegexesBefore: {}, UseNewTestExecutionFramework: True, WorkingDir: '$(ExecutableDir)'
[17.10.2017 11.55.38 Informational] No settings configured for test executable 'C:\prod\structures\BuildDrop\Work\bin_x64\GTests\LibSearchTests.exe'; running with solution settings: AdditionalTestExecutionParam: '', BatchForTestSetup: '', BatchForTestTeardown: '', BreakOnFailure: False, CatchExceptions: True, DebugMode: True, KillProcessesOnCancel: True, MaxNrOfThreads: 8, NrOfTestRepetitions: 1, ParallelTestExecution: False, ParseSymbolInformation: True, PathExtension: '', PrintTestOutput: True, RunDisabledTests: False, ShowReleaseNotes: False, ShuffleTests: False, ShuffleTestsSeed: 0, TestDiscoveryRegex: '', TestDiscoveryTimeoutInSeconds: 30, TestNameSeparator: '', TimestampOutput: False, TraitsRegexesAfter: {}, TraitsRegexesBefore: {}, UseNewTestExecutionFramework: True, WorkingDir: '$(ExecutableDir)'
[17.10.2017 11.55.38 Informational] File does not seem to be Google Test executable: 'C:\prod\structures\BuildDrop\Work\bin_x64\GTests\LibSearchTests.exe'
[17.10.2017 11.55.38 Informational] Test discovery completed, overall duration: 00:00:00.0404224
[17.10.2017 11.55.38 Informational] ========== Discover test finished: 0 found (0:00:01,3344684) ==========

[csoltenborn]

@jmecosta I assume that your gtest DLL is not named gtest.dll? Otherwise GTA should recognize your executable...

No, there is no issue for that yet - we wanted to see whether this is a common case (since the alternative, i.e. scanning all imports, is quite a bit to implement). For now, you are stuck with either configuring a test discovery regex or providing a .is_google_test file.

[jmecosta]

Righy, we have both gmock and gtest in same assembly so we called it gtestmock :)

…

On Tue, 17 Oct 2017, 21:07 Christian Soltenborn, ***@***.***> wrote: @jmecosta <https://github.com/jmecosta> I assume that your gtest DLL is not named gtest.dll? Otherwise GTA should recognize your executable... No, there is no issue for that yet - we wanted to see whether this is a common case (since the alternative, i.e. scanning all imports, is quite a bit to implement). For now, you are stuck with either configuring a test discovery regex or providing a .is_google_test file. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#169 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA_jyLqZFca19xt6utoCZ3Ef2RV8uIsjks5stOxZgaJpZM4PkRna> .

[Knitschi]

Hi, I also have the "File does not seem to be Google Test executable" problem when using dynamicly linked gtest. I would like to avoid generating the FooTests.exe.is_google_test file.

You mentioned that GoogleTestAdapter will check for an import of gtest.dll. This does not seem to work in my case. Can you point me to the VS project property that must be set to make this work?

I am using GoolgeTestAdapter version 0.15.0.1305 and a cmake based project.

[csoltenborn]

There is no VS project property. If automatic test executable recognition does not work, you can either use the mentioned marker file or a regex (see trouble shooting).

[Knitschi]

Reading your comment

Version 0.11.1 will check for an import of DLL gtest.dll (and should thus work fine with your example executable)

I assumed that it GoogleTestAdapter can identify a test-executable that uses dynamic gtest without the need for setting the regexp or the helper file. If this is the case I would appreciate if you could give me some information on how this is implemented so I can try to fix it in my project.

[csoltenborn]

This is the method which checks whether the given file is a gtest executable:

GoogleTestAdapter/GoogleTestAdapter/Core/GoogleTestDiscoverer.cs

Line 83 in db6f65b

public static bool IsGoogleTestExecutable(string executable, string customRegex, ILogger logger)

As you can see, it checks whether a file gtest.dll is imported (or looks for some fixed gtest strings in the case of static inclusion). Thus, make sure your gtest dll has that exact name.