Very late report for a missed expectation

[Alex0vSky]

Unexpected behavior

• Unexpected behavior after completing a single test..
• Double "Reporting" on single API hook.

Description

I’m writing about two at once, because one follows from the other and it’s difficult to separate them.

Unexpected behavior after completing a single test..

Leads to: it is impossible to detect an error/failure after the completion of a particular test, there is no process termination error code to be processed, and there are also no error records in the xml/json output file.

Double "Reporting" on single API hook.

Leads to: additional reporting of false expectations.

---

Additional context

1

After complete single test

Expected Behavior 1

[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from verifyAndClear_unexpectedBehavior_reportAfterTests
[ RUN      ] verifyAndClear_unexpectedBehavior_reportAfterTests.gmockWin32
C:\Prj\_PullRequest\gmock-win32\_changes\verifyAndClear\expectedButFail.cpp(13): error: Actual function call count doesn't match EXPECT_CALL(mock_module_GetActiveWindow::instance(), GetActiveWindow())...
         Expected: to be called once
           Actual: never called - unsatisfied and active
[  FAILED  ] verifyAndClear_unexpectedBehavior_reportAfterTests.gmockWin32 (2 ms)
[----------] 1 test from verifyAndClear_unexpectedBehavior_reportAfterTests (5 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (11 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] verifyAndClear_unexpectedBehavior_reportAfterTests.gmockWin32

 1 FAILED TEST

C:\Prj\_PullRequest\gmock-win32\_changes\verifyAndClear\bin\x64\Debug\expectedButFail.exe (process 5816) exited with code 1.
Press any key to close this window . . .

Note: (process Xxx) exited with code 1 !

Actual Behavior 1

[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from verifyAndClear_unexpectedBehavior_reportAfterTests
[ RUN      ] verifyAndClear_unexpectedBehavior_reportAfterTests.gmockWin32
[       OK ] verifyAndClear_unexpectedBehavior_reportAfterTests.gmockWin32 (0 ms)
[----------] 1 test from verifyAndClear_unexpectedBehavior_reportAfterTests (1 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (3 ms total)
[  PASSED  ] 1 test.
C:\Prj\_PullRequest\gmock-win32\_changes\verifyAndClear\expectedButFail.cpp(13): error: Actual function call count doesn't match EXPECT_CALL(mock_module_GetActiveWindow::instance(), GetActiveWindow())...
         Expected: to be called once
           Actual: never called - unsatisfied and active

C:\Prj\_PullRequest\gmock-win32\_changes\verifyAndClear\bin\x64\Debug\expectedButFail.exe (process 7920) exited with code 0.
Press any key to close this window . . .

Note: (process Xxx) exited with code 0 !.

Steps to Reproduce the Problem 1

1.1. Source code
expectedButFail.cpp

#pragma comment( lib, "kernel32" )
#pragma comment( lib, "user32" )
#include "..\src\gtest\gtest-all.cc"
#include "..\src\gmock\gmock-all.cc"
#include "..\..\src\gmock-win32.cpp"
#include "..\include\gtest\gtest-spi.h" 
int main(int argc, char **argv) {
	const gmock_win32::init_scope gmockWin32{ };
	return ::testing::InitGoogleTest( &argc, argv ), RUN_ALL_TESTS( );
}
MOCK_CDECL_FUNC( HWND, GetCapture );
TEST(verifyAndClear_unexpectedBehavior_reportAfterTests, gmockWin32) { 
	EXPECT_MODULE_FUNC_CALL( GetCapture );
//	bool result = true;
//	EXPECT_NONFATAL_FAILURE( 
//			VERIFY_AND_CLEAR_MODULE_FUNC( GetCapture )
//			, "Actual function call count doesn't match EXPECT_CALL(mock_module_GetCapture::instance(), GetCapture())..."
//		);
//	ASSERT_TRUE( result );
}

2.1. Building in cmd line

Include path to googletest(NuGet) and current project

set googletest=googletest\gmock\include
set gmock-win32=gmock-win32\include

Building for cl from Msvc 2019

cl expectedButFail.cpp /EHsc /I"%googletest%" /I"%gmock-win32%"

3.1. Run expectedButFail.exe

2

Double report

Expected Behavior 2

[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from verifyAndClear_unexpectedBehavior_doubleReport
[ RUN      ] verifyAndClear_unexpectedBehavior_doubleReport.addingToInternals
C:\Prj\_PullRequest\gmock-win32\_changes\verifyAndClear\verifyAndClear.cpp(13): error: Actual function call count doesn't match EXPECT_CALL(mock_module_GetACP::instance(), GetACP())...
         Expected: to be called once
           Actual: never called - unsatisfied and active
[  FAILED  ] verifyAndClear_unexpectedBehavior_doubleReport.addingToInternals (1 ms)
[ RUN      ] verifyAndClear_unexpectedBehavior_doubleReport.forCurrentGetACP_and_previousGetACP
C:\Prj\_PullRequest\gmock-win32\_changes\verifyAndClear\verifyAndClear.cpp(17): error: Actual function call count doesn't match EXPECT_CALL(mock_module_GetACP::instance(), GetACP())...
         Expected: to be called once
           Actual: never called - unsatisfied and active
[  FAILED  ] verifyAndClear_unexpectedBehavior_doubleReport.forCurrentGetACP_and_previousGetACP (0 ms)

Actual Behavior 2

[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from verifyAndClear_unexpectedBehavior_doubleReport
[ RUN      ] verifyAndClear_unexpectedBehavior_doubleReport.addingToInternals
[       OK ] verifyAndClear_unexpectedBehavior_doubleReport.addingToInternals (1 ms)
[ RUN      ] verifyAndClear_unexpectedBehavior_doubleReport.forCurrentGetACP_and_previousGetACP
C:\Prj\_PullRequest\gmock-win32\_changes\verifyAndClear\doubleReport.cpp(13): error: Actual function call count doesn't match EXPECT_CALL(mock_module_GetACP::instance(), GetACP())...
         Expected: to be called once
           Actual: never called - unsatisfied and active
C:\Prj\_PullRequest\gmock-win32\_changes\verifyAndClear\doubleReport.cpp(17): error: Actual function call count doesn't match EXPECT_CALL(mock_module_GetACP::instance(), GetACP())...
         Expected: to be called once
           Actual: never called - unsatisfied and active
[  FAILED  ] verifyAndClear_unexpectedBehavior_doubleReport.forCurrentGetACP_and_previousGetACP (1 ms)

Steps to Reproduce the Problem 2

2.1. Source code
doubleReport.cpp

#pragma comment( lib, "kernel32" )
#pragma comment( lib, "user32" )
#include "..\src\gtest\gtest-all.cc"
#include "..\src\gmock\gmock-all.cc"
#include "..\..\src\gmock-win32.cpp"
#include "..\include\gtest\gtest-spi.h" 
int main(int argc, char **argv) {
	const gmock_win32::init_scope gmockWin32{ };
	return ::testing::InitGoogleTest( &argc, argv ), RUN_ALL_TESTS( );
}
MOCK_MODULE_FUNC( UINT, GetACP );
TEST(verifyAndClear_unexpectedBehavior_doubleReport, addingToInternals) { 
	EXPECT_MODULE_FUNC_CALL( GetACP );
	//VERIFY_AND_CLEAR_MODULE_FUNC( GetACP );
}
TEST(verifyAndClear_unexpectedBehavior_doubleReport, forCurrentGetACP_and_previousGetACP) { 
	EXPECT_MODULE_FUNC_CALL( GetACP );
	VERIFY_AND_CLEAR_MODULE_FUNC( GetACP );
}

2.2. Building in cmd line

Include path to googletest(NuGet) and current project

set googletest=googletest\gmock\include
set gmock-win32=gmock-win32\include

Building for cl from Msvc 2019

cl doubleReport.cpp /EHsc /I"%googletest%" /I"%gmock-win32%"

3.2. Run doubleReport.exe

Specifications

• Version: latest
• Platform: Windows
• Subsystem: Visual Studio Community 2019