Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Increase test timeout on Windows #522

Closed
wants to merge 1 commit into from
Closed

Increase test timeout on Windows #522

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion ament_cmake_test/cmake/ament_add_test.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,11 @@ function(ament_add_test testname)
set(ARG_RUNNER "${ament_cmake_test_DIR}/run_test.py")
endif()
if(NOT ARG_TIMEOUT)
set(ARG_TIMEOUT 60)
if(WIN32)
set(ARG_TIMEOUT 180)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This affects the timeouts of all tests, which feels very broad. It could mask issues that cause all tests to take longer. If there are particular tests taking longer than 60 seconds then we could increase those with an issue to look at them closer later.

In the linked build I'm not sure that increasing the timeout will the test failures. It looks like there might be a real bug. For example this job seems like it might be timing out because the server died

[test_client_scope_consistency_client_cpp-2] [ RUN      ] service_client.client_scope_consistency_regression_test
[test_client_scope_consistency_client_cpp-2] creating first client
[test_client_scope_consistency_client_cpp-2] sending first request
[test_client_scope_consistency_server_cpp-1] responding to request
[test_client_scope_consistency_client_cpp-2] received first result
[test_client_scope_consistency_client_cpp-2] received correct result
[test_client_scope_consistency_client_cpp-2] destroying first client
[test_client_scope_consistency_client_cpp-2] creating second client
[test_client_scope_consistency_client_cpp-2] sending second request
[ERROR] [test_client_scope_consistency_server_cpp-1]: process has died [pid 1296, exit code 3221226505, cmd 'C:/ci/ws/build/test_rclcpp/Debug/test_client_scope_consistency_server_cpp.exe'].
[test_client_scope_consistency_client_cpp-2] second result not received: TIMEOUT (2)
[test_client_scope_consistency_client_cpp-2] C:\ci\ws\src\ros2\system_tests\test_rclcpp\test\test_client_scope_consistency_client.cpp(120): error: Expected equality of these values:
[test_client_scope_consistency_client_cpp-2]   ret2
[test_client_scope_consistency_client_cpp-2]     Which is: TIMEOUT (2)
[test_client_scope_consistency_client_cpp-2]   ret1
[test_client_scope_consistency_client_cpp-2]     Which is: SUCCESS (0)
[test_client_scope_consistency_client_cpp-2] Both clients should have the same behavior.
[test_client_scope_consistency_client_cpp-2] 
[test_client_scope_consistency_client_cpp-2] [  FAILED  ] service_client.client_scope_consistency_regression_test (5415 ms)

And this one seems to be a crash too

[ RUN      ] test_two_service_calls.two_service_calls
Sending two requests...
Waiting for first reply...
Received first reply
Waiting for second reply...
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\xtree(182) : Assertion failed: cannot dereference value-initialized map/set iterator
-- run_test.py: return code 3221226505
-- run_test.py: generate result file 'C:/ci/ws/build/test_rclcpp/test_results/test_rclcpp/gtest_multiple_service_calls__rmw_connextdds.gtest.xml' with failed test
-- run_test.py: verify result file 'C:/ci/ws/build/test_rclcpp/test_results/test_rclcpp/gtest_multiple_service_calls__rmw_connextdds.gtest.xml'
C:\ci\ws\install\Lib\site-packages\ament_cmake_test\__init__.py:133: ResourceWarning: unclosed file <_io.BufferedReader name=4>
  return _run_test(parser, args, failure_result_file, output_handle)
ResourceWarning: Enable tracemalloc to get the object allocation traceback

Is there a new issue with connextdds on windows?

projectroot.gtest_multiple_service_calls__rmw_connextdds
projectroot.gtest_local_parameters__rmw_connextdds
projectroot.test_parameter_server_cpp__rmw_connextdds
projectroot.test_services_cpp__rmw_connextdds
projectroot.test_client_scope_cpp__rmw_connextdds
projectroot.test_client_scope_consistency_cpp__rmw_connextdds
projectroot.test_tutorial_use_logger_service__rmw_connextdds
projectroot.test_service__rmw_connextdds
projectroot.test_composable_player__rmw_connextdds
projectroot.test_play_services__rmw_connextdds
projectroot.test_requester_replier__rclpy__rmw_connextdds
projectroot.test_action_client_server__rclpy__rmw_connextdds
projectroot.test_requester_replier__rclpy__rclcpp__rmw_connextdds
projectroot.test_action_client_server__rclpy__rclcpp__rmw_connextdds
projectroot.test_requester_replier__rclcpp__rclpy__rmw_connextdds
projectroot.test_action_client_server__rclcpp__rclpy__rmw_connextdds
projectroot.test_requester_replier__rclcpp__rmw_connextdds
projectroot.test_action_client_server__rclcpp__rmw_connextdds
test_rclcpp.gtest_multithreaded__rmw_connextdds.gtest.missing_result
test_rclcpp.gtest_local_parameters__rmw_connextdds.gtest.missing_result
test_rclcpp.gtest_multiple_service_calls__rmw_connextdds.gtest.missing_result
test_rclcpp.test_parameter_server_cpp__rmw_connextdds.xunit.missing_result
test_rclcpp.test_services_cpp__rmw_connextdds.xunit.missing_result
test_rclcpp.test_client_scope_cpp__rmw_connextdds.xunit.missing_result

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See ros2/rmw_connextdds#146 ; most of these are a regression in rmw_connextdds.

else()
set(ARG_TIMEOUT 60)
endif()
endif()
if(NOT ARG_TIMEOUT GREATER 0)
message(FATAL_ERROR "ament_add_test() the TIMEOUT argument must be a "
Expand Down