Increase test timeout on Windows #522
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Alejandro Hernández Cordero <ahcorde@gmail.com>
sloretz
requested changes
Apr 5, 2024
| @@ -73,7 +73,11 @@ function(ament_add_test testname) | |||
| set(ARG_RUNNER "${ament_cmake_test_DIR}/run_test.py") | |||
| endif() | |||
| if(NOT ARG_TIMEOUT) | |||
| set(ARG_TIMEOUT 60) | |||
| if(WIN32) | |||
| set(ARG_TIMEOUT 180) | |||
There was a problem hiding this comment.
This affects the timeouts of all tests, which feels very broad. It could mask issues that cause all tests to take longer. If there are particular tests taking longer than 60 seconds then we could increase those with an issue to look at them closer later.
In the linked build I'm not sure that increasing the timeout will the test failures. It looks like there might be a real bug. For example this job seems like it might be timing out because the server died
[test_client_scope_consistency_client_cpp-2] [ RUN ] service_client.client_scope_consistency_regression_test
[test_client_scope_consistency_client_cpp-2] creating first client
[test_client_scope_consistency_client_cpp-2] sending first request
[test_client_scope_consistency_server_cpp-1] responding to request
[test_client_scope_consistency_client_cpp-2] received first result
[test_client_scope_consistency_client_cpp-2] received correct result
[test_client_scope_consistency_client_cpp-2] destroying first client
[test_client_scope_consistency_client_cpp-2] creating second client
[test_client_scope_consistency_client_cpp-2] sending second request
[ERROR] [test_client_scope_consistency_server_cpp-1]: process has died [pid 1296, exit code 3221226505, cmd 'C:/ci/ws/build/test_rclcpp/Debug/test_client_scope_consistency_server_cpp.exe'].
[test_client_scope_consistency_client_cpp-2] second result not received: TIMEOUT (2)
[test_client_scope_consistency_client_cpp-2] C:\ci\ws\src\ros2\system_tests\test_rclcpp\test\test_client_scope_consistency_client.cpp(120): error: Expected equality of these values:
[test_client_scope_consistency_client_cpp-2] ret2
[test_client_scope_consistency_client_cpp-2] Which is: TIMEOUT (2)
[test_client_scope_consistency_client_cpp-2] ret1
[test_client_scope_consistency_client_cpp-2] Which is: SUCCESS (0)
[test_client_scope_consistency_client_cpp-2] Both clients should have the same behavior.
[test_client_scope_consistency_client_cpp-2]
[test_client_scope_consistency_client_cpp-2] [ FAILED ] service_client.client_scope_consistency_regression_test (5415 ms)
And this one seems to be a crash too
[ RUN ] test_two_service_calls.two_service_calls
Sending two requests...
Waiting for first reply...
Received first reply
Waiting for second reply...
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\xtree(182) : Assertion failed: cannot dereference value-initialized map/set iterator
-- run_test.py: return code 3221226505
-- run_test.py: generate result file 'C:/ci/ws/build/test_rclcpp/test_results/test_rclcpp/gtest_multiple_service_calls__rmw_connextdds.gtest.xml' with failed test
-- run_test.py: verify result file 'C:/ci/ws/build/test_rclcpp/test_results/test_rclcpp/gtest_multiple_service_calls__rmw_connextdds.gtest.xml'
C:\ci\ws\install\Lib\site-packages\ament_cmake_test\__init__.py:133: ResourceWarning: unclosed file <_io.BufferedReader name=4>
return _run_test(parser, args, failure_result_file, output_handle)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Is there a new issue with connextdds on windows?
projectroot.gtest_multiple_service_calls__rmw_connextdds
projectroot.gtest_local_parameters__rmw_connextdds
projectroot.test_parameter_server_cpp__rmw_connextdds
projectroot.test_services_cpp__rmw_connextdds
projectroot.test_client_scope_cpp__rmw_connextdds
projectroot.test_client_scope_consistency_cpp__rmw_connextdds
projectroot.test_tutorial_use_logger_service__rmw_connextdds
projectroot.test_service__rmw_connextdds
projectroot.test_composable_player__rmw_connextdds
projectroot.test_play_services__rmw_connextdds
projectroot.test_requester_replier__rclpy__rmw_connextdds
projectroot.test_action_client_server__rclpy__rmw_connextdds
projectroot.test_requester_replier__rclpy__rclcpp__rmw_connextdds
projectroot.test_action_client_server__rclpy__rclcpp__rmw_connextdds
projectroot.test_requester_replier__rclcpp__rclpy__rmw_connextdds
projectroot.test_action_client_server__rclcpp__rclpy__rmw_connextdds
projectroot.test_requester_replier__rclcpp__rmw_connextdds
projectroot.test_action_client_server__rclcpp__rmw_connextdds
test_rclcpp.gtest_multithreaded__rmw_connextdds.gtest.missing_result
test_rclcpp.gtest_local_parameters__rmw_connextdds.gtest.missing_result
test_rclcpp.gtest_multiple_service_calls__rmw_connextdds.gtest.missing_result
test_rclcpp.test_parameter_server_cpp__rmw_connextdds.xunit.missing_result
test_rclcpp.test_services_cpp__rmw_connextdds.xunit.missing_result
test_rclcpp.test_client_scope_cpp__rmw_connextdds.xunit.missing_result
There was a problem hiding this comment.
See ros2/rmw_connextdds#146 ; most of these are a regression in rmw_connextdds.
|
I created this other PR on ament_uncrustify ament/ament_lint#485 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I have seen many timeouts in particular on Windows (this build has more than 20 timeouts). This PR increases the timeout of WIndows test 3 times. 🤞