Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Windows] offload source spaces into order_paths.py #1113

Merged
merged 5 commits into from
Aug 12, 2020

Conversation

seanyen
Copy link
Contributor

@seanyen seanyen commented Aug 7, 2020

One limitation of Windows command processor is that the full command line cannot exceed 8191 characters. And in the code path where catkin sorts a list of paths, it is passing all the workspaces plus source spaces by command line to order_paths.py. When the workspaces is huge enough, it will render as an error - The command line is too long..

To workaround this limitation, this patch is proposing to synthesize the source spaces in order_paths.py instead of passing the long string by command line.

Also, please consider to back-port to kinetic-devel if it makes sense. Thanks!

=ON -DCMAKE_BUILD_TYPE=RelWithDebInfo -G NMake Makefiles in 'c:\workspace\noetic_rel\build_isolated\rosbag'
-- The C compiler identification is MSVC 19.26.28806.0
-- The CXX compiler identification is MSVC 19.26.28806.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.26.28801/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.26.28801/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using CATKIN_DEVEL_PREFIX: C:/workspace/noetic_rel/devel_isolated
-- Using CMAKE_PREFIX_PATH: C:/workspace/noetic_rel/install_isolated
-- This workspace overlays: C:/workspace/noetic_rel/install_isolated
-- Found PythonInterp: C:/opt/ros/xxxx/x64/python.exe (found suitable version "3.8.3", minimum required is "3")
-- Using PYTHON_EXECUTABLE: C:/opt/ros/xxxx/x64/python.exe
-- Using default Python package layout
-- Found PY_em: C:\opt\ros\xxxx\x64\Lib\site-packages\em.py
-- Using empy: C:/opt/ros/xxxx/x64/Lib/site-packages/em.py
-- Using CATKIN_ENABLE_TESTING: ON
-- Call enable_testing()
-- Using CATKIN_TEST_RESULTS_DIR: C:/workspace/noetic_rel/build_isolated/rosbag/test_results
-- nosetests not found, Python tests can not be run (try installing package 'python3-nose')
-- catkin 0.8.8
-- BUILD_SHARED_LIBS is on
The command line is too long.
CMake Error at C:/workspace/noetic_rel/install_isolated/share/catkin/cmake/safe_execute_process.cmake:11 (message):

  execute_process(C:/workspace/noetic_rel/build_isolated/rosbag/catkin_generated/env_cached.bat
  "C:/opt/ros/xxxx/x64/python.exe"
  "C:/workspace/noetic_rel/install_isolated/share/catkin/cmake/order_paths.py"
  "C:/workspace/noetic_rel/build_isolated/rosbag/catkin_generated/ordered_paths.cmake"
  "--paths-to-order"
  "C:/workspace/noetic_rel/src/ros_comm/rosbag_storage/include"
  "C:/opt/ros/xxxx/x64/include"
  "C:/workspace/noetic_rel/src/pluginlib/include"
  "C:/workspace/noetic_rel/src/class_loader/include"
  "C:/workspace/noetic_rel/src/rosconsole/include"
  "C:/workspace/noetic_rel/src/roscpp_core/cpp_common/include"
  "C:/workspace/noetic_rel/src/roscpp_core/rostime/include"
  "C:/workspace/noetic_rel/src/ros/roslib/include"
  "C:/workspace/noetic_rel/src/rospack/include"
  "C:/workspace/noetic_rel/src/ros_comm/roslz4/include"
  "C:/workspace/noetic_rel/devel_isolated/include"
  "C:/workspace/noetic_rel/src/ros_comm/roscpp/include"
  "C:/workspace/noetic_rel/devel_isolated/include/ros"
  "C:/workspace/noetic_rel/src/roscpp_core/roscpp_serialization/include"
  "C:/workspace/noetic_rel/src/roscpp_core/roscpp_traits/include"
  "C:/workspace/noetic_rel/src/std_msgs/include"
  "C:/workspace/noetic_rel/src/ros_comm/xmlrpcpp/include"
  "C:/workspace/noetic_rel/src/ros_comm/xmlrpcpp/include/xmlrpcpp"
  "C:/workspace/noetic_rel/src/ros_comm/topic_tools/include" "--prefixes"
  "C:/workspace/noetic_rel/devel_isolated"
  "C:/workspace/noetic_rel/src/catkin" "C:/workspace/noetic_rel/src/genmsg"
  "C:/workspace/noetic_rel/src/gencpp" "C:/workspace/noetic_rel/src/geneus"
  "C:/workspace/noetic_rel/src/genlisp"
  "C:/workspace/noetic_rel/src/gennodejs" "C:/workspace/noetic_rel/src/genpy"
  "C:/workspace/noetic_rel/src/bond_core/bond_core"
  "C:/workspace/noetic_rel/src/cmake_modules"
  "C:/workspace/noetic_rel/src/class_loader"
  "C:/workspace/noetic_rel/src/common_msgs/common_msgs"
  "C:/workspace/noetic_rel/src/common_tutorials/common_tutorials"
  "C:/workspace/noetic_rel/src/roscpp_core/cpp_common"
  "C:/workspace/noetic_rel/src/metapackages/desktop"
  "C:/workspace/noetic_rel/src/diagnostics/diagnostics"
  "C:/workspace/noetic_rel/src/executive_smach/executive_smach"
  "C:/workspace/noetic_rel/src/geometry/geometry"
  "C:/workspace/noetic_rel/src/geometry_tutorials/geometry_tutorials"
  "C:/workspace/noetic_rel/src/gl_dependency"
  "C:/workspace/noetic_rel/src/joint_state_publisher/joint_state_publisher_gui"
  "C:/workspace/noetic_rel/src/media_export"
  "C:/workspace/noetic_rel/src/message_generation"
  "C:/workspace/noetic_rel/src/message_runtime"
  "C:/workspace/noetic_rel/src/ros/mk"
  "C:/workspace/noetic_rel/src/nodelet_core/nodelet_core"
  "C:/workspace/noetic_rel/src/qt_gui_core/qt_dotgraph"
  "C:/workspace/noetic_rel/src/qt_gui_core/qt_gui"
  "C:/workspace/noetic_rel/src/qt_gui_core/qt_gui_py_common"
  "C:/workspace/noetic_rel/src/qwt_dependency"
  "C:/workspace/noetic_rel/src/metapackages/robot"
  "C:/workspace/noetic_rel/src/ros/ros"
  "C:/workspace/noetic_rel/src/metapackages/ros_base"
  "C:/workspace/noetic_rel/src/ros_comm/ros_comm"
  "C:/workspace/noetic_rel/src/metapackages/ros_core"
  "C:/workspace/noetic_rel/src/ros_environment"
  "C:/workspace/noetic_rel/src/ros_tutorials/ros_tutorials"
  "C:/workspace/noetic_rel/src/rosbag_migration_rule"
  "C:/workspace/noetic_rel/src/ros/rosbash"
  "C:/workspace/noetic_rel/src/ros/rosboost_cfg"
  "C:/workspace/noetic_rel/src/ros/rosbuild"
  "C:/workspace/noetic_rel/src/ros/rosclean"
  "C:/workspace/noetic_rel/src/roscpp_core/roscpp_core"
  "C:/workspace/noetic_rel/src/roscpp_core/roscpp_traits"
  "C:/workspace/noetic_rel/src/ros/roscreate"
  "C:/workspace/noetic_rel/src/ros_comm/rosgraph"
  "C:/workspace/noetic_rel/src/ros/roslang"
  "C:/workspace/noetic_rel/src/roslint" "C:/workspace/noetic_rel/src/roslisp"
  "C:/workspace/noetic_rel/src/ros/rosmake"
  "C:/workspace/noetic_rel/src/ros_comm/rosmaster"
  "C:/workspace/noetic_rel/src/rospack"
  "C:/workspace/noetic_rel/src/ros/roslib"
  "C:/workspace/noetic_rel/src/ros_comm/rosparam"
  "C:/workspace/noetic_rel/src/ros_comm/rospy"
  "C:/workspace/noetic_rel/src/ros_comm/rosservice"
  "C:/workspace/noetic_rel/src/roscpp_core/rostime"
  "C:/workspace/noetic_rel/src/roscpp_core/roscpp_serialization"
  "C:/workspace/noetic_rel/src/python_qt_binding"
  "C:/workspace/noetic_rel/src/ros_comm/roslaunch"
  "C:/workspace/noetic_rel/src/ros/rosunit"
  "C:/workspace/noetic_rel/src/angles"
  "C:/workspace/noetic_rel/src/rosconsole"
  "C:/workspace/noetic_rel/src/pluginlib"
  "C:/workspace/noetic_rel/src/qt_gui_core/qt_gui_cpp"
  "C:/workspace/noetic_rel/src/resource_retriever"
  "C:/workspace/noetic_rel/src/rosconsole_bridge"
  "C:/workspace/noetic_rel/src/ros_comm/roslz4"
  "C:/workspace/noetic_rel/src/ros_comm/rostest"
  "C:/workspace/noetic_rel/src/rqt_action"
  "C:/workspace/noetic_rel/src/rqt_bag/rqt_bag"
  "C:/workspace/noetic_rel/src/rqt_bag/rqt_bag_plugins"
  "C:/workspace/noetic_rel/src/rqt_common_plugins"
  "C:/workspace/noetic_rel/src/rqt_console"
  "C:/workspace/noetic_rel/src/rqt_dep"
  "C:/workspace/noetic_rel/src/rqt_graph"
  "C:/workspace/noetic_rel/src/rqt/rqt_gui"
  "C:/workspace/noetic_rel/src/rqt_logger_level"
  "C:/workspace/noetic_rel/src/rqt_moveit"
  "C:/workspace/noetic_rel/src/rqt_msg"
  "C:/workspace/noetic_rel/src/rqt_nav_view"
  "C:/workspace/noetic_rel/src/rqt_plot"
  "C:/workspace/noetic_rel/src/rqt_pose_view"
  "C:/workspace/noetic_rel/src/rqt_publisher"
  "C:/workspace/noetic_rel/src/rqt_py_console"
  "C:/workspace/noetic_rel/src/rqt_robot_dashboard"
  "C:/workspace/noetic_rel/src/rqt_robot_monitor"
  "C:/workspace/noetic_rel/src/rqt_robot_plugins"
  "C:/workspace/noetic_rel/src/rqt_robot_steering"
  "C:/workspace/noetic_rel/src/rqt_runtime_monitor"
  "C:/workspace/noetic_rel/src/rqt_service_caller"
  "C:/workspace/noetic_rel/src/rqt_shell"
  "C:/workspace/noetic_rel/src/rqt_srv"
  "C:/workspace/noetic_rel/src/rqt_tf_tree"
  "C:/workspace/noetic_rel/src/rqt_top"
  "C:/workspace/noetic_rel/src/rqt_topic"
  "C:/workspace/noetic_rel/src/rqt_web"
  "C:/workspace/noetic_rel/src/executive_smach/smach"
  "C:/workspace/noetic_rel/src/bond_core/smclib"
  "C:/workspace/noetic_rel/src/std_msgs"
  "C:/workspace/noetic_rel/src/common_msgs/actionlib_msgs"
  "C:/workspace/noetic_rel/src/bond_core/bond"
  "C:/workspace/noetic_rel/src/common_msgs/diagnostic_msgs"
  "C:/workspace/noetic_rel/src/common_msgs/geometry_msgs"
  "C:/workspace/noetic_rel/src/geometry/eigen_conversions"
  "C:/workspace/noetic_rel/src/geometry/kdl_conversions"
  "C:/workspace/noetic_rel/src/common_msgs/nav_msgs"
  "C:/workspace/noetic_rel/src/ros_comm_msgs/rosgraph_msgs"
  "C:/workspace/noetic_rel/src/ros_comm/rosmsg"
  "C:/workspace/noetic_rel/src/rqt/rqt_py_common"
  "C:/workspace/noetic_rel/src/common_msgs/shape_msgs"
  "C:/workspace/noetic_rel/src/executive_smach/smach_msgs"
  "C:/workspace/noetic_rel/src/ros_comm_msgs/std_srvs"
  "C:/workspace/noetic_rel/src/geometry2/tf2_msgs"
  "C:/workspace/noetic_rel/src/geometry2/tf2"
  "C:/workspace/noetic_rel/src/common_msgs/trajectory_msgs"
  "C:/workspace/noetic_rel/src/control_msgs"
  "C:/workspace/noetic_rel/src/urdf/urdf_parser_plugin"
  "C:/workspace/noetic_rel/src/common_msgs/visualization_msgs"
  "C:/workspace/noetic_rel/src/visualization_tutorials/visualization_tutorials"
  "C:/workspace/noetic_rel/src/metapackages/viz"
  "C:/workspace/noetic_rel/src/webkit_dependency"
  "C:/workspace/noetic_rel/src/ros_comm/xmlrpcpp"
  "C:/workspace/noetic_rel/src/ros_comm/roscpp"
  "C:/workspace/noetic_rel/src/bond_core/bondcpp"
  "C:/workspace/noetic_rel/src/bond_core/bondpy"
  "C:/workspace/noetic_rel/src/nodelet_core/nodelet"
  "C:/workspace/noetic_rel/src/common_tutorials/nodelet_tutorial_math"
  "C:/workspace/noetic_rel/src/common_tutorials/pluginlib_tutorials"
  "C:/workspace/noetic_rel/src/ros_tutorials/roscpp_tutorials"
  "C:/workspace/noetic_rel/src/ros_comm/rosout"
  "C:/workspace/noetic_rel/src/diagnostics/diagnostic_aggregator"
  "C:/workspace/noetic_rel/src/diagnostics/diagnostic_updater"
  "C:/workspace/noetic_rel/src/diagnostics/diagnostic_common_diagnostics"
  "C:/workspace/noetic_rel/src/dynamic_reconfigure"
  "C:/workspace/noetic_rel/src/filters"
  "C:/workspace/noetic_rel/src/joint_state_publisher/joint_state_publisher"
  "C:/workspace/noetic_rel/src/ros_comm/message_filters"
  "C:/workspace/noetic_rel/src/ros_comm/rosbag_storage"
  "C:/workspace/noetic_rel/src/ros_comm/rosnode"
  "C:/workspace/noetic_rel/src/ros_tutorials/rospy_tutorials"
  "C:/workspace/noetic_rel/src/ros_comm/rostopic"
  "C:/workspace/noetic_rel/src/rqt/rqt_gui_cpp"
  "C:/workspace/noetic_rel/src/rqt/rqt_gui_py"
  "C:/workspace/noetic_rel/src/rqt_reconfigure"
  "C:/workspace/noetic_rel/src/diagnostics/self_test"
  "C:/workspace/noetic_rel/src/executive_smach/smach_ros"
  "C:/workspace/noetic_rel/src/geometry2/tf2_py"
  "C:/workspace/noetic_rel/src/ros_comm/topic_tools"
  "C:/workspace/noetic_rel/src/ros_comm/rosbag"
  "C:/workspace/noetic_rel/install_isolated") returned error code 1
Call Stack (most recent call first):
  C:/workspace/noetic_rel/install_isolated/share/catkin/cmake/list_insert_in_workspace_order.cmake:29 (safe_execute_process)
  C:/workspace/noetic_rel/install_isolated/share/catkin/cmake/catkinConfig.cmake:108 (list_insert_in_workspace_order)
  CMakeLists.txt:8 (find_package)


-- Configuring incomplete, errors occurred!

@seanyen
Copy link
Contributor Author

seanyen commented Aug 11, 2020

@dirk-thomas This is ready for review.

@dirk-thomas dirk-thomas changed the title [noetic][Windows] offload source spaces into order_paths.py [Windows] offload source spaces into order_paths.py Aug 12, 2020
@dirk-thomas
Copy link
Member

Thanks for the patch.

@dirk-thomas dirk-thomas merged commit fc78d87 into ros:noetic-devel Aug 12, 2020
dirk-thomas added a commit that referenced this pull request Aug 12, 2020
* offload source spaces into order_paths.py

* missing import.

* Fixing the missing pieces.

* separate imports and fix error message

* use list.append instead

Co-authored-by: Dirk Thomas <dirk-thomas@users.noreply.github.com>
@dirk-thomas
Copy link
Member

Cherry-picked to kinetic-devel in 9d7e4b4.

seanyen added a commit to ms-iot/catkin that referenced this pull request Aug 31, 2020
* offload source spaces into order_paths.py

* missing import.

* Fixing the missing pieces.

* separate imports and fix error message

* use list.append instead

Co-authored-by: Dirk Thomas <dirk-thomas@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants