RoMu4o: A Robotic Manipulation Unit for Orchard Operations

RoMu4o, is a robotic manipulation unit for orchard operations, designed to provide an autonomous solution for continuous orchard monitoring. This robotic system is capable of navigating unstructured agricultural environments for crop monitoring and proximal hyperspectral leaf sensing. The hardware is comprised of 1) 6-DOF robotic manipulator 2) RGBD camera 3) two-finger gripper end-effector integrated with a hyperspectral sensing system 4) an independent VIS-NIR light source for referencing and self-calibration 5) crawler-type ground robot 6) electric and control boxes, hardware drivers, portable batteries, processing unit, Ethernet hub, and auxiliary hardware needed for the UGV's navigation.

Repository structure

UCM-AgBot-ROS2/
├── agbot/                   # Main robot packages
│   │
│   ├── robot_bringup/       # RoMu4o launch files and setup
│   ├── robot_control/       # Mobile Robot control 
│   ├── robot_description/   # URDF and robot model 
│   └── robot_simulator/     # Simulation setup 
│
├── cobot/                   # TM5 collaborative arm 
├── custom_interfaces/       # Custom messages and services
├── devices/                 # Driver packages
├── sensors/                 # Driver packages
├── vision/                  # Perception module
├── .gitignore               
├── .gitmodules              # Forked Repositories
├── LICENSE                  
└── README.md                # Project documentation

Requirements

• Ubuntu 20.04
• ROS2 (Foxy)
• MoveIt2 (Foxy)
• NAV2 (Foxy)

Forked Repositories

• RealSense D435i
• TM/OMRON TM5M-900
• Vectornav
• Slamtec

Installation

1. Create a ROS2 workspace:

   mkdir -p ~/ros2_ws/src
   cd ~/ros2_ws/src

2. Clone the repository:

   git clone --recurse-submodules https://github.com/mehradmrt/UCM-AgBot-ROS2

3. Return to the workspace root:

   cd ../

4. Install ROS dependencies:

   rosdep update
   rosdep install --from-paths src --ignore-src -r -y

5. Build the workspace:

   colcon build

6. Source the environment:

   source /opt/ros/foxy/setup.bash
   source install/setup.bash

---

Usage

RoMu4o Navigation

1. Launch sensors and devices:

   ros2 launch robot_bringup sensors.launch.py imu:=true gnss:=true lidar2d:=true realsense:=true encoders:=true

2. Launch robot bringup:

   ros2 launch robot_bringup bringup.launch.py

3. Launch robot localization:

   ros2 launch robot_bringup localization.launch.py

4. Launch robot navigation:

   ros2 launch robot_bringup navigation.launch.py

RoMu4o Perception and Leaf Manipulation

1. Launch sensors and devices:

   ros2 launch robot_bringup sensors_arm.launch.py

2. Launch the robotic manipulation pipeline:

   ros2 launch tm_moveit_cpp_demo tm5-900_run_moveit_cpp.launch.py robot_ip:=192.168.1.19

3. Run the perception pipeline:

   ros2 run leaf_extraction instance_segmentation

---

Reference

Mortazavi, M., Cappelleri, D.J., Ehsani, R., 2025. RoMu4o: A Robotic Manipulation Unit
For Orchard Operations Automating Proximal Hyperspectral Leaf Sensing. arXiv preprint 
arXiv:2409.19786.