Skip to content

Master Thesis (2021, KAIST): The development of the scale-aware monocular depth estimation aided monocular visual SLAM system for real-time robot navigation

Notifications You must be signed in to change notification settings

surfii3z/jy_master_thesis

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

14 Commits
 
 
 
 
 
 

Repository files navigation

Master thesis

This thesis work was done as a part of research at Unmanned System Research Group (USRG) led by Professor David Hyunchul Shim, KAIST, South Korea.

General information

Important materials

  1. Presentation
  2. Dissertation

Title

The development of the scale-aware monocular depth estimation aided monocular visual SLAM system for real-time robot navigation

Motivation

Human pilots could fly any drone by relying only on the monocular visual signal in the goggle. In this work, we want to use high performance computer to replace the human brain to navigate the drone by relying only on the monocular camera sensor.

Objective

To enable the mobile robot to autonomously navigate in an unknown environment using only a monocular camera as its sensor

Approaches

  1. RGB-Deep D visual SLAM: Augmented the monocular depth estimation (Packnet-sfm) to provide the pseudo depth information to the monocular visual SLAM (ORB_SLAM2) to solves its inherent problems: scale-ambiguity, tracking robustness, and map initialization delay (See presentation file page 5).

RGB_Deep_D_visual_SLAM

  1. Navigation stack: Following the see-think-act scheme for autonomous mobile robot, we integrated visual SLAM (see, RGB-Deep D SLAM, localization/ mapping), local path planner (think, Fast-Planner, planning/ obstacle avoidance), and waypoint tracking controller (act, PD controller, control).
Module Employed system
Visual SLAM ORB_SLAM2
Monocular Depth Estimation packnet-sfm
Local path planner Fast-Planner
Controller PD controller

see_think_act

Navigation_stack

Dependencies

We make 3 seperated workspaces. The installation instruction for each package is in the correspoding link.

  1. catkin_ws
catkin_ws/src
├── [drone_controller](https://github.com/surfii3z/drone_controller/tree/thesis)
├── [Fast-Planner](https://github.com/surfii3z/Fast-Planner/tree/thesis)
├── [image_proc](https://github.com/surfii3z/image_proc/tree/thesis)
├── [image_undistort](https://github.com/surfii3z/image_undistort/tree/thesis)
└── [tello_driver](https://github.com/surfii3z/tello_driver/tree/thesis)
  1. packnet_ros_ws
packnet_ros_ws/src/
├── [packnet_sfm_ros](https://github.com/surfii3z/packnet_sfm_ros)
└── vision_opencv
  1. ORB_SLAM2
[ORB_SLAM2](https://github.com/surfii3z/ORB_SLAM2/tree/thesis)

How to run

I experimented this system on DGX-Station (V100), tested with Ubuntu 18.04 LTS, python 3.6.9, TensorRT-7.1.3.4, PyTorch 1.4.0, Cuda 11.0 and CuDNN 8.0.5. We need to use 8 terminals to run

# -------------------------
# 00_roscore
roscore

# -------------------------
# 01_depth_estimation_module
# pwd: /path/to/packnet_ros_ws
source install/setup.bash --extend
rosrun packnet_sfm_ros trt_packnet_node

# -------------------------
# 02_tello_driver
# pwd: /path/to/catkin_ws
source devel/setup.bash
roslaunch tello_driver thesis_tello_node.launch

# -------------------------
# 03_visual_SLAM
# pwd: /path/to/ORB_SLAM2/ros
source devel/setup.bash
roslaunch orb_slam2 rgbd_tello_crop.launch

# -------------------------
# 04_path_planner_visualization
# pwd: /path/to/catkin_ws
source devel/setup.bash
roslaunch plan_manage rviz.launch

# -------------------------
# 05_path_planner
# pwd: /path/to/catkin_ws
source devel/setup.bash
roslaunch plan_manage tello_kino_replan.launch


# -------------------------
# 06_drone_controller
# pwd: /path/to/catkin_ws
source devel/setup.bash
roslaunch drone_controller waypoint_controller.launch

# -------------------------
# 07_mission
# pwd: /path/to/catkin_ws
source devel/setup.bash
rosrun drone_controller thesis_mission.py

terminal

rosgraph

NOTE: The monocular depth estimation network must be trained based on the application.

Results

Please see this playlist on Youtube

About

Master Thesis (2021, KAIST): The development of the scale-aware monocular depth estimation aided monocular visual SLAM system for real-time robot navigation

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages