Skip to content

The CartPole3D environment solved using reinforcement learning.

Notifications You must be signed in to change notification settings

PierreExeter/cartpole3d_ros

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

CartPole3d

The Cart Pole problem is solved in 3D using Reinforcement Learning (RL). The cartpole3d environment is implemented in ROS with the package openai_ros. The RL algorithms used for training are DQN, A2C, ACKTR, PPO, TRPO implemented in the stable-baselines library.

Installation

  1. Install ROS Melodic

  2. Install the openai_ros package.

cd ~/catkin_make/src/
git clone https://bitbucket.org/theconstructcore/openai_ros.git
cd ~/catkin_make
catkin_make
source devel/setup.bash
rosdep install openai_ros
  1. Install Gym
pip install gym
  1. Install stable-baselines

  2. Install cartpole3d

cd ~/catkin_make/src/
git clone https://github.com/PierreExeter/cartpole3d_ros.git
cd ~/catkin_make
catkin_make
  1. Make the scripts executable
chmod +x ~/catkin_make/src/cartpole3d/scripts/cartpole3d_random.py
chmod +x ~/catkin_make/src/cartpole3d/scripts/cartpole3d_dqn.py
chmod +x ~/catkin_make/src/cartpole3d/scripts/cartpole3d_trpo.py
chmod +x ~/catkin_make/src/cartpole3d/scripts/cartpole3d_train_all.py
chmod +x ~/catkin_make/src/cartpole3d/scripts/cartpole3d_enjoy_all.py
  1. Create simulation workspace
mkdir -p ~/simluation_ws/src
cd ..
catkin_make
source devel/setup.bash
rospack profile

Test the installation

roslaunch cartpole3d start_training_cartpole3d_random.launch 

You should see the cartpole executing random actions in Gazebo. cartpole

Perform training

roslaunch cartpole3d start_training_cartpole3d_train_all.launch 

Visualise training metrics in Tensorboard

cd ~/catkin_make/src/cartpole3d/results/
tensorboard --logdir=A2C:tensorboard_logs/A2C/,ACKTR:tensorboard_logs/ACKTR/,PPO2:tensorboard_logs/PPO2/,TRPO:tensorboard_logs/TRPO/

Observe trained agents

roslaunch cartpole3d start_training_cartpole3d_enjoy_all.launch 

Plot the results

cd ~/catkin_make/src/cartpole3d/scripts/
python plot_reward.py

Supported systems

Tested with:

  • Ubuntu 18.04
  • Python 2.7
  • ROS Melodic
  • Gazebo 9.12

About

The CartPole3D environment solved using reinforcement learning.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published