This work presents an extension of the initial OpenAI gym for robotics using ROS and Gazebo. A whitepaper about this work is available at https://arxiv.org/abs/1608.05742. Please use the following BibTex entry to cite our work:
@misc{1608.05742,
Author = {Iker Zamora and Nestor Gonzalez Lopez and Victor Mayoral Vilches and Alejandro Hernandez Cordero},
Title = {Extending the OpenAI Gym for robotics: a toolkit for reinforcement learning using ROS and Gazebo},
Year = {2016},
Eprint = {arXiv:1608.05742},
}
Visit erlerobotics/gym for more information and videos.
The following are some of the available gazebo environments:
| Name | Status | Description |
|---|---|---|
GazeboCircuit2TurtlebotLidar-v0 |
Maintained | A simple circuit with straight tracks and 90 degree turns. Highly discretized LIDAR readings are used to train the Turtlebot. Scripts implementing Q-learning and Sarsa can be found in the examples folder. |
GazeboCircuit2TurtlebotLidarNn-v0 |
A simple circuit with straight tracks and 90 degree turns. A LIDAR is used to train the Turtlebot. Scripts implementing DQN can be found in the examples folder. | |
GazeboCircuit2cTurtlebotCameraNnEnv-v0 |
A simple circuit with straight tracks and 90 degree turns with high contrast colors between the floor and the walls. A camera is used to train the Turtlebot. Scripts implementing DQN using CNN can be found in the examples folder. |
Build/fetch the container:
docker pull erlerobotics/gym-gazebo:latest # to fetch the container
# docker build -t gym-gazebo .Enter the container
docker run -it erlerobotics/gym-gazeboIf you wish to run examples that require plugins like cameras, create a fake screen with:
xvfb-run -s "-screen 0 1400x900x24" bash
If you have an equivalent release of Gazebo installed locally, you can connect to the gzserver inside the container using gzclient GUI by setting the address of the master URI to the containers public address.
export GAZEBO_MASTER_IP=$(sudo docker inspect --format '{{ .NetworkSettings.IPAddress }}' "id of running container")
export GAZEBO_MASTER_URI=$GAZEBO_MASTER_IP:11345
gzclient
Basic requirements:
- ROS Kinetic (
/rosversion: 1.12.7) - Gazebo 8.1.1
- Python 3.5.2
sudo pip3 install rospkg catkin_pkg
sudo apt-get install \
python3-defusedxml
ros-kinetic-octomap-msgs \
ros-kinetic-joy \
ros-kinetic-geodesy \
ros-kinetic-octomap-ros \
ros-kinetic-control-toolbox \
ros-kinetic-pluginlib \
ros-kinetic-trajectory-msgs \
ros-kinetic-control-msgs \
ros-kinetic-std-srvs \
ros-kinetic-nodelet \
ros-kinetic-urdf \
ros-kinetic-rviz \
ros-kinetic-kdl-conversions \
ros-kinetic-eigen-conversions \
ros-kinetic-tf2-sensor-msgs \
ros-kinetic-pcl-ros
git clone https://github.com/erlerobot/gym-gazebo
cd gym-gazebo
sudo pip3 install -e .If successful, expect something like this.
sudo pip3 install h5py
sudo apt-get install python3-skimage
# install Theano
cd ~/
git clone git://github.com/Theano/Theano.git
cd Theano/
sudo python3 setup.py develop
#install Keras
sudo pip3 install keras
Install the Robot Operating System via:
ros-indigo-desktop-full is the only recommended installation.
- Ubuntu: http://wiki.ros.org/indigo/Installation/Ubuntu
- Others: http://wiki.ros.org/indigo/Installation
- Setup your computer to accept software from packages.osrfoundation.org:
sudo sh -c 'echo "deb http://packages.osrfoundation.org/gazebo/ubuntu-stable `lsb_release -cs` main" > /etc/apt/sources.list.d/gazebo-stable.list'- Setup keys:
wget http://packages.osrfoundation.org/gazebo.key -O - | sudo apt-key add -- Install Gazebo:
sudo apt-get update
sudo apt-get remove .*gazebo.* && sudo apt-get update && sudo apt-get install gazebo7 libgazebo7-devgit clone https://github.com/erlerobot/gym-gazebo
cd gym-gazebo
sudo pip install -e .There are two options to install dependencies: automatic installation or step-by-step installation
Install dependencies running setup.bash. If you are going to use DQN with Keras, also install Keras and Theano.
cd gym_gazebo/envs/installation
bash setup.bashBefore running a environment, load the corresponding setup script. For example, to load the Turtlebot execute:
- Turtlebot
cd gym_gazebo/envs/installation
bash turtlebot_setup.bashNeeds to be updated, use automatic installation.
This part of the installation is required only for the environments using DQN.
# install dependencies
sudo apt-get install gfortran
# install sript specific dependencies (temporal)
sudo apt-get install python-skimage
# install Theano
git clone git://github.com/Theano/Theano.git
cd Theano/
sudo python setup.py develop
#isntall Keras
sudo pip install kerasdot_parser error fix:
sudo pip install --upgrade pydot
sudo pip install --upgrade pyparsingFollow the instructions here and change $PATH instead of $CUDA_ROOT.
Working on a clean installation of Ubuntu 14.04 using CUDA 7.5.
The following flags are needed in order to execute in GPU mode, using an alias is recommended.
THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32In the root directory of the repository:
sudo pip install -e .- Load the environment variables corresponding to the robot you want to launch. E.g. to load the Turtlebot:
cd gym_gazebo/envs/installation
bash turtlebot_setup.bashNote: all the setup scripts are available in gym_gazebo/envs/installation
- Run any of the examples available in
examples/. E.g.:
cd examples/scripts_turtlebot
python circuit2_turtlebot_lidar_qlearn.pyTo see what's going on in Gazebo during a simulation, simply run gazebo client:
gzclientDisplay a graph showing the current reward history by running the following script:
cd examples/utilities
python display_plot.pyHINT: use --help flag for more options.
Sometimes, after ending or killing the simulation gzserver and rosmaster stay on the background, make sure you end them before starting new tests.
We recommend creating an alias to kill those processes.
echo "alias k='killall -9 gzserver gzclient roslaunch rosmaster'" >> ~/.bashrc