Skip to content

Latest commit

 

History

History
93 lines (70 loc) · 2.76 KB

README.md

File metadata and controls

93 lines (70 loc) · 2.76 KB
Raw

RL_DRONE

Diploma work based on PyBullet physics simulator. Target of this work is to develop position control system using only following input data:

  • linear and angular velocity in drone local coordinates
  • projected gravity [orientation analog]
  • position displacement

This means drone can be operated from integrated imu data without additional velocity postprocessing from local to global coordinates

Also this work contains absolute position controllers from https://github.com/utiasDSL/gym-pybullet-drones.git

imu flight

Main contributions

  • IMU Sensor model gym_pybullet_drones/devices/sensors/imu.py
  • Physical drone model gym_pybullet_drones/assets/custom.urdf
  • Environment for system learning gym_pybullet_drones/envs/single_agent_rl/hover
  • Sensor asynchronous clocking Added model of physical drone

Drone Model

  • Calculated inertia from physical analog using Autodesk Inventor
  • Estimated Lift and Torque coefficients from open database, duplicated in gym_pybullet_drones/thrust_torque_estimation

imu flight

imu flight

Environment

Target displacement clipped by absolute max value equal 1, so system has direction information when displacement greater than 1 and has exact measure when displacement less than 1 Sigmoid function with out scaling used to convert prediction values to range of motor speed [0, max_speed] Reward function motivates agent fly to center and angular rate reward penalizes keeps policy from extra rotation

$$reward = closeness*angles \\$$ $$closeness = \frac{||displacement||}{radius_{max}} \\$$ $$angles = exp(||\omega||*0.1)$$

During training drone initialized with random position from uniform distribution in sphere with radius 1.5 meters

IMU sensor

measurement simulation based on two-component noise model

$$measurement(t) = estimation(t) + bias(t) + noise \\$$ $$bias(t) =bias(t-1)+N(1,\sigma_{r}*\sqrt{dt})\\$$ $$noise = N(1,\frac{\sigma_{d}}{\sqrt{dt}})$$

Installation

git clone https://github.com/L-ED/RL_Drone.git
cd gym-pybullet-drones/

conda create -n drones python=3.10
conda activate drones

pip3 install --upgrade pip
pip3 install -e . # if needed, `sudo apt install build-essentials` to install `gcc` and build `pybullet`

Use

RL Position Control from local coordinates

cd gym_pybullet_drones/examples/
python hover/hover_learn_multienv.py

PID position control example

cd gym_pybullet_drones/examples/
python3 pid.py

Stable-baselines3 PPO RL example

cd gym_pybullet_drones/examples/
python3 learn.py