Developmental Reinforcement Learning of Control Policy of a Quadcopter
  UAV with Thrust Vectoring Rotors
  In this paper, we present a novel developmental reinforcement learning-based
controller for a quadcopter with thrust vectoring capabilities. This multirotor
UAV design has tilt-enabled rotors. It utilizes the rotor force magnitude and
direction to achieve the desired state during flight. The control policy of
this robot is learned using the policy transfer from the learned controller of
the quadcopter (comparatively simple UAV design without thrust vectoring). This
approach allows learning a control policy for systems with multiple inputs and
multiple outputs. The performance of the learned policy is evaluated by
physics-based simulations for the tasks of hovering and way-point navigation.
The flight simulations utilize a flight controller based on reinforcement
learning without any additional PID components. The results show faster
learning with the presented approach as opposed to learning the control policy
from scratch for this new UAV design created by modifications in a conventional
quadcopter, i.e., the addition of more degrees of freedom (4-actuators in
conventional quadcopter to 8-actuators in tilt-rotor quadcopter). We
demonstrate the robustness of our learned policy by showing the recovery of the
tilt-rotor platform in the simulation from various non-static initial
conditions in order to reach a desired state. The developmental policy for the
tilt-rotor UAV also showed superior fault tolerance when compared with the
policy learned from the scratch. The results show the ability of the presented
approach to bootstrap the learned behavior from a simpler system
(lower-dimensional action-space) to a more complex robot (comparatively
higher-dimensional action-space) and reach better performance faster.
