On Simple Reactive Neural Networks for Behaviour-Based Reinforcement
  Learning
  We present a behaviour-based reinforcement learning approach, inspired by
Brook's subsumption architecture, in which simple fully connected networks are
trained as reactive behaviours. Our working assumption is that a pick and place
robotic task can be simplified by leveraging domain knowledge of a robotics
developer to decompose and train such reactive behaviours; namely, approach,
grasp, and retract. Then the robot autonomously learns how to combine them via
an Actor-Critic architecture. The Actor-Critic policy is to determine the
activation and inhibition mechanisms of the reactive behaviours in a particular
temporal sequence. We validate our approach in a simulated robot environment
where the task is picking a block and taking it to a target position while
orienting the gripper from a top grasp. The latter represents an extra
degree-of-freedom of which current end-to-end reinforcement learning fail to
generalise. Our findings suggest that robotic learning can be more effective if
each behaviour is learnt in isolation and then combined them to accomplish the
task. That is, our approach learns the pick and place task in 8,000 episodes,
which represents a drastic reduction in the number of training episodes
required by an end-to-end approach and the existing state-of-the-art
algorithms.
