Q-attention: Enabling Efficient Learning for Vision-based Robotic
  Manipulation
  Despite the success of reinforcement learning methods, they have yet to have
their breakthrough moment when applied to a broad range of robotic manipulation
tasks. This is partly due to the fact that reinforcement learning algorithms
are notoriously difficult and time consuming to train, which is exacerbated
when training from images rather than full-state inputs. As humans perform
manipulation tasks, our eyes closely monitor every step of the process with our
gaze focusing sequentially on the objects being manipulated. With this in mind,
we present our Attention-driven Robotic Manipulation (ARM) algorithm, which is
a general manipulation algorithm that can be applied to a range of
sparse-rewarded tasks, given only a small number of demonstrations. ARM splits
the complex task of manipulation into a 3 stage pipeline: (1) a Q-attention
agent extracts relevant pixel locations from RGB and point cloud inputs, (2) a
next-best pose agent that accepts crops from the Q-attention agent and outputs
poses, and (3) a control agent that takes the goal pose and outputs joint
actions. We show that current learning algorithms fail on a range of RLBench
tasks, whilst ARM is successful.
