Graph Neural Networks for Relational Inductive Bias in Vision-based Deep
  Reinforcement Learning of Robot Control
  State-of-the-art reinforcement learning algorithms predominantly learn a
policy from either a numerical state vector or images. Both approaches
generally do not take structural knowledge of the task into account, which is
especially prevalent in robotic applications and can benefit learning if
exploited. This work introduces a neural network architecture that combines
relational inductive bias and visual feedback to learn an efficient position
control policy for robotic manipulation. We derive a graph representation that
models the physical structure of the manipulator and combines the robot's
internal state with a low-dimensional description of the visual scene generated
by an image encoding network. On this basis, a graph neural network trained
with reinforcement learning predicts joint velocities to control the robot. We
further introduce an asymmetric approach of training the image encoder
separately from the policy using supervised learning. Experimental results
demonstrate that, for a 2-DoF planar robot in a geometrically simplistic 2D
environment, a learned representation of the visual scene can replace access to
the explicit coordinates of the reaching target without compromising on the
quality and sample efficiency of the policy. We further show the ability of the
model to improve sample efficiency for a 6-DoF robot arm in a visually
realistic 3D environment.
