Learning Task-Driven Control Policies via Information Bottlenecks
  This paper presents a reinforcement learning approach to synthesizing
task-driven control policies for robotic systems equipped with rich sensory
modalities (e.g., vision or depth). Standard reinforcement learning algorithms
typically produce policies that tightly couple control actions to the entirety
of the system's state and rich sensor observations. As a consequence, the
resulting policies can often be sensitive to changes in task-irrelevant
portions of the state or observations (e.g., changing background colors). In
contrast, the approach we present here learns to create a task-driven
representation that is used to compute control actions. Formally, this is
achieved by deriving a policy gradient-style algorithm that creates an
information bottleneck between the states and the task-driven representation;
this constrains actions to only depend on task-relevant information. We
demonstrate our approach in a thorough set of simulation results on multiple
examples including a grasping task that utilizes depth images and a
ball-catching task that utilizes RGB images. Comparisons with a standard policy
gradient approach demonstrate that the task-driven policies produced by our
algorithm are often significantly more robust to sensor noise and
task-irrelevant changes in the environment.
