Zero-Shot Reinforcement Learning with Deep Attention Convolutional
  Neural Networks
  Simulation-to-simulation and simulation-to-real world transfer of neural
network models have been a difficult problem. To close the reality gap, prior
methods to simulation-to-real world transfer focused on domain adaptation,
decoupling perception and dynamics and solving each problem separately, and
randomization of agent parameters and environment conditions to expose the
learning agent to a variety of conditions. While these methods provide
acceptable performance, the computational complexity required to capture a
large variation of parameters for comprehensive scenarios on a given task such
as autonomous driving or robotic manipulation is high. Our key contribution is
to theoretically prove and empirically demonstrate that a deep attention
convolutional neural network (DACNN) with specific visual sensor configuration
performs as well as training on a dataset with high domain and parameter
variation at lower computational complexity. Specifically, the attention
network weights are learned through policy optimization to focus on local
dependencies that lead to optimal actions, and does not require tuning in
real-world for generalization. Our new architecture adapts perception with
respect to the control objective, resulting in zero-shot learning without
pre-training a perception network. To measure the impact of our new deep
network architecture on domain adaptation, we consider autonomous driving as a
use case. We perform an extensive set of experiments in
simulation-to-simulation and simulation-to-real scenarios to compare our
approach to several baselines including the current state-of-art models.
