Trajectory-Constrained Deep Latent Visual Attention for Improved Local
  Planning in Presence of Heterogeneous Terrain
  We present a reward-predictive, model-based deep learning method featuring
trajectory-constrained visual attention for local planning in visual navigation
tasks. Our method learns to place visual attention at locations in latent image
space which follow trajectories caused by vehicle control actions to enhance
predictive accuracy during planning. The attention model is jointly optimized
by the task-specific loss and an additional trajectory-constraint loss,
allowing adaptability yet encouraging a regularized structure for improved
generalization and reliability. Importantly, visual attention is applied in
latent feature map space instead of raw image space to promote efficient
planning. We validated our model in visual navigation tasks of planning low
turbulence, collision-free trajectories in off-road settings and hill climbing
with locking differentials in the presence of slippery terrain. Experiments
involved randomized procedural generated simulation and real-world
environments. We found our method improved generalization and learning
efficiency when compared to no-attention and self-attention alternatives.
