Visual Learning-based Planning for Continuous High-Dimensional POMDPs
  The Partially Observable Markov Decision Process (POMDP) is a powerful
framework for capturing decision-making problems that involve state and
transition uncertainty. However, most current POMDP planners cannot effectively
handle very high-dimensional observations they often encounter in the real
world (e.g. image observations in robotic domains). In this work, we propose
Visual Tree Search (VTS), a learning and planning procedure that combines
generative models learned offline with online model-based POMDP planning. VTS
bridges offline model training and online planning by utilizing a set of deep
generative observation models to predict and evaluate the likelihood of image
observations in a Monte Carlo tree search planner. We show that VTS is robust
to different observation noises and, since it utilizes online, model-based
planning, can adapt to different reward structures without the need to
re-train. This new approach outperforms a baseline state-of-the-art on-policy
planning algorithm while using significantly less offline training time.
