Variational Recurrent Models for Solving Partially Observable Control
  Tasks
  In partially observable (PO) environments, deep reinforcement learning (RL)
agents often suffer from unsatisfactory performance, since two problems need to
be tackled together: how to extract information from the raw observations to
solve the task, and how to improve the policy. In this study, we propose an RL
algorithm for solving PO tasks. Our method comprises two parts: a variational
recurrent model (VRM) for modeling the environment, and an RL controller that
has access to both the environment and the VRM. The proposed algorithm was
tested in two types of PO robotic control tasks, those in which either
coordinates or velocities were not observable and those that require long-term
memorization. Our experiments show that the proposed algorithm achieved better
data efficiency and/or learned more optimal policy than other alternative
approaches in tasks in which unobserved states cannot be inferred from raw
observations in a simple manner.
