Variational Temporal Abstraction
  We introduce a variational approach to learning and inference of temporally
hierarchical structure and representation for sequential data. We propose the
Variational Temporal Abstraction (VTA), a hierarchical recurrent state space
model that can infer the latent temporal structure and thus perform the
stochastic state transition hierarchically. We also propose to apply this model
to implement the jumpy-imagination ability in imagination-augmented
agent-learning in order to improve the efficiency of the imagination. In
experiments, we demonstrate that our proposed method can model 2D and 3D visual
sequence datasets with interpretable temporal structure discovery and that its
application to jumpy imagination enables more efficient agent-learning in a 3D
navigation task.
