Sparsely Changing Latent States for Prediction and Planning in Partially
  Observable Domains
  A common approach to prediction and planning in partially observable domains
is to use recurrent neural networks (RNNs), which ideally develop and maintain
a latent memory about hidden, task-relevant factors. We hypothesize that many
of these hidden factors in the physical world are constant over time, changing
only sparsely. To study this hypothesis, we propose Gated $L_0$ Regularized
Dynamics (GateL0RD), a novel recurrent architecture that incorporates the
inductive bias to maintain stable, sparsely changing latent states. The bias is
implemented by means of a novel internal gating function and a penalty on the
$L_0$ norm of latent state changes. We demonstrate that GateL0RD can compete
with or outperform state-of-the-art RNNs in a variety of partially observable
prediction and control tasks. GateL0RD tends to encode the underlying
generative factors of the environment, ignores spurious temporal dependencies,
and generalizes better, improving sampling efficiency and overall performance
in model-based planning and reinforcement learning tasks. Moreover, we show
that the developing latent states can be easily interpreted, which is a step
towards better explainability in RNNs.
