Life-Long Disentangled Representation Learning with Cross-Domain Latent
  Homologies
  Intelligent behaviour in the real-world requires the ability to acquire new
knowledge from an ongoing sequence of experiences while preserving and reusing
past knowledge. We propose a novel algorithm for unsupervised representation
learning from piece-wise stationary visual data: Variational Autoencoder with
Shared Embeddings (VASE). Based on the Minimum Description Length principle,
VASE automatically detects shifts in the data distribution and allocates spare
representational capacity to new knowledge, while simultaneously protecting
previously learnt representations from catastrophic forgetting. Our approach
encourages the learnt representations to be disentangled, which imparts a
number of desirable properties: VASE can deal sensibly with ambiguous inputs,
it can enhance its own representations through imagination-based exploration,
and most importantly, it exhibits semantically meaningful sharing of latents
between different datasets. Compared to baselines with entangled
representations, our approach is able to reason beyond surface-level statistics
and perform semantically meaningful cross-domain inference.
