Understanding disentangling in $\beta$-VAE
  We present new intuitions and theoretical assessments of the emergence of
disentangled representation in variational autoencoders. Taking a
rate-distortion theory perspective, we show the circumstances under which
representations aligned with the underlying generative factors of variation of
data emerge when optimising the modified ELBO bound in $\beta$-VAE, as training
progresses. From these insights, we propose a modification to the training
regime of $\beta$-VAE, that progressively increases the information capacity of
the latent code during training. This modification facilitates the robust
learning of disentangled representations in $\beta$-VAE, without the previous
trade-off in reconstruction accuracy.
