A Closer Look at Disentangling in $\beta$-VAE
  In many data analysis tasks, it is beneficial to learn representations where
each dimension is statistically independent and thus disentangled from the
others. If data generating factors are also statistically independent,
disentangled representations can be formed by Bayesian inference of latent
variables. We examine a generalization of the Variational Autoencoder (VAE),
$\beta$-VAE, for learning such representations using variational inference.
$\beta$-VAE enforces conditional independence of its bottleneck neurons
controlled by its hyperparameter $\beta$. This condition is in general not
compatible with the statistical independence of latents. By providing
analytical and numerical arguments, we show that this incompatibility leads to
a non-monotonic inference performance in $\beta$-VAE with a finite optimal
$\beta$.
