Improving the Reconstruction of Disentangled Representation Learners via
  Multi-Stage Modelling
  Current autoencoder-based disentangled representation learning methods
achieve disentanglement by penalizing the (aggregate) posterior to encourage
statistical independence of the latent factors. This approach introduces a
trade-off between disentangled representation learning and reconstruction
quality since the model does not have enough capacity to learn correlated
latent variables that capture detail information present in most image data. To
overcome this trade-off, we present a novel multi-stage modelling approach
where the disentangled factors are first learned using a preexisting
disentangled representation learning method (such as $\beta$-TCVAE); then, the
low-quality reconstruction is improved with another deep generative model that
is trained to model the missing correlated latent variables, adding detail
information while maintaining conditioning on the previously learned
disentangled factors. Taken together, our multi-stage modelling approach
results in a single, coherent probabilistic model that is theoretically
justified by the principal of D-separation and can be realized with a variety
of model classes including likelihood-based models such as variational
autoencoders, implicit models such as generative adversarial networks, and
tractable models like normalizing flows or mixtures of Gaussians. We
demonstrate that our multi-stage model has much higher reconstruction quality
than current state-of-the-art methods with equivalent disentanglement
performance across multiple standard benchmarks.
