Local Disentanglement in Variational Auto-Encoders Using Jacobian $L_1$
  Regularization
  There have been many recent advances in representation learning; however,
unsupervised representation learning can still struggle with model
identification issues related to rotations of the latent space. Variational
Auto-Encoders (VAEs) and their extensions such as $\beta$-VAEs have been shown
to improve local alignment of latent variables with PCA directions, which can
help to improve model disentanglement under some conditions. Borrowing
inspiration from Independent Component Analysis (ICA) and sparse coding, we
propose applying an $L_1$ loss to the VAE's generative Jacobian during training
to encourage local latent variable alignment with independent factors of
variation in images of multiple objects or images with multiple parts. We
demonstrate our results on a variety of datasets, giving qualitative and
quantitative results using information theoretic and modularity measures that
show our added $L_1$ cost encourages local axis alignment of the latent
representation with individual factors of variation.
