A Spectral Regularizer for Unsupervised Disentanglement
  A generative model with a disentangled representation allows for independent
control over different aspects of the output. Learning disentangled
representations has been a recent topic of great interest, but it remains
poorly understood. We show that even for GANs that do not possess disentangled
representations, one can find curved trajectories in latent space over which
local disentanglement occurs. These trajectories are found by iteratively
following the leading right-singular vectors of the Jacobian of the generator
with respect to its input. Based on this insight, we describe an efficient
regularizer that aligns these vectors with the coordinate axes, and show that
it can be used to induce disentangled representations in GANs, in a completely
unsupervised manner.
