Disentangled Representations in Neural Models
  Representation learning is the foundation for the recent success of neural
network models. However, the distributed representations generated by neural
networks are far from ideal. Due to their highly entangled nature, they are di
cult to reuse and interpret, and they do a poor job of capturing the sparsity
which is present in real- world transformations. In this paper, I describe
methods for learning disentangled representations in the two domains of
graphics and computation. These methods allow neural methods to learn
representations which are easy to interpret and reuse, yet they incur little or
no penalty to performance. In the Graphics section, I demonstrate the ability
of these methods to infer the generating parameters of images and rerender
those images under novel conditions. In the Computation section, I describe a
model which is able to factorize a multitask learning problem into subtasks and
which experiences no catastrophic forgetting. Together these techniques provide
the tools to design a wide range of models that learn disentangled
representations and better model the factors of variation in the real world.
