Image-to-image Mapping with Many Domains by Sparse Attribute Transfer
  Unsupervised image-to-image translation consists of learning a pair of
mappings between two domains without known pairwise correspondences between
points. The current convention is to approach this task with cycle-consistent
GANs: using a discriminator to encourage the generator to change the image to
match the target domain, while training the generator to be inverted with
another mapping. While ending up with paired inverse functions may be a good
end result, enforcing this restriction at all times during training can be a
hindrance to effective modeling. We propose an alternate approach that directly
restricts the generator to performing a simple sparse transformation in a
latent layer, motivated by recent work from cognitive neuroscience suggesting
an architectural prior on representations corresponding to consciousness. Our
biologically motivated approach leads to representations more amenable to
transformation by disentangling high-level abstract concepts in the latent
space. We demonstrate that image-to-image domain translation with many
different domains can be learned more effectively with our architecturally
constrained, simple transformation than with previous unconstrained
architectures that rely on a cycle-consistency loss.
