Conditional Generative Modeling via Learning the Latent Space
  Although deep learning has achieved appealing results on several machine
learning tasks, most of the models are deterministic at inference, limiting
their application to single-modal settings. We propose a novel general-purpose
framework for conditional generation in multimodal spaces, that uses latent
variables to model generalizable learning patterns while minimizing a family of
regression cost functions. At inference, the latent variables are optimized to
find optimal solutions corresponding to multiple output modes. Compared to
existing generative solutions, in multimodal spaces, our approach demonstrates
faster and stable convergence, and can learn better representations for
downstream tasks. Importantly, it provides a simple generic model that can beat
highly engineered pipelines tailored using domain expertise on a variety of
tasks, while generating diverse outputs. Our codes will be released.
