The deep generative decoder: Using MAP estimates of representations
  A deep generative model is characterized by a representation space, its
distribution, and a neural network mapping the representation to a distribution
over vectors in feature space. Common methods such as variational autoencoders
(VAEs) apply variational inference for training the neural network, but
optimizing these models is often non-trivial. The encoder adds to the
complexity of the model and introduces an amortization gap and the quality of
the variational approximation is usually unknown. Additionally, the balance of
the loss terms of the objective function heavily influences performance.
Therefore, we argue that it is worthwhile to investigate a much simpler
approximation which finds representations and their distribution by maximizing
the model likelihood via back-propagation. In this approach, there is no
encoder, and we therefore call it a Deep Generative Decoder (DGD). Using the
CIFAR10 data set, we show that the DGD is easier and faster to optimize than
the VAE, achieves more consistent low reconstruction errors of test data, and
alleviates the problem of balancing the reconstruction and distribution loss
terms. Although the model in its simple form cannot compete with
state-of-the-art image generation approaches, it obtains better image
generation scores than the variational approach on the CIFAR10 data. We
demonstrate on MNIST data how the use of a Gaussian mixture with priors can
lead to a clear separation of classes in a 2D representation space, and how the
DGD can be used with labels to obtain a supervised representation.
