Deterministic Decoding for Discrete Data in Variational Autoencoders
  Variational autoencoders are prominent generative models for modeling
discrete data. However, with flexible decoders, they tend to ignore the latent
codes. In this paper, we study a VAE model with a deterministic decoder
(DD-VAE) for sequential data that selects the highest-scoring tokens instead of
sampling. Deterministic decoding solely relies on latent codes as the only way
to produce diverse objects, which improves the structure of the learned
manifold. To implement DD-VAE, we propose a new class of bounded support
proposal distributions and derive Kullback-Leibler divergence for Gaussian and
uniform priors. We also study a continuous relaxation of deterministic decoding
objective function and analyze the relation of reconstruction accuracy and
relaxation parameters. We demonstrate the performance of DD-VAE on multiple
datasets, including molecular generation and optimization problems.
