Modal Uncertainty Estimation via Discrete Latent Representation
  Many important problems in the real world don't have unique solutions. It is
thus important for machine learning models to be capable of proposing different
plausible solutions with meaningful probability measures. In this work we
introduce such a deep learning framework that learns the one-to-many mappings
between the inputs and outputs, together with faithful uncertainty measures. We
call our framework {\it modal uncertainty estimation} since we model the
one-to-many mappings to be generated through a set of discrete latent
variables, each representing a latent mode hypothesis that explains the
corresponding type of input-output relationship. The discrete nature of the
latent representations thus allows us to estimate for any input the conditional
probability distribution of the outputs very effectively. Both the discrete
latent space and its uncertainty estimation are jointly learned during
training. We motivate our use of discrete latent space through the multi-modal
posterior collapse problem in current conditional generative models, then
develop the theoretical background, and extensively validate our method on both
synthetic and realistic tasks. Our framework demonstrates significantly more
accurate uncertainty estimation than the current state-of-the-art methods, and
is informative and convenient for practical use.
