Accelerating Monte Carlo Bayesian Inference via Approximating Predictive
  Uncertainty over Simplex
  Estimating the predictive uncertainty of a Bayesian learning model is
critical in various decision-making problems, e.g., reinforcement learning,
detecting adversarial attack, self-driving car. As the model posterior is
almost always intractable, most efforts were made on finding an accurate
approximation the true posterior. Even though a decent estimation of the model
posterior is obtained, another approximation is required to compute the
predictive distribution over the desired output. A common accurate solution is
to use Monte Carlo (MC) integration. However, it needs to maintain a large
number of samples, evaluate the model repeatedly and average multiple model
outputs. In many real-world cases, this is computationally prohibitive. In this
work, assuming that the exact posterior or a decent approximation is obtained,
we propose a generic framework to approximate the output probability
distribution induced by model posterior with a parameterized model and in an
amortized fashion. The aim is to approximate the true uncertainty of a specific
Bayesian model, meanwhile alleviating the heavy workload of MC integration at
testing time. The proposed method is universally applicable to Bayesian
classification models that allow for posterior sampling. Theoretically, we show
that the idea of amortization incurs no additional costs on approximation
performance. Empirical results validate the strong practical performance of our
approach.
