Understanding Uncertainty in Bayesian Deep Learning
  Neural Linear Models (NLM) are deep Bayesian models that produce predictive
uncertainty by learning features from the data and then performing Bayesian
linear regression over these features. Despite their popularity, few works have
focused on formally evaluating the predictive uncertainties of these models.
Furthermore, existing works point out the difficulties of encoding domain
knowledge in models like NLMs, making them unsuitable for applications where
interpretability is required. In this work, we show that traditional training
procedures for NLMs can drastically underestimate uncertainty in data-scarce
regions. We identify the underlying reasons for this behavior and propose a
novel training method that can both capture useful predictive uncertainties as
well as allow for incorporation of domain knowledge.
