Variable Selection with Rigorous Uncertainty Quantification using Deep
  Bayesian Neural Networks: Posterior Concentration and Bernstein-von Mises
  Phenomenon
  This work develops rigorous theoretical basis for the fact that deep Bayesian
neural network (BNN) is an effective tool for high-dimensional variable
selection with rigorous uncertainty quantification. We develop new Bayesian
non-parametric theorems to show that a properly configured deep BNN (1) learns
the variable importance effectively in high dimensions, and its learning rate
can sometimes "break" the curse of dimensionality. (2) BNN's uncertainty
quantification for variable importance is rigorous, in the sense that its 95%
credible intervals for variable importance indeed covers the truth 95% of the
time (i.e., the Bernstein-von Mises (BvM) phenomenon). The theoretical results
suggest a simple variable selection algorithm based on the BNN's credible
intervals. Extensive simulation confirms the theoretical findings and shows
that the proposed algorithm outperforms existing classic and
neural-network-based variable selection methods, particularly in high
dimensions.
