Asymptotic Properties for Bayesian Neural Network in Besov Space
Neural networks have shown great predictive power when dealing with various
unstructured data such as images and natural languages. The Bayesian neural
network captures the uncertainty of prediction by putting a prior distribution
for the parameter of the model and computing the posterior distribution. In
this paper, we show that the Bayesian neural network using spike-and-slab prior
has consistency with nearly minimax convergence rate when the true regression
function is in the Besov space. Even when the smoothness of the regression
function is unknown the same posterior convergence rate holds and thus the
spike and slab prior is adaptive to the smoothness of the regression function.
We also consider the shrinkage prior and show that it has the same convergence
rate. In other words, we propose a practical Bayesian neural network with
guaranteed asymptotic properties.