Neural Bootstrapper
  Bootstrapping has been a primary tool for ensemble and uncertainty
quantification in machine learning and statistics. However, due to its nature
of multiple training and resampling, bootstrapping deep neural networks is
computationally burdensome; hence it has difficulties in practical application
to the uncertainty estimation and related tasks. To overcome this computational
bottleneck, we propose a novel approach called \emph{Neural Bootstrapper}
(NeuBoots), which learns to generate bootstrapped neural networks through
single model training. NeuBoots injects the bootstrap weights into the
high-level feature layers of the backbone network and outputs the bootstrapped
predictions of the target, without additional parameters and the repetitive
computations from scratch. We apply NeuBoots to various machine learning tasks
related to uncertainty quantification, including prediction calibrations in
image classification and semantic segmentation, active learning, and detection
of out-of-distribution samples. Our empirical results show that NeuBoots
outperforms other bagging based methods under a much lower computational cost
without losing the validity of bootstrapping.
