Make (Nearly) Every Neural Network Better: Generating Neural Network
  Ensembles by Weight Parameter Resampling
  Deep Neural Networks (DNNs) have become increasingly popular in computer
vision, natural language processing, and other areas. However, training and
fine-tuning a deep learning model is computationally intensive and
time-consuming. We propose a new method to improve the performance of nearly
every model including pre-trained models. The proposed method uses an ensemble
approach where the networks in the ensemble are constructed by reassigning
model parameter values based on the probabilistic distribution of these
parameters, calculated towards the end of the training process. For pre-trained
models, this approach results in an additional training step (usually less than
one epoch). We perform a variety of analysis using the MNIST dataset and
validate the approach with a number of DNN models using pre-trained models on
the ImageNet dataset.
