Unsupervised model compression for multilayer bootstrap networks
  Recently, multilayer bootstrap network (MBN) has demonstrated promising
performance in unsupervised dimensionality reduction. It can learn compact
representations in standard data sets, i.e. MNIST and RCV1. However, as a
bootstrap method, the prediction complexity of MBN is high. In this paper, we
propose an unsupervised model compression framework for this general problem of
unsupervised bootstrap methods. The framework compresses a large unsupervised
bootstrap model into a small model by taking the bootstrap model and its
application together as a black box and learning a mapping function from the
input of the bootstrap model to the output of the application by a supervised
learner. To specialize the framework, we propose a new technique, named
compressive MBN. It takes MBN as the unsupervised bootstrap model and deep
neural network (DNN) as the supervised learner. Our initial result on MNIST
showed that compressive MBN not only maintains the high prediction accuracy of
MBN but also is over thousands of times faster than MBN at the prediction
stage. Our result suggests that the new technique integrates the effectiveness
of MBN on unsupervised learning and the effectiveness and efficiency of DNN on
supervised learning together for the effectiveness and efficiency of
compressive MBN on unsupervised learning.
