Large-Scale Feature Learning With Spike-and-Slab Sparse Coding
  We consider the problem of object recognition with a large number of classes.
In order to overcome the low amount of labeled examples available in this
setting, we introduce a new feature learning and extraction procedure based on
a factor model we call spike-and-slab sparse coding (S3C). Prior work on S3C
has not prioritized the ability to exploit parallel architectures and scale S3C
to the enormous problem sizes needed for object recognition. We present a novel
inference procedure for appropriate for use with GPUs which allows us to
dramatically increase both the training set size and the amount of latent
factors that S3C may be trained with. We demonstrate that this approach
improves upon the supervised learning capabilities of both sparse coding and
the spike-and-slab Restricted Boltzmann Machine (ssRBM) on the CIFAR-10
dataset. We use the CIFAR-100 dataset to demonstrate that our method scales to
large numbers of classes better than previous methods. Finally, we use our
method to win the NIPS 2011 Workshop on Challenges In Learning Hierarchical
Models? Transfer Learning Challenge.
