Joint Training Deep Boltzmann Machines for Classification
  We introduce a new method for training deep Boltzmann machines jointly. Prior
methods of training DBMs require an initial learning pass that trains the model
greedily, one layer at a time, or do not perform well on classification tasks.
In our approach, we train all layers of the DBM simultaneously, using a novel
training procedure called multi-prediction training. The resulting model can
either be interpreted as a single generative model trained to maximize a
variational approximation to the generalized pseudolikelihood, or as a family
of recurrent networks that share parameters and may be approximately averaged
together using a novel technique we call the multi-inference trick. We show
that our approach performs competitively for classification and outperforms
previous methods in terms of accuracy of approximate inference and
classification with missing inputs.
