Generalising the Discriminative Restricted Boltzmann Machine
  We present a novel theoretical result that generalises the Discriminative
Restricted Boltzmann Machine (DRBM). While originally the DRBM was defined
assuming the {0, 1}-Bernoulli distribution in each of its hidden units, this
result makes it possible to derive cost functions for variants of the DRBM that
utilise other distributions, including some that are often encountered in the
literature. This is illustrated with the Binomial and {-1, +1}-Bernoulli
distributions here. We evaluate these two DRBM variants and compare them with
the original one on three benchmark datasets, namely the MNIST and USPS digit
classification datasets, and the 20 Newsgroups document classification dataset.
Results show that each of the three compared models outperforms the remaining
two in one of the three datasets, thus indicating that the proposed theoretical
generalisation of the DRBM may be valuable in practice.
