A Discriminative Gaussian Mixture Model with Sparsity
  In probabilistic classification, a discriminative model based on the softmax
function has a potential limitation in that it assumes unimodality for each
class in the feature space. The mixture model can address this issue, although
it leads to an increase in the number of parameters. We propose a sparse
classifier based on a discriminative GMM, referred to as a sparse
discriminative Gaussian mixture (SDGM). In the SDGM, a GMM-based discriminative
model is trained via sparse Bayesian learning. Using this sparse learning
framework, we can simultaneously remove redundant Gaussian components and
reduce the number of parameters used in the remaining components during
learning; this learning method reduces the model complexity, thereby improving
the generalization capability. Furthermore, the SDGM can be embedded into
neural networks (NNs), such as convolutional NNs, and can be trained in an
end-to-end manner. Experimental results demonstrated that the proposed method
outperformed the existing softmax-based discriminative models.
