Maximum mutual information regularized classification
  In this paper, a novel pattern classification approach is proposed by
regularizing the classifier learning to maximize mutual information between the
classification response and the true class label. We argue that, with the
learned classifier, the uncertainty of the true class label of a data sample
should be reduced by knowing its classification response as much as possible.
The reduced uncertainty is measured by the mutual information between the
classification response and the true class label. To this end, when learning a
linear classifier, we propose to maximize the mutual information between
classification responses and true class labels of training samples, besides
minimizing the classification error and reduc- ing the classifier complexity.
An objective function is constructed by modeling mutual information with
entropy estimation, and it is optimized by a gradi- ent descend method in an
iterative algorithm. Experiments on two real world pattern classification
problems show the significant improvements achieved by maximum mutual
information regularization.
