Derivations of Normalized Mutual Information in Binary Classifications
  This correspondence studies the basic problem of classifications - how to
evaluate different classifiers. Although the conventional performance indexes,
such as accuracy, are commonly used in classifier selection or evaluation,
information-based criteria, such as mutual information, are becoming popular in
feature/model selections. In this work, we propose to assess classifiers in
terms of normalized mutual information (NI), which is novel and well defined in
a compact range for classifier evaluation. We derive close-form relations of
normalized mutual information with respect to accuracy, precision, and recall
in binary classifications. By exploring the relations among them, we reveal
that NI is actually a set of nonlinear functions, with a concordant
power-exponent form, to each performance index. The relations can also be
expressed with respect to precision and recall, or to false alarm and hitting
rate (recall).
