Dimension Reduction by Mutual Information Feature Extraction
  During the past decades, to study high-dimensional data in a large variety of
problems, researchers have proposed many Feature Extraction algorithms. One of
the most effective approaches for optimal feature extraction is based on mutual
information (MI). However it is not always easy to get an accurate estimation
for high dimensional MI. In terms of MI, the optimal feature extraction is
creating a feature set from the data which jointly have the largest dependency
on the target class and minimum redundancy. In this paper, a
component-by-component gradient ascent method is proposed for feature
extraction which is based on one-dimensional MI estimates. We will refer to
this algorithm as Mutual Information Feature Extraction (MIFX). The performance
of this proposed method is evaluated using UCI databases. The results indicate
that MIFX provides a robust performance over different data sets which are
almost always the best or comparable to the best ones.
