TCMI: a non-parametric mutual-dependence estimator for multivariate
  continuous distributions
  The identification of relevant features, i.e., the driving variables that
determine a process or the properties of a system, is an essential part of the
analysis of data sets with a large number of variables. A mathematical rigorous
approach to quantifying the relevance of these features is mutual information.
Mutual information determines the relevance of features in terms of their joint
mutual dependence to the property of interest. However, mutual information
requires as input probability distributions, which cannot be reliably estimated
from continuous distributions such as physical quantities like lengths or
energies. Here, we introduce total cumulative mutual information (TCMI), a
measure of the relevance of mutual dependences that extends mutual information
to random variables of continuous distribution based on cumulative probability
distributions. TCMI is a non-parametric, robust, and deterministic measure that
facilitates comparisons and rankings between feature sets with different
cardinality. The ranking induced by TCMI allows for feature selection, i.e.,
the identification of variable sets that are nonlinear statistically related to
a property of interest, taking into account the number of data samples as well
as the cardinality of the set of variables. We evaluate the performance of our
measure with simulated data, compare its performance with similar
multivariate-dependence measures, and demonstrate the effectiveness of our
feature-selection method on a set of standard data sets and a typical scenario
in materials science.
