Model-Augmented Estimation of Conditional Mutual Information for Feature
  Selection
  Markov blanket feature selection, while theoretically optimal, is generally
challenging to implement. This is due to the shortcomings of existing
approaches to conditional independence (CI) testing, which tend to struggle
either with the curse of dimensionality or computational complexity. We propose
a novel two-step approach which facilitates Markov blanket feature selection in
high dimensions. First, neural networks are used to map features to
low-dimensional representations. In the second step, CI testing is performed by
applying the $k$-NN conditional mutual information estimator to the learned
feature maps. The mappings are designed to ensure that mapped samples both
preserve information and share similar information about the target variable if
and only if they are close in Euclidean distance. We show that these properties
boost the performance of the $k$-NN estimator in the second step. The
performance of the proposed method is evaluated on both synthetic and real
data.
