A Unified View of Causal and Non-causal Feature Selection
  In this paper, we aim to develop a unified view of causal and non-causal
feature selection methods. The unified view will fill in the gap in the
research of the relation between the two types of methods. Based on the
Bayesian network framework and information theory, we first show that causal
and non-causal feature selection methods share the same objective. That is to
find the Markov blanket of a class attribute, the theoretically optimal feature
set for classification. We then examine the assumptions made by causal and
non-causal feature selection methods when searching for the optimal feature
set, and unify the assumptions by mapping them to the restrictions on the
structure of the Bayesian network model of the studied problem. We further
analyze in detail how the structural assumptions lead to the different levels
of approximations employed by the methods in their search, which then result in
the approximations in the feature sets found by the methods with respect to the
optimal feature set. With the unified view, we are able to interpret the output
of non-causal methods from a causal perspective and derive the error bounds of
both types of methods. Finally, we present practical understanding of the
relation between causal and non-causal methods using extensive experiments with
synthetic data and various types of real-word data.
