Directed information and Pearl's causal calculus
  Probabilistic graphical models are a fundamental tool in statistics, machine
learning, signal processing, and control. When such a model is defined on a
directed acyclic graph (DAG), one can assign a partial ordering to the events
occurring in the corresponding stochastic system. Based on the work of Judea
Pearl and others, these DAG-based "causal factorizations" of joint probability
measures have been used for characterization and inference of functional
dependencies (causal links). This mostly expository paper focuses on several
connections between Pearl's formalism (and in particular his notion of
"intervention") and information-theoretic notions of causality and feedback
(such as causal conditioning, directed stochastic kernels, and directed
information). As an application, we show how conditional directed information
can be used to develop an information-theoretic version of Pearl's "back-door"
criterion for identifiability of causal effects from passive observations. This
suggests that the back-door criterion can be thought of as a causal analog of
statistical sufficiency.
