Bayesian Nonparametric Causal Inference: Information Rates and Learning
  Algorithms
  We investigate the problem of estimating the causal effect of a treatment on
individual subjects from observational data, this is a central problem in
various application domains, including healthcare, social sciences, and online
advertising. Within the Neyman Rubin potential outcomes model, we use the
Kullback Leibler (KL) divergence between the estimated and true distributions
as a measure of accuracy of the estimate, and we define the information rate of
the Bayesian causal inference procedure as the (asymptotic equivalence class of
the) expected value of the KL divergence between the estimated and true
distributions as a function of the number of samples. Using Fano method, we
establish a fundamental limit on the information rate that can be achieved by
any Bayesian estimator, and show that this fundamental limit is independent of
the selection bias in the observational data. We characterize the Bayesian
priors on the potential (factual and counterfactual) outcomes that achieve the
optimal information rate. As a consequence, we show that a particular class of
priors that have been widely used in the causal inference literature cannot
achieve the optimal information rate. On the other hand, a broader class of
priors can achieve the optimal information rate. We go on to propose a prior
adaptation procedure (which we call the information based empirical Bayes
procedure) that optimizes the Bayesian prior by maximizing an information
theoretic criterion on the recovered causal effects rather than maximizing the
marginal likelihood of the observed (factual) data. Building on our analysis,
we construct an information optimal Bayesian causal inference algorithm.
