Evaluating Causal Inference Methods
  The fundamental challenge of drawing causal inference is that counterfactual
outcomes are not fully observed for any unit. Furthermore, in observational
studies, treatment assignment is likely to be confounded. Many statistical
methods have emerged for causal inference under unconfoundedness conditions
given pre-treatment covariates, including propensity score-based methods,
prognostic score-based methods, and doubly robust methods. Unfortunately for
applied researchers, there is no `one-size-fits-all' causal method that can
perform optimally universally. In practice, causal methods are primarily
evaluated quantitatively on handcrafted simulated data. Such data-generative
procedures can be of limited value because they are typically stylized models
of reality. They are simplified for tractability and lack the complexities of
real-world data. For applied researchers, it is critical to understand how well
a method performs for the data at hand. Our work introduces a deep generative
model-based framework, Credence, to validate causal inference methods. The
framework's novelty stems from its ability to generate synthetic data anchored
at the empirical distribution for the observed sample, and therefore virtually
indistinguishable from the latter. The approach allows the user to specify
ground truth for the form and magnitude of causal effects and confounding bias
as functions of covariates. Thus simulated data sets are used to evaluate the
potential performance of various causal estimation methods when applied to data
similar to the observed sample. We demonstrate Credence's ability to accurately
assess the relative performance of causal estimation techniques in an extensive
simulation study and two real-world data applications from Lalonde and Project
STAR studies.
