Sense and Sensitivity Analysis: Simple Post-Hoc Analysis of Bias Due to
  Unobserved Confounding
  It is a truth universally acknowledged that an observed association without
known mechanism must be in want of a causal estimate. However, causal
estimation from observational data often relies on the (untestable) assumption
of `no unobserved confounding'. Violations of this assumption can induce bias
in effect estimates. In principle, such bias could invalidate or reverse the
conclusions of a study. However, in some cases, we might hope that the
influence of unobserved confounders is weak relative to a `large' estimated
effect, so the qualitative conclusions are robust to bias from unobserved
confounding. The purpose of this paper is to develop \emph{Austen plots}, a
sensitivity analysis tool to aid such judgments by making it easier to reason
about potential bias induced by unobserved confounding. We formalize
confounding strength in terms of how strongly the confounder influences
treatment assignment and outcome. For a target level of bias, an Austen plot
shows the minimum values of treatment and outcome influence required to induce
that level of bias. Domain experts can then make subjective judgments about
whether such strong confounders are plausible. To aid this judgment, the Austen
plot additionally displays the estimated influence strength of (groups of) the
observed covariates. Austen plots generalize the classic sensitivity analysis
approach of Imbens [Imb03]. Critically, Austen plots allow any approach for
modeling the observed data and producing the initial estimate. We illustrate
the tool by assessing biases for several real causal inference problems, using
a variety of machine learning approaches for the initial data analysis. Code is
available at https://github.com/anishazaveri/austen_plots
