Counterfactual Normalization: Proactively Addressing Dataset Shift and
  Improving Reliability Using Causal Mechanisms
  Predictive models can fail to generalize from training to deployment
environments because of dataset shift, posing a threat to model reliability and
the safety of downstream decisions made in practice. Instead of using samples
from the target distribution to reactively correct dataset shift, we use
graphical knowledge of the causal mechanisms relating variables in a prediction
problem to proactively remove relationships that do not generalize across
environments, even when these relationships may depend on unobserved variables
(violations of the "no unobserved confounders" assumption). To accomplish this,
we identify variables with unstable paths of statistical influence and remove
them from the model. We also augment the causal graph with latent
counterfactual variables that isolate unstable paths of statistical influence,
allowing us to retain stable paths that would otherwise be removed. Our
experiments demonstrate that models that remove vulnerable variables and use
estimates of the latent variables transfer better, often outperforming in the
target domain despite some accuracy loss in the training domain.
