Counterfactual Cross-Validation: Stable Model Selection Procedure for
  Causal Inference Models
  We study the model selection problem in conditional average treatment effect
(CATE) prediction. Unlike previous works on this topic, we focus on preserving
the rank order of the performance of candidate CATE predictors to enable
accurate and stable model selection. To this end, we analyze the model
performance ranking problem and formulate guidelines to obtain a better
evaluation metric. We then propose a novel metric that can identify the ranking
of the performance of CATE predictors with high confidence. Empirical
evaluations demonstrate that our metric outperforms existing metrics in both
model selection and hyperparameter tuning tasks.
