Benign-Overfitting in Conditional Average Treatment Effect Prediction
  with Linear Regression
  We study the benign overfitting theory in the prediction of the conditional
average treatment effect (CATE), with linear regression models. As the
development of machine learning for causal inference, a wide range of
large-scale models for causality are gaining attention. One problem is that
suspicions have been raised that the large-scale models are prone to
overfitting to observations with sample selection, hence the large models may
not be suitable for causal prediction. In this study, to resolve the
suspicious, we investigate on the validity of causal inference methods for
overparameterized models, by applying the recent theory of benign overfitting
(Bartlett et al., 2020). Specifically, we consider samples whose distribution
switches depending on an assignment rule, and study the prediction of CATE with
linear models whose dimension diverges to infinity. We focus on two methods:
the T-learner, which based on a difference between separately constructed
estimators with each treatment group, and the inverse probability weight
(IPW)-learner, which solves another regression problem approximated by a
propensity score. In both methods, the estimator consists of interpolators that
fit the samples perfectly. As a result, we show that the T-learner fails to
achieve the consistency except the random assignment, while the IPW-learner
converges the risk to zero if the propensity score is known. This difference
stems from that the T-learner is unable to preserve eigenspaces of the
covariances, which is necessary for benign overfitting in the overparameterized
setting. Our result provides new insights into the usage of causal inference
methods in the overparameterizated setting, in particular, doubly robust
estimators.
