On Robustness of Principal Component Regression
  Principal component regression (PCR) is a simple, but powerful and
ubiquitously utilized method. Its effectiveness is well established when the
covariates exhibit low-rank structure. However, its ability to handle settings
with noisy, missing, and mixed-valued, i.e., discrete and continuous,
covariates is not understood and remains an important open challenge. As the
main contribution of this work we establish the robustness of PCR, without any
change, in this respect and provide meaningful finite-sample analysis. To do
so, we establish that PCR is equivalent to performing linear regression after
pre-processing the covariate matrix via hard singular value thresholding
(HSVT). As a result, in the context of counterfactual analysis using
observational data, we show PCR is equivalent to the recently proposed robust
variant of the synthetic control method, known as robust synthetic control
(RSC). As an immediate consequence, we obtain finite-sample analysis of the RSC
estimator that was previously absent. As an important contribution to the
synthetic controls literature, we establish that an (approximate) linear
synthetic control exists in the setting of a generalized factor model or latent
variable model; traditionally in the literature, the existence of a synthetic
control needs to be assumed to exist as an axiom. We further discuss a
surprising implication of the robustness property of PCR with respect to noise,
i.e., PCR can learn a good predictive model even if the covariates are
tactfully transformed to preserve differential privacy. Finally, this work
advances the state-of-the-art analysis for HSVT by establishing stronger
guarantees with respect to the $\ell_{2, \infty}$-norm rather than the
Frobenius norm as is commonly done in the matrix estimation literature, which
may be of interest in its own right.
