Robust regression with covariate filtering: Heavy tails and adversarial
  contamination
  We study the problem of linear regression where both covariates and responses
are potentially (i) heavy-tailed and (ii) adversarially contaminated. Several
computationally efficient estimators have been proposed for the simpler setting
where the covariates are sub-Gaussian and uncontaminated; however, these
estimators may fail when the covariates are either heavy-tailed or contain
outliers. In this work, we show how to modify the Huber regression, least
trimmed squares, and least absolute deviation estimators to obtain estimators
which are simultaneously computationally and statistically efficient in the
stronger contamination model. Our approach is quite simple, and consists of
applying a filtering algorithm to the covariates, and then applying the
classical robust regression estimators to the remaining data. We show that the
Huber regression estimator achieves near-optimal error rates in this setting,
whereas the least trimmed squares and least absolute deviation estimators can
be made to achieve near-optimal error after applying a postprocessing step.
