Overparameterized Linear Regression under Adversarial Attacks
  As machine learning models start to be used in critical applications, their
vulnerabilities and brittleness become a pressing concern. Adversarial attacks
are a popular framework for studying these vulnerabilities. In this work, we
study the error of linear regression in the face of adversarial attacks. We
provide bounds of the error in terms of the traditional risk and the parameter
norm and show how these bounds can be leveraged and make it possible to use
analysis from non-adversarial setups to study the adversarial risk. The
usefulness of these results is illustrated by shedding light on whether or not
overparameterized linear models can be adversarially robust. We show that
adding features to linear models might be either a source of additional
robustness or brittleness. We show that these differences appear due to scaling
and how the $\ell_1$ and $\ell_2$ norms of random projections concentrate. We
also show how the reformulation we propose allows for solving adversarial
training as a convex optimization problem. This is then used as a tool to study
how adversarial training and other regularization methods might affect the
robustness of the estimated models.
