How Many Data Are Needed for Robust Learning?
  We show that the sample complexity of robust interpolation problem could be
exponential in the input dimensionality and discover a phase transition
phenomenon when the data are in a unit ball. Robust interpolation refers to the
problem of interpolating $n$ noisy training data in $\R^d$ by a Lipschitz
function. Although this problem has been well understood when the covariates
are drawn from an isoperimetry distribution, much remains unknown concerning
its performance under generic or even the worst-case distributions. Our results
are two-fold: 1) too many data hurt robustness; we provide a tight and
universal Lipschitzness lower bound $\Omega(n^{1/d})$ of the interpolating
function for arbitrary data distributions. Our result disproves potential
existence of an $\mathcal{O}(1)$-Lipschitz function in the overparametrization
scenario when $n=\exp(\omega(d))$. 2) Small data hurt robustness:
$n=\exp(\Omega(d))$ is necessary for obtaining a good population error under
certain distributions by any $\mathcal{O}(1)$-Lipschitz learning algorithm.
Perhaps surprisingly, our results shed light on the curse of big data and the
blessing of dimensionality for robustness, and discover an intriguing
phenomenon of phase transition at $n=\exp(\Theta(d))$.
