Interpolation and Learning with Scale Dependent Kernels
  We study the learning properties of nonparametric ridge-less least squares.
In particular, we consider the common case of estimators defined by scale
dependent kernels, and focus on the role of the scale. These estimators
interpolate the data and the scale can be shown to control their stability
through the condition number. Our analysis shows that are different regimes
depending on the interplay between the sample size, its dimensions, and the
smoothness of the problem. Indeed, when the sample size is less than
exponential in the data dimension, then the scale can be chosen so that the
learning error decreases. As the sample size becomes larger, the overall error
stop decreasing but interestingly the scale can be chosen in such a way that
the variance due to noise remains bounded. Our analysis combines, probabilistic
results with a number of analytic techniques from interpolation theory.
