Generalization Error Rates in Kernel Regression: The Crossover from the
  Noiseless to Noisy Regime
  In this manuscript we consider Kernel Ridge Regression (KRR) under the
Gaussian design. Exponents for the decay of the excess generalization error of
KRR have been reported in various works under the assumption of power-law decay
of eigenvalues of the features co-variance. These decays were, however,
provided for sizeably different setups, namely in the noiseless case with
constant regularization and in the noisy optimally regularized case.
Intermediary settings have been left substantially uncharted. In this work, we
unify and extend this line of work, providing characterization of all regimes
and excess error decay rates that can be observed in terms of the interplay of
noise and regularization. In particular, we show the existence of a transition
in the noisy setting between the noiseless exponents to its noisy values as the
sample complexity is increased. Finally, we illustrate how this crossover can
also be observed on real data sets.
