Adaptivity for Regularized Kernel Methods by Lepskii's Principle
  We address the problem of {\it adaptivity} in the framework of reproducing
kernel Hilbert space (RKHS) regression. More precisely, we analyze estimators
arising from a linear regularization scheme $g_\lam$. In practical
applications, an important task is to choose the regularization parameter
$\lam$ appropriately, i.e. based only on the given data and independently on
unknown structural assumptions on the regression function. An attractive
approach avoiding data-splitting is the {\it Lepskii Principle} (LP), also
known as the {\it Balancing Principle} is this setting. We show that a modified
parameter choice based on (LP) is minimax optimal adaptive, up to
$\log\log(n)$. A convenient result is the fact that balancing in $L^2(\nu)-$
norm, which is easiest, automatically gives optimal balancing in all stronger
norms, interpolating between $L^2(\nu)$ and the RKHS. An analogous result is
open for other classical approaches to data dependent choices of the
regularization parameter, e.g. for Hold-Out.
