Beyond Tikhonov: Faster Learning with Self-Concordant Losses via
  Iterative Regularization
  The theory of spectral filtering is a remarkable tool to understand the
statistical properties of learning with kernels. For least squares, it allows
to derive various regularization schemes that yield faster convergence rates of
the excess risk than with Tikhonov regularization. This is typically achieved
by leveraging classical assumptions called source and capacity conditions,
which characterize the difficulty of the learning task. In order to understand
estimators derived from other loss functions, Marteau-Ferey et al. have
extended the theory of Tikhonov regularization to generalized self concordant
loss functions (GSC), which contain, e.g., the logistic loss. In this paper, we
go a step further and show that fast and optimal rates can be achieved for GSC
by using the iterated Tikhonov regularization scheme, which is intrinsically
related to the proximal point method in optimization, and overcomes the
limitation of the classical Tikhonov regularization.
