Kernel based regression with robust loss function via iteratively
  reweighted least squares
  Least squares kernel based methods have been widely used in regression
problems due to the simple implementation and good generalization performance.
Among them, least squares support vector regression (LS-SVR) and extreme
learning machine (ELM) are popular techniques. However, the noise sensitivity
is a major bottleneck. To address this issue, a generalized loss function,
called $\ell_s$-loss, is proposed in this paper. With the support of novel loss
function, two kernel based regressors are constructed by replacing the
$\ell_2$-loss in LS-SVR and ELM with the proposed $\ell_s$-loss for better
noise robustness. Important properties of $\ell_s$-loss, including robustness,
asymmetry and asymptotic approximation behaviors, are verified theoretically.
Moreover, iteratively reweighted least squares (IRLS) is utilized to optimize
and interpret the proposed methods from a weighted viewpoint. The convergence
of the proposal are proved, and detailed analyses of robustness are given.
Experiments on both artificial and benchmark datasets confirm the validity of
the proposed methods.
