Robust Linear Predictions: Analyses of Uniform Concentration, Fast Rates
  and Model Misspecification
  The problem of linear predictions has been extensively studied for the past
century under pretty generalized frameworks. Recent advances in the robust
statistics literature allow us to analyze robust versions of classical linear
models through the prism of Median of Means (MoM). Combining these approaches
in a piecemeal way might lead to ad-hoc procedures, and the restricted
theoretical conclusions that underpin each individual contribution may no
longer be valid. To meet these challenges coherently, in this study, we offer a
unified robust framework that includes a broad variety of linear prediction
problems on a Hilbert space, coupled with a generic class of loss functions.
Notably, we do not require any assumptions on the distribution of the outlying
data points ($\mathcal{O}$) nor the compactness of the support of the inlying
ones ($\mathcal{I}$). Under mild conditions on the dual norm, we show that for
misspecification level $\epsilon$, these estimators achieve an error rate of
$O(\max\left\{|\mathcal{O}|^{1/2}n^{-1/2}, |\mathcal{I}|^{1/2}n^{-1}
\right\}+\epsilon)$, matching the best-known rates in literature. This rate is
slightly slower than the classical rates of $O(n^{-1/2})$, indicating that we
need to pay a price in terms of error rates to obtain robust estimates.
Additionally, we show that this rate can be improved to achieve so-called "fast
rates" under additional assumptions.
