A Precise Performance Analysis of Support Vector Regression
  In this paper, we study the hard and soft support vector regression
techniques applied to a set of $n$ linear measurements of the form
$y_i=\boldsymbol{\beta}_\star^{T}{\bf x}_i +n_i$ where
$\boldsymbol{\beta}_\star$ is an unknown vector, $\left\{{\bf
x}_i\right\}_{i=1}^n$ are the feature vectors and
$\left\{{n}_i\right\}_{i=1}^n$ model the noise. Particularly, under some
plausible assumptions on the statistical distribution of the data, we
characterize the feasibility condition for the hard support vector regression
in the regime of high dimensions and, when feasible, derive an asymptotic
approximation for its risk. Similarly, we study the test risk for the soft
support vector regression as a function of its parameters. Our results are then
used to optimally tune the parameters intervening in the design of hard and
soft support vector regression algorithms. Based on our analysis, we illustrate
that adding more samples may be harmful to the test performance of support
vector regression, while it is always beneficial when the parameters are
optimally selected. Such a result reminds a similar phenomenon observed in
modern learning architectures according to which optimally tuned architectures
present a decreasing test performance curve with respect to the number of
samples.
