Risk bounds when learning infinitely many response functions by ordinary
  linear regression
  Consider the problem of learning a large number of response functions
simultaneously based on the same input variables. The training data consist of
a single independent random sample of the input variables drawn from a common
distribution together with the associated responses. The input variables are
mapped into a high-dimensional linear space, called the feature space, and the
response functions are modelled as linear functionals of the mapped features,
with coefficients calibrated via ordinary least squares. We provide convergence
guarantees on the worst-case excess prediction risk by controlling the
convergence rate of the excess risk uniformly in the response function. The
dimension of the feature map is allowed to tend to infinity with the sample
size. The collection of response functions, although potentially infinite, is
supposed to have a finite Vapnik-Chervonenkis dimension. The bound derived can
be applied when building multiple surrogate models in a reasonable computing
time.
