When is there a Representer Theorem? Reflexive Banach spaces
  We consider a general regularised interpolation problem for learning a
parameter vector from data. The well known representer theorem says that under
certain conditions on the regulariser there exists a solution in the linear
span of the data points. This is at the core of kernel methods in machine
learning as it makes the problem computationally tractable. Most literature
deals only with sufficient conditions for representer theorems in Hilbert
spaces. We prove necessary and sufficient conditions for the existence of
representer theorems in reflexive Banach spaces and illustrate why in a sense
reflexivity is the minimal requirement on the function space. We further show
that if the learning relies on the linear representer theorem, then the
solution is independent of the regulariser and in fact determined by the
function space alone. This in particular shows the value of generalising
Hilbert space learning theory to Banach spaces.
