Non-parametric Models for Non-negative Functions
  Linear models have shown great effectiveness and flexibility in many fields
such as machine learning, signal processing and statistics. They can represent
rich spaces of functions while preserving the convexity of the optimization
problems where they are used, and are simple to evaluate, differentiate and
integrate. However, for modeling non-negative functions, which are crucial for
unsupervised learning, density estimation, or non-parametric Bayesian methods,
linear models are not applicable directly. Moreover, current state-of-the-art
models like generalized linear models either lead to non-convex optimization
problems, or cannot be easily integrated. In this paper we provide the first
model for non-negative functions which benefits from the same good properties
of linear models. In particular, we prove that it admits a representer theorem
and provide an efficient dual formulation for convex problems. We study its
representation power, showing that the resulting space of functions is strictly
richer than that of generalized linear models. Finally we extend the model and
the theoretical results to functions with outputs in convex cones. The paper is
complemented by an experimental evaluation of the model showing its
effectiveness in terms of formulation, algorithmic derivation and practical
results on the problems of density estimation, regression with heteroscedastic
errors, and multiple quantile regression.
