Hard Shape-Constrained Kernel Machines
  Shape constraints (such as non-negativity, monotonicity, convexity) play a
central role in a large number of applications, as they usually improve
performance for small sample size and help interpretability. However enforcing
these shape requirements in a hard fashion is an extremely challenging problem.
Classically, this task is tackled (i) in a soft way (without out-of-sample
guarantees), (ii) by specialized transformation of the variables on a
case-by-case basis, or (iii) by using highly restricted function classes, such
as polynomials or polynomial splines. In this paper, we prove that hard affine
shape constraints on function derivatives can be encoded in kernel machines
which represent one of the most flexible and powerful tools in machine learning
and statistics. Particularly, we present a tightened second-order cone
constrained reformulation, that can be readily implemented in convex solvers.
We prove performance guarantees on the solution, and demonstrate the efficiency
of the approach in joint quantile regression with applications to economics and
to the analysis of aircraft trajectories, among others.
