Overestimation learning with guarantees
  We describe a complete method that learns a neural network which is
guaranteed to overestimate a reference function on a given domain. The neural
network can then be used as a surrogate for the reference function. The method
involves two steps. In the first step, we construct an adaptive set of Majoring
Points. In the second step, we optimize a well-chosen neural network to
overestimate the Majoring Points. In order to extend the guarantee on the
Majoring Points to the whole domain, we necessarily have to make an assumption
on the reference function. In this study, we assume that the reference function
is monotonic. We provide experiments on synthetic and real problems. The
experiments show that the density of the Majoring Points concentrate where the
reference function varies. The learned over-estimations are both guaranteed to
overestimate the reference function and are proven empirically to provide good
approximations of it. Experiments on real data show that the method makes it
possible to use the surrogate function in embedded systems for which an
underestimation is critical; when computing the reference function requires too
many resources.
