Regression Under Human Assistance
  Decisions are increasingly taken by both humans and machine learning models.
However, machine learning models are currently trained for full automation --
they are not aware that some of the decisions may still be taken by humans. In
this paper, we take a first step towards the development of machine learning
models that are optimized to operate under different automation levels. More
specifically, we first introduce the problem of ridge regression under human
assistance and show that it is NP-hard. Then, we derive an alternative
representation of the corresponding objective function as a difference of
nondecreasing submodular functions. Building on this representation, we further
show that the objective is nondecreasing and satisfies $\alpha$-submodularity,
a recently introduced notion of approximate submodularity. These properties
allow a simple and efficient greedy algorithm to enjoy approximation guarantees
at solving the problem. Experiments on synthetic and real-world data from two
important applications -- medical diagnosis and content moderation-demonstrate
that our algorithm outsources to humans those samples in which the prediction
error of the ridge regression model would have been the highest if it had to
make a prediction, it outperforms several competitive baselines, and its
performance is robust with respect to several design choices and
hyperparameters used in the experiments.
