Learning to Complement Humans
  A rising vision for AI in the open world centers on the development of
systems that can complement humans for perceptual, diagnostic, and reasoning
tasks. To date, systems aimed at complementing the skills of people have
employed models trained to be as accurate as possible in isolation. We
demonstrate how an end-to-end learning strategy can be harnessed to optimize
the combined performance of human-machine teams by considering the distinct
abilities of people and machines. The goal is to focus machine learning on
problem instances that are difficult for humans, while recognizing instances
that are difficult for the machine and seeking human input on them. We
demonstrate in two real-world domains (scientific discovery and medical
diagnosis) that human-machine teams built via these methods outperform the
individual performance of machines and people. We then analyze conditions under
which this complementarity is strongest, and which training methods amplify it.
Taken together, our work provides the first systematic investigation of how
machine learning systems can be trained to complement human reasoning.
