A Unifying Framework for Combining Complementary Strengths of Humans and
  ML toward Better Predictive Decision-Making
  Hybrid human-ML systems are increasingly in charge of consequential decisions
in a wide range of domains. A growing body of work has advanced our
understanding of these systems by providing empirical and theoretical analyses.
However, existing empirical results are mixed, and theoretical proposals are
often incompatible with each other. Our goal in this work is to bring
much-needed organization to this field by offering a unifying framework for
understanding conditions under which combining complementary strengths of human
and ML leads to higher quality decisions than those produced by them
individually -- a state to which we refer to as human-ML complementarity. We
focus specifically on the context of human-ML predictive decision-making
systems and investigate optimal ways of combining human and ML-based predictive
decisions, accounting for the underlying causes of variation in their
judgments. Within this scope, we present two crucial contributions. First,
drawing upon prior literature in human psychology, machine learning, and
human-computer interaction, we introduce a taxonomy characterizing a wide
variety of criteria across which human and machine decision-making differ.
Building on our taxonomy, our second contribution presents a unifying
optimization-based framework for formalizing how human and ML predictive
decisions should be aggregated optimally. We show that our proposed framework
encompasses several existing models of human-ML complementarity as special
cases. Last but not least, the exploratory analysis of our framework offers a
critical piece of insight for future work in this area: the mechanism by which
we combine human-ML judgments should be informed by the underlying causes of
their diverging decisions.
